SOLUTION
SIDEBAR 1.1 TERMINOLOGY FOR DESCRIBING BUGS
O
ften, we talk about “bugs” in software, meaning many things that depend on the context.A“bug” can be a mistake in interpreting a requirement, a syntax error in a piece of code, or the (as-yet-unknown) cause of a system crash. The Institute of Electrical and Electronics Engineers (IEEE) has suggested a standard terminology (in IEEE Standard 729) for describ- ing “bugs” in our software products (IEEE 1983).
Afaultoccurs when a human makes a mistake, called an error, in performing some soft- ware activity. For example, a designer may misunderstand a requirement and create a design that does not match the actual intent of the requirements analyst and the user. This design fault is an encoding of the error, and it can lead to other faults, such as incorrect code and an incor- rect description in a user manual. Thus, a single error can generate many faults, and a fault can reside in any development or maintenance product.
Afailureis a departure from the system’s required behavior. It can be discovered before or after system delivery, during testing, or during operation and maintenance. As we will see in Chapter 4, the requirements documents can contain faults. So a failure may indicate that the system is not performing as required, even though it may be performing as specified.
Thus, a fault is an inside view of the system, as seen by the eyes of the developers, whereas a failure is an outside view: a problem that the user sees. Not every fault corresponds to a failure; for example, if faulty code is never executed or a particular state is never entered, then the fault will never cause the code to fail. Figure 1.4 shows the genesis of a failure.
?!
human error fault
failure
can lead to can lead to
FIGURE 1.4 How human error causes a failure.
Section 1.2 How Successful Have We Been? 7 In the early 1980s, the United States Internal Revenue Service (IRS) hired Sperry Corporation to build an automated federal income tax form processing system. Accord- ing to the Washington Post, the “system . . . proved inadequate to the workload, cost nearly twice what was expected and must be replaced soon” (Sawyer 1985). In 1985, an extra $90 million was needed to enhance the original $103 million worth of Sperry equip- ment. In addition, because the problem prevented the IRS from returning refunds to tax- payers by the deadline, the IRS was forced to pay $40.2 million in interest and $22.3 million in overtime wages for its employees who were trying to catch up. In 1996, the situ- ation had not improved. The Los Angeles Timesreported on March 29 that there was still no master plan for the modernization of IRS computers, only a 6000-page technical docu- ment. Congressman Jim Lightfoot called the project “a $4-billion fiasco that is flounder- ing because of inadequate planning” (Vartabedian 1996).
Situations such as these still occur. In the United States, the Federal Bureau of Investigation’s (FBI’s) Trilogy project attempted to upgrade the FBI’s computer systems.
The results were devastating: “After more than four years of hard work and half a billion dollars spent, however, Trilogy has had little impact on the FBI’s antiquated case-man- agement system, which today remains a morass of mainframe green screens and vast stores of paper records” (Knorr 2005). Similarly, in the United Kingdom, the cost of over- hauling the National Health Service’s information systems was double the original esti- mate (Ballard 2006). We will see in Chapter 2 why project planning is essential to the production of quality software.
For many years, the public accepted the infusion of software in their daily lives with little question. But President Reagan’s proposed Strategic Defense Initiative (SDI) heightened the public’s awareness of the difficulty of producing a fault-free soft- ware system. Popular newspaper and magazine reports (such as Jacky 1985; Parnas 1985, Rensburger 1985) expressed skepticism in the computer science community. And now, years later, when the U.S. Congress is asked to allocate funds to build a similar sys- tem, many computer scientists and software engineers continue to believe there is no way to write and test the software to guarantee adequate reliability.
For example, many software engineers think that an antiballistic-missile system would require at least 10 million lines of code; some estimates range as high as one hun- dred million. By comparison, the software supporting the American space shuttle consists of 3 million lines of code, including computers on the ground controlling the launch and the flight; there were 100,000 lines of code in the shuttle itself in 1985 (Rensburger 1985).
Thus, an antimissile software system would require the testing of an enormous amount of code. Moreover, the reliability constraints would be impossible to test. To see why, con- sider the notion of safety-critical software. Typically, we say that something that is safety- critical (i.e., something whose failure poses a threat to life or health) should have a reliability of at least 10–9. As we shall see in Chapter 9, this terminology means that the system can fail no more often than once in 109hours of operation. To observe this degree of reliability, we would have to run the system for at least 109hours to verify that it does not fail. But 109hours is over 114,000 years—far too long as a testing interval!
We will also see in Chapter 9 that helpful technology can become deadly when software is improperly designed or programmed. For example, the medical community was aghast when the Therac-25, a radiation therapy and X-ray machine, malfunctioned and killed several patients. The software designers had not anticipated the use of several
arrow keys in nonstandard ways; as a consequence, the software retained its high set- tings and issued a highly concentrated dose of radiation when low levels were intended (Leveson and Turner 1993).
Similar examples of unanticipated use and its dangerous consequences are easy to find. For example, recent efforts to use off-the-shelf components (as a cost savings mea- sure instead of custom-crafting of software) result in designs that use components in ways not intended by the original developers. Many licensing agreements explicitly point to the risks of unanticipated use:“Because each end-user system is customized and differs from utilized testing platforms and because a user or application designer may use the software in combination with other products in a manner not evaluated or con- templated by [the vendor] or its suppliers, the user or application designer is ultimately responsible for verifying and validating the [software]” (Lookout Directn.d.).
Unanticipated use of the system should be considered throughout software design activities. These uses can be handled in at least two ways: by stretching your imagination to think of how the system can be abused (as well as used properly), and by assuming that the system will be abused and designing the software to handle the abuses. We discuss these approaches in Chapter 8.
Although many vendors strive for zero-defect software, in fact most software products are not fault-free. Market forces encourage software developers to deliver products quickly, with little time to test thoroughly. Typically, the test team will be able to test only those functions most likely to be used, or those that are most likely to endanger or irritate users. For this reason, many users are wary of installing the first version of code, knowing that the bugs will not be worked out until the second version.
Furthermore, the modifications needed to fix known faults are sometimes so difficult to make that it is easier to rewrite a whole system than to change existing code. We will investigate the issues involved in software maintenance in Chapter 11.
In spite of some spectacular successes and the overall acceptance of software as a fact of life, there is still much room for improvement in the quality of the software we produce. For example, lack of quality can be costly; the longer a fault goes undetected, the more expensive it is to correct. In particular, the cost of correcting an error made during the initial analysis of a project is estimated to be only one-tenth the cost of cor- recting a similar error after the system has been turned over to the customer. Unfortu- nately, we do not catch most of the errors early on. Half of the cost of correcting faults found during testing and maintenance comes from errors made much earlier in the life of a system. In Chapters 12 and 13, we will look at ways to evaluate the effectiveness of our development activities and improve the processes to catch mistakes as early as possible.
One of the simple but powerful techniques we will propose is the use of review and inspection. Many students are accustomed to developing and testing software on their own. But their testing may be less effective than they think. For example, Fagan studied the way faults were detected. He discovered that testing a program by running it with test data revealed only about a fifth of the faults located during systems develop- ment. However, peer review, the process whereby colleagues examine and comment on each other’s designs and code, uncovered the remaining four out of five faults found (Fagan 1986). Thus, the quality of your software can be increased dramatically just by having your colleagues review your work. We will learn more in later chapters about how the review and inspection processes can be used after each major development
Section 1.3 What Is Good Software? 9 step to find and fix faults as early as possible. And we will see in Chapter 13 how to improve the inspection process itself.
1.3 WHAT IS GOOD SOFTWARE?
Just as manufacturers look for ways to ensure the quality of the products they produce, so too must software engineers find methods to ensure that their products are of acceptable quality and utility.Thus, good software engineering must always include a strategy for pro- ducing quality software. But before we can devise a strategy, we must understand what we mean by quality software. Sidebar 1.2 shows us how perspective influences what we mean by “quality.” In this section, we examine what distinguishes good software from bad.