in mind, has the largest potential for reducing defects in software under develop- ment. The primary goal of static testing is defect reduction in the software by reduc- ing defects in the documentation from which the software is developed. A secondary goal is correct operation of the software. This statement may seem like a statement of the obvious, but many software systems and products have been abandoned after
•
•
•
Chapter 6
Static Testing
Software Testing: Testing Across the Entire Software Development Life Cycle, by G. D. Everett and R. McLeod, Jr.
Copyright © 2007 John Wiley & Sons, Inc.
an investment of millions of dollars because the end users could not fi gure out how to properly operate the software to do their routine business. Human nature being what it is, the recipients of poor documentation and training do not say, “I will fi gure out how to operate this software”; rather, they say, “this software doesn’t do what I need it to do, so I want the old system back.”
6.3 CANDIDATE DOCUMENTS FOR STATIC TESTING
It is time to take a look at a reasonably representative list of software development document candidates for static testing.
Software development manager’s documents (a) Software requirements
(b) Software project plans Software developers’ documents (a) Use cases
(b) Software designs (c) Software specifi cations (d) Data fl ow diagrams (e) Database and fi le designs
(f) Online operating environment specifi cations (g) Batch operating environment specifi cations (h) Interfaces
(i) Connectivity (network) specifi cations (j) Security specifi cations
(k) Screen/window/page specifi cations (l) Report specifi cations
(m) Code
Testers’ documentation (a) Test plans
(b) Test cases
(c) Test environment specifi cations (d) Test data sources and preparation (e) Test tool installation and operation Administrator’s documentation (a) Installation guides
(b) Operation/administration guides End users’ documentation
(a) Users guides (b) Help screens (c) Training manuals 1.
2.
3.
4.
5.
6.3.1 Software Development Manager’s Documents
Software requirements are the foundation on which all subsequent documentation and code are written. If these critical documents are either missing (see Chapter 1’s game of “Gossip”) or incorrect, then no programming staff can deliver the correct soft- ware. The format for requirements varies widely from company to company. If your company is new to writing requirements for your software, consider starting with the software requirements templates found in the Rational Software Development Tool RequisitePro. [23] The templates in this tool enjoy a good industry reputation. So why not use recognized quality templates until your company has gained enough of its own experience to customize the requirements templates where appropriate … if you really fi nd the need to customize.
Software project plans contain the roadmap that the software development will follow for this particular system or product. If your software development projects always come in on time and under budget, then perhaps you do not need to static test this document. If your software development projects tend to be late or over budget, then perhaps a little static testing can help identify some of the planning gaps that contribute to those overruns.
6.3.2 Software Developers’ Documents
Use cases are the formal documentation of the business processes that the new system must provide. This documentation identifi es the actors (different roles the system users play) and activities (processes that are performed by specifi c actors). There is no industry consensus about who should write these use cases or how much process detail the use cases need to express. Because the develop- ment managers typically do not write use cases and the test teams typically base part of their test plans on use cases, it seems expedient to place the authorship of use cases somewhere in the development team’s realm with or without business assistance.
Software designs contain the high-level technical solution (the how) for the business requirements (the what) that are expressed in part by use cases. It is the what tohow linkage that is most often incomplete, leading to missed, or poorly understood features and capabilities. All the software developer’s documents below software designs, that is, software specifi cations, data fl ow diagrams, database and fi le designs, online operating environment specifi cations, batch operating environment specifi - cations, interfaces, connectivity (network) specifi cations, security specifi cations, screen/window/page specifi cations, report specifi cations, and code are successively deeper details of the technical implementation of the business requirements. Not all documents are needed for every system or application. Part of the software develop- ment manager’s job is to identify which of these documents need to be written for the new project. There are two static testing concerns that run through this list of more detailed documents. The fi rst static testing concern is completeness, that is, the more detailed documents must completely describe the next level of technology needed to 6.3 Candidate Documents for Static Testing 95
fully respond to the higher level documents. The second static testing concern is that as the technical details are written, some of the details stray from or are not really a part of the original business requirements (out of scope).
6.3.3 Testers’ Documentation
Tester’s documentation (test plans, test cases, test environment specifi cations, test data sources and preparation, and test tool installation and operation) needs to be tested as thoroughly as other project documents. The sequence of development events that allow test plans to be developed were covered in the bathtub example in Chapter 5. The static testing concern is similar to the more detailed developer documentation: completeness of testing plans against the business requirements and intended development with no gratuitous testing.
6.3.4 Administrator’s Documentation
Administrators’ documentation (installation guides and operation/administration guides) documents the steps that must be taken by technical staff to install and sup- port the new system or application. The static testing concern in these documents is double-sided. The fi rst side is clear and unequivocal support of the business require- ments. The second side is stepwise instructions that can be carried out successfully by the support staff at their expected administrative skill level.
6.3.5 End Users’ Documentation
End users’ documentation (users guides, help screens, and training manuals) docu- ments the steps that must be taken by business end users to perform their routine daily activities on the new system or application. As with the administrative manu- als, the static testing concern with these documents is the double-sided business requirements/operations within expected end user business skill level.
6.4 STATIC TESTING TECHNIQUES
Because static testing requires a high degree of interaction among software develop- ment team members, the most challenging aspect of static testing is to keep the test- ing activities objective and impersonal. The force working against you is the human tendency to take ownership of documents to the extent that a perceived attack on the document is a perceived personal attack on the owner-author. Document owner- ship is, in fact, a good thing. It causes the author to do his or her best job and gain a sense of pride in good-quality results. Static testing strives to help the owner-author produce the best possible (most correct) document by helping identify corrections and improvements to documents that might already be of good quality. This pride of
ownership reaches its zenith with program code. Because no large software applica- tion is 100% defect free, all program code has room for improvement. Static testing is one of the more cost-effective ways to identify these improvements.
Consider using a two-step approach to static testing. For the fi rst step, clean up the cosmetic appearance of the document: check spelling, check grammar, check punctuation, and check formatting. The benefi t of doing the fi rst step is that when the document is cosmetically clean, the readers can concentrate on the content. The liability of skipping the fi rst step is that if the document is not cosmetically clean, the readers will surely stop reading the document for meaning and start proofreading—
to the detriment of content review.
For the second step, use whatever techniques seem appropriate to focus expert review on document contents. Here are some of the more popular and effective tech- niques used for content review.
Static testing techniques for content review desk checking
inspections walk-throughs
Desk checking is the least formal and least time-consuming static testing tech- nique. Of all the techniques, desk checking is the only one whereby the author is en- couraged to test his or her own document. The remaining techniques rely on indepen- dent eyes to provide a more thorough and objective review. Desk checking involves fi rst running a spellchecker, grammar checker, syntax checker, or whatever tools are available to clean up the cosmetic appearance of the document. Then, the author slowly reviews the document trying to look for inconsistencies, incompleteness, and missing information. Problems detected in the contents should be corrected directly by the au- thor with the possible advice of the project manager and other experts on the project.
Once all corrections are made, the cosmetic testing is rerun to catch and correct all spelling, grammar, and punctuation errors introduced by the content corrections.
Inspections are a little more formal and a little more time consuming than desk checking. The technique also fi nds more document defects than desk checking. The intent of the technique is for an independent pair of eyes, usually a more senior member of the team, to read the document and discover content problems. As rec- ommended with desk checking, the document to be inspected should be made as cosmetically clean as possible by the author so that the independent reader(s) can focus on the content. The independent reader then takes the document elsewhere and reviews it. Separating the document from the author allows the document to stand on its own merit. If the reviewer inspects the document in front of the author, the human tendency is for the author to kibitz the reviewer, which defeats the purpose of the independent reviewer. Suspected problems in the content should be documented by the independent reviewer and presented to the author in a subsequent meeting.
The author then needs to provide suggested corrective action alongside the suspected problem. The project manager or someone senior on the project should then review
•
•
•
6.4 Static Testing Techniques 97
the list of reviewer’s suspected problems, and author’s suggest corrective actions in order to negotiate an agreed corrective action.
Walk-throughs are the most formal and most time-consuming document test- ing techniques, but they are the most effective at identifying content problems. The walk-through is a scheduled meeting with a facilitator, the document author, and an audience of senior technical staff and possibly business staff. The author must scrub the document for cosmetic errors and send the document to all participants in advance of the meeting. The participants read the document and formulate ques- tions about the document contents based on their own knowledge of the new system or application. At the appointed time, the author presents his or her document to the walk-through meeting. The facilitator becomes the clearinghouse for questions by the audience and answers by the author. The facilitator also ensures that the ques- tions are posed in an objective, nonthreatening way. The walk-through facilitator documents all suspected content problems and author responses for later resolution by the project manager in a manner similar to the inspection resolutions.
6.5 TRACKING DEFECTS DETECTED BY STATIC TESTING
Each of the previously described static testing techniques involved writing down sus- pected or confi rmed content problems, their suggested or negotiated corrective action, and a correction completion date. These lists can be managed using simple tools like spread- sheets or complex tools like databases. Either way, there are two compelling reasons for recording and tracking these defects to correction. The fi rst reason is to enable the project management to verify the corrections are actually applied to the tested document in a timely manner. The second reason is to demonstrate the importance of current, correct documentation to the early success of the project. Without defect tracking as a reminder to get the documentation right, the attitude of “I’ll document that after I’ve written the code”
will rise up and make Capers Jones’ gloomy prediction a reality for your project too.
6.6 PUTTING STATIC TESTING IN PERSPECTIVE
Static testing is one of the most frequently overlooked testing approaches. The mis- taken impression of static testing is that it takes up time and effort that could be better spent on designing the system or writing the software. In fact, a few well spent hours static testing the development project documentation as it is written could save the project hundreds of hours correcting or redesigning the code caused by poor documentation in the fi rst place.
KEY TERMS AND CONCEPTS Static testing
Desk checks
Inspections Presentations
99 LEARNING OBJECTIVES
to examine the benefits of use case driven functional testing to extend the effectiveness of functional testing by regression testing to discover the depth of functional testing afforded by white box testing
to discover the depth of functional testing afforded by black box testing and how it differs from white box testing
7.1 INTRODUCTION
Recalling our SPRAE testing approach in Chapter 3, the preparation for and execution of tests to validate software behavior are critical next steps in premeditation. The planning part of premeditation was accomplished in Chapters 4 and 5. Specifi cation was accomplished in Chapter 6. The remainder of premeditation will be accomplished in Chapters 7–9.
Starting with this chapter, the next three chapters discuss testing techniques that validate the behavior of software as the software matures. Behavior in this context means that a tester provides an input to the software and observes the software’s behavior (output). To make the software behave in an expected manner, the tester must execute the software code in a controlled testing environment. These execution testing techniques are fundamentally different in objective and approach from static testing where no code is executed.
The objective of functional testing is to validate the software behavior against the business functionality documented in the software requirements and specifi cations. Business functionality is generally defi ned as those activities that support routine daily business. Functional testing is achieved by a series of tests that exercise increasingly more of the software that directly enables users to accomplish this routine daily business.
•
•
•
•