The last pair of definitions is testingandquality assurance(sometimes shortened to QA). These two terms are the ones most often used to describe either the group or the process that’s verifying and validating the software. In Chapter 21, “Software
Quality Assurance,” you’ll learn more about software quality assurance, but for now, consider these definitions:
• The goal of a software tester is to find bugs, find them as early as possible, and make sure they get fixed.
• A software quality assurance person’s main responsibility is to create and enforce standards and methods to improve the development process and to prevent bugs from ever occurring.
Of course, there is overlap. Some testers will do a few QA tasks and some QA-ers will perform a bit of testing. The two jobs and their tasks are intertwined. What’s impor- tant is that you know what your primary job responsibilities are and communicate that information to the rest of the development team. Confusion among the team members about who’s testing and who’s not has caused lots of process pain in many projects.
Summary
Sausages, laws, and software—watching them being made can be pretty messy.
Hopefully the previous three chapters haven’t scared you off.
Many software testers have come into a project not knowing what was happening around them, how decisions were being made, or what procedure they should be following. It’s impossible to be effective that way. With the information you’ve learned so far about software testing and the software development process, you’ll have a head start when you begin testing for the first time. You’ll know what your role should be, or at least know what questions to ask to find your place in the big picture.
For now, all the process stuff is out of the way, and the next chapter of this book begins a new section that will introduce you to the basic techniques of software testing.
Quiz
These quiz questions are provided for your further understanding. See Appendix A,
“Answers to Quiz Questions,” for the answers—but don’t peek!
1. Given that it’s impossible to test a program completely, what information do you think should be considered when deciding whether it’s time to stop testing?
2. Start the Windows Calculator. Type 5,000-5=(the comma is important). Look at the result. Is this a bug? Why or why not?
3. If you were testing a simulation game such as a flight simulator or a city simu- lator, what do you think would be more important to test—its accuracy or its precision?
4. Is it possible to have a high-quality and low-reliability product? What might an example be?
5. Why is it impossible to test a program completely?
6. If you were testing a feature of your software on Monday and finding a new bug every hour, at what rate would you expect to find bugs on Tuesday?
Testing Fundamentals
Go out looking for one thing, and that’s all you’ll ever find.
—Old prospector’s saying The most exciting phrase to hear in science, the one that heralds the most discoveries, is not “Eureka!” but “That’s funny...”
—Isaac Asimov, science and science-fiction author
IN THIS PART
4 Examining the Specification
5 Testing the Software with Blinders On 6 Examining the Code
7 Testing the Software with X-Ray Glasses
•Getting Started
•Performing a High-Level Review of the Specification
•Low-Level Specification Test Techniques
Examining the Specification
T
his chapter will introduce you to your first real hands- on testing—but it may not be what you expect. You won’t be installing or running software and you won’t be pound- ing on the keyboard hoping for a crash. In this chapter, you’ll learn how to test the product’s specification to find bugs before they make it into the software.Testing the product spec isn’t something that all software testers have the luxury of doing. Sometimes you might come into a project midway through the development cycle after the specification is written and the coding started. If that’s the case, don’t worry—you can still use the techniques presented here to test the final specification.
If you’re fortunate enough to be involved on the project early and have access to a preliminary specification, this chapter is for you. Finding bugs at this stage can poten- tially save your project huge amounts of time and money.
Highlights of this chapter include
• What is black-box and white-box testing
• How static and dynamic testing differ
• What high-level techniques can be used for review- ing a product specification
• What specific problems you should look for when reviewing a product specification in detail
Getting Started
Think back to the four development models presented in Chapter 2, “The Software Development Process”: big-bang,
code-and-fix, waterfall, and spiral. In each model, except big-bang, the development team creates a product specification from the requirements documentto define what the software will become.
Typically, the product specification is a written document using words and pictures to describe the intended product. An excerpt from the Windows Calculator (see Figure 4.1) product spec might read something like this:
The Edit menu will have two selections: Copy and Paste. These can be chosen by one of three methods: pointing and clicking to the menu items with the mouse, using access-keys (Alt+E and then C for Copy and P for Paste), or using the standard Windows shortcut keys of Ctrl+C for Copy and Ctrl+V for Paste.
The Copy function will copy the current entry displayed in the number text box into the Windows Clipboard. The Paste function will paste the value stored in the Windows Clipboard into the number text box.
Edit Menu Number Text Box
FIGURE 4.1 The standard Windows Calculator displaying the drop-down Edit menu.
As you can see, it took quite a few words just to describe the operation of two menu items in a simple calculator program. A thoroughly detailed spec for the entire appli- cation could be a hundred pages long.
It may seem like overkill to create a meticulous document for such simple software.
Why not just let a programmer write a calculator program on his own? The problem is that you would have no idea what you’d eventually get. The programmer’s idea of what it should look like, what functionality it should have, and how the user would use it could be completely different from yours. The only way to assure that the end product is what the customer required—and to properly plan the test effort—is to thoroughly describe the product in a specification.
The other advantage of having a detailed spec, and the basis of this chapter, is that as a tester you’ll also have a document as a testable item. You can use it to find bugs before the first line of code is written.