Follow Experience, Intuition, and Hunches
9. True or False: It’s an unfair test to perform stress testing at the same time you perform load testing
•Static White-Box Testing:
Examining the Design and Code
•Formal Reviews
•Coding Standards and Guidelines
•Generic Code Review Checklist
Examining the Code
S
oftware testing isn’t limited to treating the specification or the program like a black box as described in Chapters 4,“Examining the Specification,” and 5, “Testing the Software with Blinders On.” If you have some program- ming experience, even if it’s just a little, you can also perform testing on the software’s architecture and code.
In some industries, such verification isn’t as common as black-box testing. However, if you’re testing military, financial, factory automation, or medical software, or if you’re lucky enough to be working in a highly disciplined development model, it may be routine to verify the product at this level. If you’re testing software for security issues, it’s imperative.
This chapter introduces you to the basics of performing verification on the design and code. As a new software tester, it may not be your first task, but it’s one that you can eventually move into if your interests lie in
programming.
Highlights from this chapter include
• The benefits of static white-box testing
• The different types of static white-box reviews
• Coding guidelines and standards
• How to generically review code for errors
Static White-Box Testing: Examining the Design and Code
Remember the definitions of static testing and white-box testing from Chapter 4? Static testingrefers to testing some- thing that isn’t running—examining and reviewing it.
White-box(or clear-box) testing implies having access to the code, being able to see it and review it.
Static white-box testingis the process of carefully and methodically reviewing the soft- ware design, architecture, or code for bugs without executing it. It’s sometimes referred to as structural analysis.
The obvious reason to perform static white-box testing is to find bugs early and to find bugs that would be difficult to uncover or isolate with dynamic black-box testing. Having a team of testers concentrate their efforts on the design of the soft- ware at this early stage of development is highly cost effective.
A side benefit of performing static white-box testing is that it gives the team’s black- box testers ideas for test cases to apply when they receive the software for testing.
They may not necessarily understand the details of the code, but by listening to the review comments they can identify feature areas that sound troublesome or bug- prone.
NOTE
Development teams vary in who has the responsibility for static white-box testing. In some teams the programmers are the ones who organize and run the reviews, inviting the software testers as independent observers. In other teams the software testers are the ones who perform this task, asking the programmer who wrote the code and a couple of his peers to assist in the reviews. Ultimately, either approach can work. It’s up to the development team to choose what works best for them.
The unfortunate thing about static white-box testing is that it’s not always done.
Many teams have the misconception that it’s too time-consuming, too costly, or not productive. All of these are untrue—compared to the alternative of testing, finding, and even not finding bugs at the back end of the project. The problem lies in the perception that a programmer’s job is to write lines of code and that any task that takes away from his efficiency of churning out those lines is slowing down the process.
Fortunately, the tide is changing. Many companies are realizing the benefits of testing early and are hiring and training their programmers and testers to perform white-box testing. It’s not rocket science (unless you’re designing rockets), but getting started requires knowing a few basic techniques. If you’re interested in taking it further, the opportunities are huge.
Formal Reviews
Aformal reviewis the process under which static white-box testing is performed. A formal review can range from a simple meeting between two programmers to a detailed, rigorous inspection of the software’s design or its code.
There are four essential elements to a formal review:
• Identify Problems.The goal of the review is to find problems with the soft- ware—not just items that are wrong, but missing items as well. All criticism should be directed at the design or code, not the person who created it.
Participants shouldn’t take any criticism personally. Leave your egos, emotions, and sensitive feelings at the door.
• Follow Rules.A fixed set of rules should be followed. They may set the amount of code to be reviewed (usually a couple hundred lines), how much time will be spent (a couple hours), what can be commented on, and so on.
This is important so that the participants know what their roles are and what they should expect. It helps the review run more smoothly.
• Prepare.Each participant is expected to prepare for and contribute to the review. Depending on the type of review, participants may have different roles.
They need to know what their duties and responsibilities are and be ready to actively fulfill them at the review. Most of the problems found through the review process are found during preparation, not at the actual review.
• Write a Report.The review group must produce a written report summariz- ing the results of the review and make that report available to the rest of the product development team. It’s imperative that others are told the results of the meeting—how many problems were found, where they were found, and so on.
What makes formal reviews work is following an established process. Haphazardly
“getting together to go over some code” isn’t sufficient and may actually be detri- mental. If a process is run in an ad-hoc fashion, bugs will be missed and the partici- pants will likely feel that the effort was a waste of time.
If the reviews are run properly, they can prove to be a great way to find bugs early.
Think of them as one of the initial nets (see Figure 6.1) that catches the big bugs at the beginning of the process. Sure, smaller bugs will still get through, but they’ll be caught in the next testing phases with the smaller nets with the tighter weave.
In addition to finding problems, holding formal reviews has a few indirect results:
• Communications.Information not contained in the formal report is communicated. For example, the black-box testers can get insight into where problems may lie. Inexperienced programmers may learn new techniques from more experienced programmers. Management may get a better feel for how the project is tracking its schedule.
• Quality.A programmer’s code that is being gone over in detail, function by function, line by line, often results in the programmer being more careful.
That’s not to say that he would otherwise be sloppy—just that if he knows that
his work is being carefully reviewed by his peers, he might make an extra effort to triple-check it to make sure that it’s right.
• Team Camaraderie.If a review is run properly, it can be a good place for testers and programmers to build respect for each other’s skills and to better understand each other’s jobs and job needs.
• Solutions.Solutions may be found for tough problems, although whether they are discussed depends on the rules for the review. It may be more effective to discuss solutions outside the review.
FIGURE 6.1 Formal reviews are the first nets used in catching bugs.
These indirect benefits shouldn’t be relied on, but they do happen. On many teams, for whatever reasons, the members end up working in isolation. Formal reviews are a great way to get them in the same room, all discussing the same project problems.