Condition Coverage
6. True or False: Always design your black-box test cases first
7. Of the three code coverage measures described, which one is the best? Why?
8. What’s the biggest problem of white-box testing, either static or dynamic?
Applying Your Testing Skills
For my birthday I got a humidifier and a dehumidifier…
I put them in the same room and let them fight it out.
—Steven Wright, comedian Discovery consists of looking at the same thing as every- one else does and thinking something different.
—Albert Szent-Gyorgyi, 1937 Nobel Prize winner in physiology and medicine
IN THIS PART
8 Configuration Testing 9 Compatibility Testing 10 Foreign-Language Testing 11 Usability Testing
12 Testing the Documentation 13 Testing for Software Security 14 Website Testing
•An Overview of Configuration Testing
•Approaching the Task
•Obtaining the Hardware
•Identifying Hardware Standards
•Configuration Testing Other Hardware
Configuration Testing
L
ife could be so simple. All computer hardware could be identical. All software could be written by the same company. There wouldn’t be confusing option buttons to click or check boxes to check. Everything would interface perfectly the first time, every time. How boring.In the real world, 50,000-square-foot computer superstores are offering PCs, printers, monitors, network cards, modems, scanners, digital cameras, peripherals, net-cams, and hundreds of other computer doodads from thousands of companies—all able to connect to your PC!
If you’re just getting started at software testing, one of your first tasks may be configuration testing. You’ll be making sure that your software works with as many differ- ent hardware combinations as possible. If you’re not testing software for a popular hardware platform such as a PC or a Mac—that is, if you’re testing some specialized proprietary system—you will still need to consider configu- ration issues. You can easily tailor what you learn in this chapter to your situation.
The first part of this chapter deals with the generalities of PC configuration testing and then moves into the specifics of testing printers, display adapters (video cards), and sound cards for a PC. Although the examples are based on desktop computers, you can extrapolate the methods to just about any type of configuration test problem. New and different devices are released every day, and it will be your job to figure out how to test them.
Highlights of this chapter include
• Why configuration testing is necessary
• Why configuration testing can be a huge job
• A basic approach to configuration testing
• How to find the hardware you need to test with
• What to do if you’re not testing software for a desktop computer
An Overview of Configuration Testing
The next time you’re in one of those computer superstores, look at a few software boxes and read over the system requirements. You’ll see things such as PC with a Pentium 4 processor, 1024x768 32-bit color monitor, 32-bit audio card, game port, and so on. Configuration testingis the process of checking the operation of the soft- ware you’re testing with all these various types of hardware. Consider the different configuration possibilities for a standard Windows-based PC used in homes and businesses:
• The PC.There are several well-known computer manufacturers, such as Dell, Gateway, Hewlett Packard, and others. Each one builds PCs using components designed themselves or obtained from other manufacturers. Many hobbyists even build their own PCs using off-the-shelf components available at computer superstores.
• Components.Most PCs are modular and built up from various system boards, component cards, and other internal devices such as disk drives, CD-ROM drives, DVD burners, video, sound, fax modem, and network cards (see Figure 8.1).
There are TV tuner cards and specialized cards for video capture and home automation. There are even input/output cards that can give a PC the ability to control a small factory! These internal devices are built by hundreds of differ- ent manufacturers.
• Peripherals.Peripherals, shown in Figure 8.2, are the printers, scanners, mice, keyboards, monitors, cameras, joysticks, and other devices that plug into your system and operate externally to the PC.
• Interfaces.The components and peripherals plug into your PC through various types of interface connectors (see Figure 8.3). These interfaces can be internal or external to the PC. Typical names for them are ISA, PCI, USB, PS/2, RS/232, RJ-11, RJ-45, and Firewire. There are so many different possibilities that hardware manufacturers will often create the same peripheral with different interfaces. It’s possible to buy the exact same mouse in three different configurations!
FIGURE 8.1 Numerous internal components make up a PC’s configuration.
• Options and memory.Many components and peripherals can be purchased with different hardware options and memory sizes. Printers can be upgraded to support extra fonts or accept more memory to speed up printing. Graphics cards with more memory can support additional colors and higher resolutions.
The system board can have different versions of BIOS (its Basic Input Output System) and, of course, various amounts of memory.
CD-ROM DRIVE
POWER SUPPLY SYSTEM BOARD
HARD-DISK DRIVE FLOPPY-DISK DRIVE
VIDEO ADAPTER I/O ADAPTER
•••••••••••••••••••••••••••••••••••• •••••••••••••••••••••••••••••••••••• •••••••••••••••••••••••••••••••••••• •••••••• •••••••••••••••• •••••••• •••••••••••••••••••••••••••••••••••• •••••••••••••••••••••••••••••••••••• •••••••••••••••• •••••••••••••••• •••••••• ••••••••
SIGNAL CABLES
SYSTEM UNIT
FIGURE 8.2 A PC can connect to a wide assortment of peripherals.
• Device Drivers.All components and peripherals communicate with the oper- ating system and the software applications through low-level software called device drivers. These drivers are often provided by the hardware device manufac- turer and are installed when you set up the hardware. Although technically they are software, for testing purposes they are considered part of the hardware configuration.
If you’re a tester gearing up to start configuration testing on a piece of software, you need to consider which of these configuration areas would be most closely tied to the program. A highly graphical computer game will require lots of attention to the video and sound areas. A greeting card program will be especially vulnerable to printer issues. A fax or communications program will need to be tested with numer- ous modems and network configurations.
You may be wondering why this is all necessary. After all, there are standards to meet for building hardware, whether it’s for an off-the-shelf PC or a specialized computer in a hospital. You would expect that if everyone designed their hardware to a set of standards, software would just work with it without any problems. In an ideal world, that would happen, but unfortunately, standards aren’t always followed. Sometimes, the standards are fairly loose—call them guidelines. Card and peripheral manufactur- ers are always in tight competition with one another and frequently bend the rules to squeeze in an extra feature or to get in a last little bit of performance gain. Often the device drivers are rushed and packed into the box as the hardware goes out the door. The result is software that doesn’t work correctly with certain hardware configurations.
scanner
digital camera printer
game joystick
mouse system unit monitor
keyboard
FIGURE 8.3 The back of a PC shows numerous interface connectors for attaching peripherals.