A smoke test is a group of test cases that establish that the system is stable and all major functionality is present and works under "normal" conditions. The purpose of a smoke test is
not to find bugs in the software, although you might, but rather to let the system test team know what their starting point is. It also provides a goal for the developers and lets them know when they have achieved a degree of stability.
Key Point
A smoke test is a group of test cases that establish that the system is stable and all major functionality is present and works under "normal" conditions.
The smoke test cannot be created autonomously by the testing team, but should be done jointly or at least with the consent of the developers. Otherwise, the developers will feel like the testers are dictating to them how to do their job, and no buy-in will be achieved for the smoke test. Then, all you'll have is a smoke test that may or may not work, and the
developer will say, "So what?" Remember that buy-in is the key to success.
The trick to establishing a good smoke test is to create a group of tests that are broad in scope, as opposed to depth. Remember that the purpose is to demonstrate stability, not to find every bug in the system. We like to take our smoke test cases from the regression test set, so the smoke test becomes a subset of the regression test set. We also like to target the smoke tests as the very first tests that we attempt to automate.
It doesn't really matter if the smoke test is run by the developers or testers, but it should be run in the system test environment. The developer may want to try it in his or her
environment first, but you must remember that just because it works in the development environment doesn't mean it will work in the system test environment. The test manager might want to make the test environment available to the developers, so they can run the test and have some confidence that it works prior to promoting the code to the test team.
Team-Fly
Team-Fly
Integration Testing
Integration testing is the level of test done to ensure that the various components of a system interact and pass data correctly among one another and function cohesively.
Integration testing can be accomplished at various levels. At the lowest level, integration testing is normally done by the development group to ensure that the units work properly together. Higher levels of integration or builds may also be done by the developer or by the test team. Integration testing is the process of examining how the pieces of a system work together, especially at the interfaces. Integration can occur at various levels, as illustrated in the car example in Figure 4-13.
Figure 4-13: Levels of Integration in a Typical Car
Key Point
Integration testing is also known as string, thread, build, subsystem, and by a multitude of other names.
Several units (e.g., U1, U2, U3) are used to make the fuel module, the fuel and the air module make the carburetor, the carburetor and other parts are used to make the engine, and so on. The levels of integration continue up the chain until the entire car is assembled.
Different people might do different levels of integration testing. What you call "integration" is dependent on what you call a "system." Consider the example in Figure 4-13. If you define the system as the car, then everything below the car is integration that leads to the ultimate system. If the engine is seen as the system, then all of the parts leading to the engine are part of the integration testing. The process of testing the engine with the transmission
becomes a systems integration task or a job of testing the interface between two systems.
The decision of "what is a system" was described in the Introduction (Scope) section of the master test plan and may be affected by the structure of the organization, politics, staffing, and other factors.
Audience Analysis
In most organizations, the integration test plan is used primarily by developers as a "road map" to test how all the individual parts of a system fit together. The test group also uses this plan to make sure there's no overlap between what they're testing and what the
developers or other test groups are testing.
Many companies that we've visited have never done integration testing per se and frequently deliver well-tested systems. In this case the integration testing is probably done as part of an extended unit testing effort or is accomplished as part of system testing. Integration testing can become very important, especially on larger, more complex systems, and in particular on new development. Some bugs that can be discovered as the units are integrated are impossible to find when testing isolated units. The interfaces between modules are the target of integration testing and are normally error-prone parts of the system.
Activity Timing
As always, we're anxious to start the test plan as soon as possible. Normally, this would mean that the integration test planning process could begin as soon as the design is beginning to stabilize.
NoteWhen should the Integration Test Plan be written?
Developers play an important part in the development of the integration test plan because they'll eventually be required to participate in the testing. We have always felt that the developers are the most logical people to do the integration testing, since the development team really doesn't know if they have created a viable system without doing at least some integration testing. Having the developers do the integration testing also speeds the fixing of any bugs that are discovered.
NoteWho should write the Integration Test Plan and who should do the testing?
Finally, in order to do integration testing, it is frequently necessary to create scaffolding code (stubs and drivers). The members of the test team may or may not have the skill set necessary to create this scaffolding code.
Key Point
According to Glenford Myers' The Art of Software Testing, scaffolding is code that simulates the function of non-existent components.
Drivers are modules that simulate high-level components, while stubs are modules that simulate low-level components.
One of the key issues in integration testing is determining which modules to create and integrate first. It's very desirable to integrate the riskiest components or those that are on the critical path (drivers) first, so that we can discover as early as possible if there are any major problems. There's nothing worse than discovering a fatal flaw in one of the most critical components late in the lifecycle. Another reason why the developers are often called upon to do the integration test is because the strategy of how to conduct the integration testing is controlled almost entirely by the strategy for integrating the system, which is done by the developer.
Since systems are frequently built by iteratively combining larger and larger pieces of code (and underlying functionality), there may be various levels of integration tests. The earliest levels are often used to just check the interaction of units. In many companies this is
accomplished in a fairly informal fashion and is often seen as just an extension of unit testing. Integration testing is often done by individual developers working together to see how their code works. At some point, entire modules or subsystems need to be built. This is also often known as integration or build testing. Logically, build testing should be done by the developers or, in some cases, the test group.
NoteWhat do you mean by different levels of integration testing?
Sources of Information
Typically, the integration tests are built based upon the detailed design specifications. Higher levels of integration testing may also use the high-level or architectural design specifications.
Finally, even in integration testing, an effort should be made to match the software design and its corresponding test cases to the requirements.
Integration Test Planning Issues
Many strategic issues should be considered before deciding what type of integration testing to perform:
What modules or objects should be assembled and tested as a group?
What are the functional subassemblies?
What are the critical features?
How much testing is appropriate?
Are there any implementation-based testing objectives?
How much scaffolding code or test support code is required?
How will problems be isolated?
How is testing coordinated with system and unit testing?
Figure 4-14 illustrates a sample project containing a group of interacting builds. The shaded boxes represent critical-path components that must pass integration testing before the Make Reservation function can move to the system testing phase.
Figure 4-14: Sample Build Scheme
If any of the components A, C, K, or N are incomplete, the Make Reservation function cannot be executed without including stubs or drivers.
Configuration Management
At the earliest stages of integration testing, that is to say, as the first units are being integrated, configuration management is normally done in a fairly informal fashion with the developers and/or program leads handling most of the duty. As the builds get progressively larger, the configuration management should become more formal, as changes to the
system can potentially affect the coding and testing of other developers and testers.