Figure 1-3 Overview of the testing environment.
Software testing organizations can use the results of this assessment in any one of the following three ways:
1. To determine their current testing environmental status versus the environment of a world-class testing organization. The responses to the items address will indicate an organization’s strengths and weaknesses compared to the environ- ment of a world-class testing organization.
2. To develop the goal/objectives to accomplish becoming a world-class testing organization. QAI’s world-class criteria indicate a profile of the environment of a world-class testing organization. Achieving those objectives can lead you to become a more effective software testing organization.
3. To develop an improvement plan.
By doing the assessment, you will develop a Footprint Chart that shows where improvement is needed. Those criteria in which you are deficient become the means for improving the environment of your software testing organization.
Implementation Procedures
This practice involves the following four tasks:
■■ Build the assessment team.
■■ Complete the assessment questionnaires.
■■ Build the footprint chart.
■■ Assess the results.
DRIVERS OF WORLD-CLASS TESTING DESIRED RESULTS
Test Tools
Use of Test
Processes Mana
gem ent Suppo
rt Test E
nvi ronm
ent Plann
ing
Test T raining Use
r Satisfactio n
Test Me asurement Quality Co
ntrol
World-Class Testing
Building the Assessment Team
The assessment team should combine people who in totality possess the knowledge of how your organization manages software testing. Before the team is established, the areas to address should be reviewed to determine the makeup of the team. It is recom- mended that a matrix be prepared with the seven assessment criteria on one dimension and the recommended assessment team on the other. The matrix should indicate which assessment team member is knowledgeable about each of the seven assessment criteria.
Once all seven criteria have been associated with an assessment team member, it can be concluded that the team is adequate to perform the assessment.
Completing the Assessment Questionnaire
The assessment questionnaire in Work Paper 1-1 consists of eight categories, with five items to address for each category. A Yes or No response should be made, as follows:
■■ A Yes response means all of the following:
■■ Criteria items are documented and in place.
■■ Criteria items are understood by testers.
■■ Criteria items are widely used, where applicable.
■■ Criteria items have produced some possible results.
■■ A No response means any of the following:
■■ No formal item in place.
■■ Criteria items are applied differently for different test situations.
■■ No consistency as to when used or used very seldom.
■■ No tangible results were produced.
The assessment team should read aloud each item and then discuss how that item is addressed in their testing environment. The results should be recorded on Work Paper 1-1. The assessment team may also wish to record comments that clarify the response and/or to provide insight in how that area may be improved.
Building the Footprint Chart
For this task, you should transcribe the results of Work Paper 1-1 onto Work Paper 1-2.
To do so, total the number of Yes responses for each criterion. Then place a dot on Work Paper 1-2 on the line representing the number of Yes responses. For example, if you have three Yes responses for test training, you should place a dot on the test training line at the intersection of the line representing three Yes responses. A dot should be marked on the line representing all seven criteria for the number of Yes responses. Then connect the dots with a line, resulting in what is called a “footprint” of the status of your testing environ- ment versus the environment of a world-class testing organization.
Assessing the Results
You should make the following two assessments regarding the footprint developed on the Work Paper 1-2:
1. Assess the status of each criteria versus what that criteria should be in the world-class testing environment.To do this, you need to look at the number of Yes responses you have recorded for each criterion versus a world-class organi- zation, which would have five Yes responses. For example, three Yes responses for test training would indicate that improvements could be made in your test training process. The two items that received No responses are indications of where improvements are needed to move your test training activities to a world-class level.
2. Interpret your testing environment footprint chart.The footprint in your Work Paper 1-2 provides an overview of your testing environment. Given the footprint, your assessment team should attempt to draw some conclusions about your testing environment. Three examples are given to help in drawing these conclusions, as shown in Figures 1-4, 1-5, and 1-6.
Figure 1-4 Example of a software testing organization using a test as a part of development.
Test Environment Planning
Test Training Use of Test Processes
User Satisfaction with Test
Test Tools Test Quality
Control
Management Support for Test
Test Measurement 5
4 3
2 1
Figure 1-5 Example of a testing organization using, but not enforcing, the test process.
Figure 1-6 Example of a testing organization practicing testing as an art.
Test Environment Planning
Test Training Use of Test Processes
User Satisfaction with Test
Test Tools Test Quality
Control
Management Support for Test
Test Measurement 5
4 3
2 1 Test Environment Planning
Test Training Use of Test Processes
User Satisfaction with Test
Test Tools Test Quality
Control
Management Support for Test
Test Measurement 5
4 3
2 1
Verifying the Assessment
The following list of questions, if responded to positively, would indicate that the assessment has been performed correctly:
1. Does the assessment team comprise the knowledge needed to answer all of the items to address within the seven criteria?
2. Are the individual assessors free from any bias that would cause them not to provide proper responses to the items to address?
3. Was there general consensus among the assessment team to the response for each item to address?
4. Are the items to address appropriate for your testing organization?
5. Have the items to address been properly totaled and posted to the Footprint Chart Work Paper?
6. Does the assessment team believe the Footprint Chart is representative of your testing environment?
7. Does your assessment team believe that if they improve the items to address, which have No responses, the testing organization will become more effective?
8. Does your organization believe that the overall assessment made is representa- tive of your environment?