As illustrated in Fig.10, the dynamic testing process includes four different activities:
• Test design and implementation;
• Test environment set-up and maintenance;
• Test execution;
• Test incident reporting.
Some activities of the dynamic testing process, in practice, might need to be revisited multiple times [25]. This process has various iterations that will show if the completion criteria defined in the test process plan have not been met. The dynamic test process will receive the test plan, and the control directives from the test man- agement processes and will send test measures for the test supervision, and control monitors test progress.
52 C. Patrício et al.
Table 1 Tasks and activities from the test planning process
Activity Tasks
Understand context (a) Understand the context and the software testing requirements by identifying and interacting with the relevant stakeholders (b) Initiate a communication plan Organize test plan development (a) Identify and schedule the activities that
need to be performed to complete test planning
(b) Identify the stakeholders required (c) Obtain the approval from the relevant stakeholders of the activities, schedules, and participants
(d) Organize stakeholder involvement Identify and analyse risks (a) Review previously identified risks
(b) Identify additional risks (c) Classify risks
(d) Assign levels of exposure to each risk (e) Obtain approval for the results of the risk assessments from stakeholders
(f) Record the results of the risk assessment Identify risk mitigation approaches (a) Identify the appropriate means of treating
the risk
(b) Record the results of the risk mitigation Design test strategy (a) Produce initial estimate of resources
required, defined by the organizational test strategy and policy
(b) Produce an initial assessment of the resources needed to perform the individual mitigation actions
(c) Design a test strategy
(d) Recognize metrics to be used for test monitoring and control
(e) Recognize test information
(f) Identify test environment conditions and test tool requirements
(g) Identify test deliverables
(h) Produce an initial estimate of the required resources to perform the complete set of activities from the test strategy (i) Record the test strategy
(j) Obtain the approval on the test strategy from the stakeholders
Determine staffing and scheduling (a) Identify the roles and skills of staff (b) Schedule each required test activity (c) Obtain approval on staffing scheduling from relevant stakeholders
(continued)
A Study on Software Testing Standard Using ISO/IEC/IEEE … 53
Table 1 (continued)
Activity Tasks
Record test plan (a) Calculate final estimates for the testing (b) Incorporate the test strategy, staffing profile, and schedule in the test plan
Gain consensus on the test plan (a) Gather the views of the stakeholders on the test plan
(b) Resolve conflicts between the test plan and stakeholders’ views
(c) Update test plan to take into account the feedback from stakeholders
(d) Obtain approval on the test plan from the stakeholders
Make the test plan available and communicate (a) Make available the test plan (b) Communicate to the stakeholders the availability of the test plan
Fig. 8 Test control and monitoring process
6.1 Test Design and Implementation Process
Test design and implementation processes are used to document test conditions, test cases and test procedures for execution in the test execution process [12]. This method is iterative because it usually is revisited various times throughout the test project.
54 C. Patrício et al.
Table 2 Tasks and activities from the test monitoring and control process Activity Tasks
Set-up (a) Identify a suitable measure for monitoring progress
(b) Identify appropriate means for identifying new and changing risks (c) Start monitoring activities (test status reporting and test metrics collection) Monitor (a) Record and collect test measures
(b) Start monitoring the progress against the test plan
(c) Identify and record divergences from planned testing activities (d) Identify and analyze new risks
(e) Start monitoring and identifying the changes to know risks Control (a) Perform the necessary actions to implement the test plan
(b) Perform the steps required to achieve control directives from higher-level management processes
(c) Identify the actions needed to manage the divergence of planned testing to actual testing
(d) Identify means of treating for newly identified and changed risks
(e) Issue control directives to change the way measurement is performed, any change to the test plan should be in the form of test plan updates, communicate
recommended changes to the relevant stakeholders
(f) Establish each selected test action before starting that exercise (g) Obtain approval for completion of assigned test activities
(h) Obtain consent for the test completion when the test has met is completion criteria Report (a) Communicate any test progress against the test plan to stakeholders in a test
status report during the reporting period
(b) Update the new risks and changes to existing uncertainty in the risk register and communicate them with the relevant stakeholders
Fig. 9 Test completion process
Figure11illustrates a graphic overview of the various activities of the test design and implementation process. Table4gives further details about the multiple tasks in need of completion for each action.
6.2 Test Environment Set-up and Maintenance Process
The test environment set-up and maintenance processes are needed to create an environment in which a test is executed and to maintain the said environment [12].
This process is also required to support interaction with all relevant stakeholders
A Study on Software Testing Standard Using ISO/IEC/IEEE … 55
Table 3 Tasks and activities from the test completion process
Activity Tasks
Archive test assets (a) Identify and make available the test assets using appropriate means
(b) Identify and archive the test assets that may be reused on other projects
(c) Record the availability of reusable test assets in the test completion report and communicate them to the relevant stakeholders
Clean up the test environment (a) Restore the test environment to a predefined state upon conclusion of all testing actions
Identify lessons learned (a) Record the lessons learned during the project execution (what went well and not so well)
(b) Record the outcome in the test completion report and communicate with the relevant stakeholders
Report test completion (a) Collect relevant information from, but not limited to test plans, test results, test status reports, test completion; reports and incident reports
(b) Evaluate and summarize the collected information (c) Obtain approval for the test completion report from the responsible stakeholders
(d) Distribute the approved test completion report to the relevant stakeholders
Fig. 10 Dynamic test processes activities
56 C. Patrício et al.
Fig. 11 Test design and implementation process activities
about the status of the test environment. Figure12illustrates a graphic overview of the various activities of the test design and implementation process. Table5gives further details about the multiple tasks in need of completion for each action.
6.3 Test Execution Process
The test execution process is applied to manage the test methods that are created as a consequence of the development process. The test execution demands running numerous times, considering all feasible test procedures which cannot be performed a specific iteration. Moreover, if a problem is solved, the test execution must be conducted again [12]. The actions included in this workflow are the test procedure execution, test results comparison, and test execution registration [1]. Figure 13 illustrates the overview of the test execution process graphically. Table6summarizes the actions and tasks of this method.
6.4 Test Incident Reporting Process
The testing incident reporting process is applied for examination occurrence report as a consequence of failure exposure, items among accidental or uncommon perfor- mance though performing the test or in the event of retest [9]. The actions required are: examining test sequence and design or update incident outcomes [12]. These activities are illustrated in Fig.14and described in Table7.