10.2 The Abstract Test Execution Approach
10.2.4 Evaluation of Path Space
As soon as the paths for a segment are determined, the evaluation of the path space Plim starts with the data flow and the control flow. To ensure that the code fragments contained in a path can be evaluated correctly, the associated control flow makes sense and the specified endpoint of the System Model is reached.
In the first step, the data flow analysis is executed per path, where a wide variety of faults can occur. The set of possible faults is defined as D ∶= {d1,d2,d3,d4,d5,d6}, whereby the possible fault types are represented in the following list.
d1 The code fragment of the considered node could not be solved (e.g. Verification Node originated in the Test Model)
d2 The guard of the considered transition could not be solved
d3 The code fragment of a node or a transition contains several undeclared or unini- tialized variable(s)
d4 The respective endpoint could not be determined
d5 The guard of the considered transition contains a/several time-dependent vari- able(s) (e.g. guard conditions regarding the system time)
d6 The guard of the considered transition is solved by an/several over assigned vari- able(s)
By mixing concepts from the System Model and the Test Model, the subsequent analysis does not distinguish between the domains of the model elements, but only between Variable Verifying Instructions (VVIs) and Variable Modifying Instructions (VMIs), i.e., model elements that check the system state VVI and those that change the system state VMI. For example, a VVI is considered non-solvable unless the boolean expression can be evaluated totrue in the current system state. The analysis of such a segment can be represented in pseudo-code, as shown in its basic form in [147] and illustrated in algorithm section 10.2.4 in an extended version.
The functions applied during analysis are presented in a more structured way in al- gorithm 3. The method EVALSEGMENT(s), called for one segment s of the test case at a time, represents the starting point. Here, either the variable assignments are updated for the model elements to be evaluated (see STOREVALUESOFELEMENT) or a boolean expression is evaluated based on the current variable assignment (see
VERIFYELEMENT). Finally, the PERSISTLASTSTOREDVARIABLEVALUES function en-
sures that the overassignment mechanism implemented within the segment does not affect subsequent segments by resolving overassigned variables. As a counterpart to storing these values, the different assignments are taken into account when evaluating instructions (see GETPERMUTATEDVARIABLEASSIGNMENTS(I)). Across all functions, theREGISTERFAULT(dx,e) logs the corresponding faults.
Algorithm 3Algorithm for the evaluation of an ATE segment s functionEVALSEGMENT(s)
fore∈GETELEMENTS(s)do
ifINSTANCEOF(GETINST(e), VMI)thenSTOREVALUESOFELEMENT(e) else ifINSTANCEOF(GETINST(e), VVI)thenVERIFYELEMENT(e) PERSISTLASTSTOREDVARIABLEVALUES(s)
functionVERIFYELEMENT(e)
ifNEWFAULTSREGISTERED(CHECKPRECONDITIONS(e))then return ifINSTANCEOF(e, EGPPNode)then
ifnotVERIFYINST(GETINST(e))xorISOVERASSIGNED(GETINST(e))then REGISTERFAULT(d1,e)
else ifINSTANCEOF(e, EGPPTransition)then ifVERIFYINST(GETINST(e))then
ifISOVERASSIGNED(GETINST(e))then REGISTERFAULT(d6,e)
else
REGISTERFAULT(d2,e) functionCHECKPRECONDITIONS(e)
forv∈GETVARIABLES(GETINST(e))do
ifISTIMEDEPENDENTVARIABLE(v)thenREGISTERFAULT(d5,e) ifSIZE(GETSTOREDVALUES(v)) = 0thenREGISTERFAULT(d3,e) functionISOVERASSIGNED(i)
forv∈GETVARIABLES(i)do
ifSIZE(GETSTOREDVALUES(v)) > 1then returntrue returnfalse
functionSTOREVALUESOFELEMENT(e)
ifINSTANCEOF(e, EGPPGraph)thenSTORE(GETINSTOFSTUB(e)) ifINSTANCEOF(e, EGPPNode)thenSTORE(GETINST(e))
functionVERIFYINST(i)
forva∈GETPERMUTATEDVARIABLEASSIGNMENTS(i)do ifEVAL(i,va)then returntrue
returnfalse
A special feature of the extended variant of the pseudo-code can be seen in the method
STOREVALUESOFELEMENT(e), since thestubconcept available in the EGPP context, is addressed here (see section 7.2.2). The stub graph, which is connected to anEGPPGraph, may contain black box information about already executed tests on lower integration levels. Black Box Information in this context means that the Stubcontains information about the outputs or variable assignments based on a set of input variables. Provided that the required input variables of the stub coincide with the current system state be- fore the evaluation of the relevant EGPPGraph, the new system state can be derived directly from the outputs stored in the stub. Unless there is a match in the inputs, the EGPP model contained in theEGPPGraph has to be analyzed, but this requires signifi- cantly more effort. The stub concept can be seen as a performance improvement in this context if extensive testing has been done especially on low integration levels and there- fore comprehensive information is available in the stub models. In combination with the data flow analysis of the paths, a control flow analysis is performed, which in partic- ular examines whether the path under consideration could be successfully terminated at the specified endpoint. This type of control flow analysis relies on the information of the Integration Model, where the structural relationships are stored. Analogous to the set of different fault types for data flow analysis, a setC∶= {c1,c2,c3,c4}describing the different control flow faults was defined.
c1 All verifying instructions of the path are fulfilled and the last verification point is solved by the instructions of one of the end nodes of the System Model
c2 All verifying instructions of the path are fulfilled and the last verification point could not be solved using the instructions of one of the end nodes of the System Model
c3 At least one verification point of the path could not be fulfilled, but one of the end nodes of the System Model is part of the path
c4 At least one verification point of the path could not be fulfilled, but no end node of the System Model is part of the path
We first distinguish whether the present path in the data flow analysis could satisfy all VVIs (c1andc2) or not (c3 andc4). In the first case, it is further distinguished whether the termination of the path is given in an endpoint specified by the Integration Model (c1) or whether another node is the termination (c2). In the second case, it is only dis- tinguished whether one of the specified end nodes is part of the failed path execution (c3) or not (c4).
Overall, a set of faults is recorded for each execution path, from which a set OS ∶=
{R1, . . . ,R∣Plim∣}is created for the entire segment, whereR∶=DM∪CP. Here,DMrepre- sents the set of data flow results, where an element of this set always is a pairD×M, withMbeing the set of model elements (nodes and edges) of the merged EGPP model.
CP, in contrast, represents the set of control flow results, with each element being a pair C×px.
For our considered running example, which was already introduced in section 10.2.3, the following setOS∶= {R1, . . . ,R11}results according to the algorithm.
R1 = {(d2, 2),(d1,Pr),(c3,p1)}
R2 = {(d2, 2),(d1,Pr),(c3,p2)}
R3 = {(d2, 2),(d1,A),(c4,p3)}
R4 = {(d2, 3),(d1,A),(c3,p4)}
R5 = {(d2, 3),(d1,A),(c3,p5)}
R6 = {(d2, 3),(d1,A),(c4,p6)}
R7 = {(c1,p7)}
R8 = {(d1,A),(c3,p8)}
R9 = {(d2, 3),(c3,p9)}
R10 = {(d2, 3),(d1,A),(c4,p10)}
R11 = {(d1,A),(c4,p11)}
The sets R1 - R11 show the faults for the respective paths p1 - p11. For example, R1 includes three elements, where the first element describes that the guard csmSwitch
&& !SBAvailableof the edge2 could not be satisfied. The second element describes that furthermore the conditioncsmSwitch == 0 && SBAvailable == 0 && sbiCmd !=
EMER_BRAKE_CMD of the nodePrcould not be fulfilled. Finally, the last element of the result set describes that at least one condition of a node could not be satisfied, but one of the specified end nodes is part of the path. The results of the other paths are to be interpreted analogously. All in all, this information provides the basis for the derivation of a Test Verdict, presented in the following section.