Part II Product Lines for Digital Information Products
7.4 Domain Testing
7.4 Domain Testing 131
file formats, such as HTML, since a valid HTML file can be displayed differ-ently by different browsers in different screen resolutions. This is because the respective standards do not specify all details of the rendering mechanisms [W3C07a]. In this context, a possible criterion against which can be tested is the degree to which the same file is displayed differently in different browsers;
this degree can be evaluated by automated searches in HTML code for known incompatible constructs, or empirically by human testers according to certain standardized criteria.
Finally, a domain engineer has to validate each content component by ask-ing the questions: is this what was really desired? Is the content contained in one component independent enough so that, when combined with other con-tent components in different allowed configurations, the result makes sense?
Does the way the content is spread over different components make sense?
The testing of content components stops thereafter successfully if no errors were encountered in any of the components; otherwise, a re-iteration of earlier domain engineering phases is necessary.
7.4.2 Verifying the Construction Workflow Model
The verification of the QX net representing the construction workflow model is a check which ensures that it can really generate only products with con-figurations allowed by the conceptual product line model.
There are various techniques from the area of Petri nets which can be applied in this context. As already mentioned, QX nets behave as traditional Petri nets when the following assumptions hold: 1) in the analysis, alterna-tive transitions behave like traditional transitions (i.e., it is abstracted away that they might contain other transitions; such internal transitions are ig-nored); 2) the correct functioning of an alternative transition is ensured by other mechanisms; 3) programs called inside transitions are executed without errors (ensured by other mechanisms); 4) queries in the database are executed without errors.
Reachability analysis can ensure that a marking representing the state that a certain feature was built can only be reachable from a marking repre-senting that some other features were already built. This way, one can check the satisfaction of the constraints of the conceptual product line model that demand that some features can be chosen only when some features in parent nodes were chosen (see Sect. 7.2.1). This analysis can be done by creating a marking graph.
Analysis of the Marking Graph
The definition of the marking graph of a marked Petri net is based on arc-labeled directed graphs [DR98].
Definition 7.6 (Arc-Labeled Directed Graph). An arc-labeled directed graph is given by
7.4 Domain Testing 133
• a set V of vertices
• a set of labeled edges (v, l, v′), representing source vertex, label, target ver-tex, where v, v′∈ V and l is a label of some given set L.
For a given labeled edge (v, l, v′), v′is an immediate successor of v. A finite or infinite sequence of labeled edges (v, l, v1), (v1, l, v2), . . . is called a path of an arc-labeled graph. All vertices {v, v1, v2, . . .} are called successors of v.
Definition 7.7 (Marking Graph). Let (N, W, M0) be a marked Petri net (cf. Def. 7.2) with N = (P, T, F ). The marking graph of (N, W, M0) is an arc-labeled directed graph with a distinguished initial vertex, and edges arc-labeled by transitions. Furthermore,
• the vertices represent the reachable markings,
• the distinguished initial vertex represents the initial marking M0,
• the labeled edges are triples (M, t, M′) such that M, M′are reachable mark-ings satisfying M → Mt ′, i.e., the occurrence of transition t ∈ T (which is enabled under M ) transforms the marking of the net from M to M′ according to the occurrence rule.
In a graphical notation, the vertices of the marking graph are shown as bullets, edges as labeled arrows between two bullets, and the initial marking is distinguished by an additional arrow pointing to it.
The reachability problem can be stated as follows: given a marked Petri net (N, W, M0) and a marking M′, it should be decided whether M′ is reachable from M0. It has been shown by [May84] that this problem is decidable for the types of Petri nets used here (see Defs. 7.1 – 7.3). Using the marking graph, one can deduce various properties of a Petri net, e.g. [DR98],
• if the marking M′ represented by a vertex v′ is reachable from M0, repre-sented by v0, then there exists in the marking graph a finite directed path (v0, t1, v1), (v1, t2, v2), . . . , (vi, ti, v′); by definition, a marking of a vertex vi is reachable from itself;
• a marked Petri net is called deadlock-free if every reachable marking en-ables some transition; the net is deadlock-free if and only if its marking graph has no vertex without outgoing edge; a marking that enables no transition is called dead ;
• a marked Petri net (P,T,F) is called b-bounded (or simply bounded), if all its places are b-bounded; a place p ∈ P is b-bounded if for each reachable marking, the number of tokens it contains is ≤ b, with b ∈ N. A finite Petri net is bounded if and only if its marking graph has finitely many vertices.
As discussed in Sect. 7.2.3, QX nets have exactly one sink place that does not have outgoing arrows. This means that QX nets necessarily have exactly one marking with a deadlock because there is one dead marking when the workflow reaches a “stop” state. This insight can be used for verification purposes; if a designer encounters other dead markings (i.e., in the marking
graph there is more than one node without outgoing edges) this must be a design error, and the QX net must be re-designed. In addition, the marking graph reveals the preconditions in the control flow before a certain transition can occur. By interpreting a marking as a state in which some features are created, the marking graph can be used to ensure that only those product configurations can be generated which were allowed by the conceptual product line model, by checking the appropriate edges in the marking graph.
The verification is finished successfully if it is ensured that the QX net has exactly one “stop” marking and if within the marking graph only desired state changes are possible.
Example 7.6
As an example, the marking graph is constructed for the net in Fig. 7.7 with its initial marking M0 shown there. In the corresponding marking graph in Fig.
7.11, each vertex represents a unique marking that is reachable from M0, and each vertex is given a unique number. For clarity, the complete marking vector of a marking is omitted. For example, M0 = (1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), M1 = (0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
M14= (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1).
M 0 T1
M 1 T2
M 2 T3
M 3 T4
M 4 T5
M 5 T6
M 6 M 7
T7 T8
M 8 T9
T10
T13 M 10
T14 M 11
T15 T16 T11
M 9 M 12
T18 M 14 T12
M 13
T17
Fig. 7.11. Marking graph for the net in Fig. 7.7.
As can be seen, the final marking M14 (where there is one token in the place
“stop”) is reachable from the initial marking M0. In addition, the net is bounded because the marking graph has finitely many vertices. Furthermore, one can deduce from the marking graph that for example the marking M10, where the feature “in-troduction to workflow” was created by T 13, will only be reachable if the creation of the optional workflow domain was chosen before (T 11).
Unfortunately, there are some drawbacks of this method which are mainly resulting from the computational complexity of Petri nets. The marking graph
7.4 Domain Testing 135 cannot always be computed efficiently; in terms of numbers of places, transi-tions, arcs, and tokens, it requires for bounded Petri nets exponential space [Lip76, May84, Sta90]. In addition, for some nets, the corresponding marking graph can have an infinite number of vertices (e.g., for unbounded nets).
However, typical construction workflow models in PLANT seem to be prevalently bounded and have a marking graph with a finite number of nodes (see also case study in Chapter 10). It becomes visible here why the possible types of constructs were limited in conceptual product line model: by allow-ing only common, optional, and alternative features or domains, one can use predefined workflow patterns (see Fig. 7.6) within the construction workflow model, which help to obtain nets which are likely to be bounded. In addition, marking graph analysis can be done for the reusable workflow modules stored in the workflow warehouse (see Fig. 7.8), which are usually small enough to make the computation feasible. If the marking graph approach becomes infea-sible for complex nets, there are also other analysis techniques based on linear algebra, which provide semi-decision algorithms with a weaker form of reach-ability analysis [DR98]; other techniques are also discussed in [Sta90, DR98].
7.4.3 Integration Testing of Content Components
Integration testing in PLANT focuses on testing whether the combination of different content components works, and if a product can be created as a combination of different content components. This poses in PLANT sim-ilar problems like domain integration testing in software product lines (see Sect. 4.3.4), since after domain realization, there is not yet a finished digi-tal information product that can be tested. In addition, there are potentially many configurations of different core assets that can be assembled to different information products, which makes an exhaustive integration testing almost impossible, or indeed infeasible.
As a tradeoff, PLANT follows therefore for the integration testing of con-tent components the sample application strategy approach (cf. Sect. 4.3.4).
This strategy creates one or more sample digital information products. For this, the product map template is complemented with sample versions for ev-ery feature in the product line. Thereafter, the construction workflow model is executed.
The generation of a sample digital information product has the drawback that only the creation of a specific feature configuration is tested, which poses the question of which and how many configurations to test. PLANT uses the marking graph of the construction workflow model (Sect. 7.4.2) to derive a finite set of products with sample configurations that are to be tested. Simi-lar to branch coverage testing of control flow graphs of software [Bal00], the different sample products are generated here in such a way that each edge in the marking graph of the construction workflow model is passed at least once.
The rationale behind this is that every state modification in the workflow is tried out at least once (an example is shown in later Sect. 10.4). Integration
testing is finished successfully when all sample products are generated suc-cessfully. In case or errors, another iteration of domain engineering can be used for corrections. Another related approach is described in [DOZZ97].
As already explained in Sect. 7.4.2, the marking graph could theoretically have sometimes no finite representation or may be inefficient to compute, which would mean that integration testing could not be done in this way.
However, as reasoned in the previous Section, marking graphs for the context where PLANT is used are expected to have prevalently a finite representation.
This is also supported by the results of a case study carried out a “real-world”
context, to be presented in Sect. 10.