Prescriptive Study: Developing Design Support
5.4 Task Clarification
5. Support Evaluation (Section 5.8):
- The Actual Support is evaluated for completeness, internal consistency, etc, and modified if necessary.
- Based on the results of the Support Evaluation and the modifications to the Actual Support, the Outline Evaluation Plan is updated for use in DS-II (Chapter 6).
Typically, a review of the literature at this stage investigates each relevant, existing support for its scope, functionalities, application area, underlying concept, assumptions behind the implementation, implementation technologies and evaluation results, if available. The “Support Outline: Summarising Scope and Assumptions” (Appendix B.4) can be useful here. The checklist is intended to help researchers document the scope and assumptions underlying the support they are developing, but can equally well be used to describe existing support.
The literature review is used to verify the Initial Impact Model. There will be interesting alternative sets of Key Factors that could be addressed. These alternative sets will be linked to the Success Criteria in different ways and might have a different impact. Furthermore, taking into account the context within which the support should be used, and the intent and drivers behind the support can also lead to alternative Key Factors and Criteria. The result is a set of alternative Impact Models. By exploring in each model the links between Key Factors and Success Criteria, looking for factors as close as possible to the Success Criteria and yet measurable within the project, potential Measurable Success Criteria for each alternative Impact Model can be identified.
The alternative Impact Models are compared and evaluated to select the most promising Impact Model for the Intended Support, the Intended Impact Model. The choice of the Key Factors, Measurable Success Criteria and links will be governed by their potential for scientific contribution (academically worthwhile) and the envisaged strengths of the impact of addressing the Key Factors on the Success Criteria (practically worthwhile).
The Intended Impact Model can differ from the Initial Impact Model as well as the Reference Model in various ways:
• additional nodes and links: e.g., the support intended to train novice designers to exhibit expert behaviour is expected to encourage novices to exhibit a more inquisitive behaviour (a new node and related links) asking certain questions they otherwise would not have asked;
• removed nodes and links: e.g., novices are expected to no longer make certain mistakes they made when they did not use the support;
• modified links, i.e., links with changed signs: e.g., novices who use the support less often ask typical ‘novice’ questions.
Most of the links in the Impact Model are assumptions. Many of the links are modifications from existing links in the Reference Model or are introduced as an anticipated consequence of the introduction of the support to be developed. Even the links that are understood well in the existing situation might not be the same in the new desired situation because of unexpected side-effects of the introduction of the support. Each link in an Impact Model should be provided with argumentation about why the effect is expected, based on the literature, assumptions and reasoning.
The Intended Impact Model describes the effects of the Intended Support, the requirements are not known yet. To start with, a problem statement is formulated, describing the core problem addressed by the support. Then, a list of requirements to be satisfied by the support is formulated, taking into account the documentation and information gathered thus far. The list should cover the entire life of the
support, including its implementation, testing, installation, introduction, use and maintenance phases, and keeping in mind organisational, technical and other contextual pre-requisites. Many of these requirements cannot be formulated yet or formulated precisely; the list will become more complete and detailed as the PS stage progresses. At this stage, it is important not to presume a particular type of support: the problem statement and the requirements list should be as solution- neutral as possible. Appendix B1.1 lists several existing requirements identification and evaluation techniques.
At this stage it is useful to start the process of documenting the scope and assumptions of the Intended Support using the already mentioned Checklist for Summarising Scope and Assumptions.
The outcome of the Task Clarification is an Intended Impact Model, a problem statement and a list of requirements for the Intended Support.
Reliability Example
To clarify the development task in our reliability example, the Reference and Initial Impact Models resulting from DS-I, shown in Figures 4.7 to 4.10, are analysed.
Some of the main conclusions are:
• Increased ‘market share’ has been taken as the Success Criterion, because the company involved had identified market share as their primary concern.
The suggested Key Factor has been ‘reliability of embodiment’.
• Primary findings from DS-I (see Figure 4.8) indicate that previous failures in improving reliability at the detailed design stage could be due to poor reliability of embodiments, and that the use of general rules for embodiment (clarity, simplicity and unity) adequately demonstrated their impact on improving product reliability. In particular, the ‘level of clarity’
and the ‘level of simplicity’ were found to have a significant link with
‘product reliability’ as long as the ‘level of unity’ (assessed in terms of strength of components of a product) remained adequate. The Initial Impact Model at the end of DS-I therefore suggests support to determine ‘level of clarity’ and ‘level of simplicity’ only, with ‘level of clarity’ as the most influential factor.
• From other available evidence, it was seen that ‘early failure detection and analysis’ (see Figure 4.9) has additional impact on ‘market share’ via ‘lead time’. The Initial Impact Model from DS-I therefore suggested to focus on improving reliability of embodiments, which will involve earlier analyses of failures than in detailed design.
• That the three factors ‘levels of clarity’ etc., can be assessed early in embodiment, was also the result of investigations in DS-I. Improvement of
‘reliability of embodiment’ by means of providing ‘support for early embodiment’ was therefore chosen as the target for the support to be developed in PS.
In order to identify existing support, if any, for aiding improvement of clarity and simplicity, a literature review is undertaken, but no such support found other than very generic guidelines and examples. Sufficient support exists for determining the
‘level of unity’. However, the link between the level of unity and the levels of clarity and simplicity is not addressed at all, although the levels of clarity, unity and simplicity together influence the reliability of the embodiment; the level of unity must remain adequate for clarity and simplicity levels to become effective. It was therefore decided to revise the earlier suggestion and to address reliability of early embodiment by means of all three factors: level of clarity, of simplicity and of unity. The main reason for not choosing these as Key Factors is that the levels of clarity, unity and simplicity together influence the reliability of the embodiment:
addressing each one separately will not have the same effect.
The focus of the PS-stage is thus on supporting the assessment of all three levels as early as possible during the embodiment stage, the combination of these into one measure of reliability of embodiment, and the improvement of the reliability of the embodiment. This results in the updated Intended Impact Model shown in Figure 5.2. The factors and links shown with dashed lines are outside the scope of the PS as discussed in Section 4.8.2. Note that the choice of any other set of Key Factors to be influenced by the support, e.g., ‘use of DfR methods’, or
‘quality of product use’, would have led to a different, alternative Intended Impact Model.
Figure 5.2 Lower part of the Intended Impact Model after Task Clarification in PS Based on all above information, a list of requirements for the support is formulated.
A partial list is given in Table 5.1, which is based on the checklist of Roozenburg and Eekels (1995). As it is not certain whether it is possible to suggest modifications on the basis of the assessed levels of clarity, simplicity and unity, the related requirement is listed as a wish, rather than a demand.
added
product reliability
quality of bought-in components quality of production quality of
product use
clarity of instruction reliability of
detail design
+ +
+ +
_ _
+ _
+
motivation of use
_ +
use of DfR methods
+
+ +
+
Upper part of Initial Impact Model (as in Fig. 4.9)
level of unity level of
simplicity level of
clarity
+ +
+ +
+ +
Key Factor number of
iterations
+_
reliability of embodiment
support early
embodiment % of
project time left to improve
+
added
product reliability
quality of bought-in components quality of production quality of
product use
clarity of instruction reliability of
detail design
+ +
+ +
_ _
+ _
+
motivation of use
_ +
use of DfR methods
+
+ +
+
Upper part of Initial Impact Model (as in Fig. 4.9)
level of unity level of
simplicity level of
clarity
+ +
+ +
+ +
Key Factor number of
iterations
+_
reliability of embodiment
support early
embodiment % of
project time left to improve
+
Table 5.1 Partial requirements list for the support related to the Intended Impact Model in Figure 5.2 (D = demand, W = wish)
Problem statement:
Develop a support to help experienced mechanical designers improve reliability of early design embodiments
Performance:
D The support should help assess the levels of clarity, simplicity and unity.
D The support should help assess the reliability of the embodiment.
W The support should suggest modifications of the embodiment to improve these levels.
D The support should be able to support the design of mechanical systems W The support should be able to support the design of electromechanical systems D The support should be able to use the information available in engineering
drawings or CAD models as input.
W The assessment should be fast enough to be used as a regular activity within the design process
Ergonomics:
D The support should be usable by individual experienced mechanical designers D The support should be easy to introduce
D The support should be easy to learn D The support should be easy to use W The support should be easy to maintain D The support should be easy to install Cost:
D The support should cost less than XX to buy W The support should cost less than YY to maintain Introduction:
D It should be possible to use the support in conjunction with existing support available in mechanical design offices
W No additional hardware or software will be necessary Life:
W The support, with maintenance, should have indefinite life Disposal:
W If the support involves hardware and consumables, these should be limited and environmentally benign.