DRM, a Design Research Methodology - Smart Library UMRI

One of our students' consistent observations is the lack of material for researchers on design research methodology: also on DRM. Design research aims to increase our understanding of the phenomena of design in all its complexity”.

Introduction

Design
Design Research
Main Issues

Lack of Overview of Existing Research
Lack of Use of Results in Practice
Lack of Scientific Rigour

Need for a Design Research Methodology
Objectives
Structure of this Book
Main Points

There are many definitions of design research, often including the goal of improving design in practice. It should be noted that the term 'design research' and related terms such as

Figure 1.1 illustrates the overall aim and objectives, and highlights the facets of design, discussed earlier in detail in Section 1.1

DRM: A Design Research Methodology

Introduction

A methodology should be used in a flexible and opportunistic way to adapt to the specifics of the research topic and any interesting opportunities that may arise (see also section 3.8: general guidelines for conducting research). As indicated in Chapter 1, the purpose of DRM is to make design research more effective and efficient.

Methodological Framework

In the Prescriptive Study (PS) phase, researchers use increased understanding of the existing situation to refine and refine the initial description of the desired situation. The researchers decide to focus on improving the quality of the problem definition as the most promising factor to address.

Types of Research Within the DRM Framework

Many research projects we have seen end with the realization rather than an evaluation of the support. Had the researchers known that existing support was ineffective, but not known the precise problems, they could have decided to focus on a systematic evaluation of the use and usefulness of existing support (DS-II) and development of proposals for improvement (PS) . .

Representing Existing and Desired Situations

Graphical Representation
From Reference Model to Impact Model

The impact model represents the desired situation and shows the assumed effect of the support to be developed. Therefore, it is necessary to generate a model of the desired situation based on the reference model, i.e. influence model.

Figure 2.3 A Reference Model representing the – partly assumed – existing situation

Success Criteria and Measurable Success Criteria

The factors whose desired values are considered measurable success criteria are the measurable success factors. It is therefore important that the measurable success factors are chosen so that they are as close as possible to the success factors, i.e.

Figure 2.7 The chosen Success and Measurable Success Criteria as well as Key Factors for the Reference Model shown in Figure 2.3

The Main Stages

Research Clarification (RC)
Descriptive Study I (DS-I)
Prescriptive Study (PS)
Descriptive Study II (DS-II)
Summary

For example, the term "improve" in a goal has a significant effect on the support rating: "improve" is a relative term. In the example shown in Figure 2.8, this is the effect of support on the "quality of problem definition".

Comparison with Other Methodologies

Because of the latter, they also distinguish between descriptive and prescriptive models: the first based on reality (our Reference Model) and the second on “the intended or anticipated reality that would be considered an improvement of the design practice” (our Impact Model). On this basis, a description of the ideal situation is made, which is compared with reality to generate proposals to improve reality.

Figure 2.9 DRM framework: stages, basic means and deliverables

Main Points

In DRM, descriptions of the existing and the desired situation are modeled as networks of influencing factors. The impact model represents the desired situation, and shows the intended impact of the support.

Research Clarification

Research Clarification Process

This involves identifying potential criteria against which the results of the research can be judged; and the formulation of appropriate research questions and hypotheses, based on the Initial Reference and Impact Models (see Section 3.4). This involves identifying the type of design research that needs to be undertaken to solve the research problem (see Section 3.5).

Identifying Overall Topic of Interest

What benefits should solving/answering these problems/questions bring to each of the stakeholders. It is useful to start developing models of the existing and desired situation (the reference model and the influence model) to arrive at a common view of the initial understanding and expectations.

Figure 3.2 The shared understanding of the existing situation (all links still based on assumptions [A])

Clarifying Current Understanding and Expectations

The research begins with a descriptive objective: understanding the causes of unsatisfactory support outcomes. Even in this situation, support information should be gathered and a model of the desired situation should be developed if it is not already available.

Figure 3.8 Part of the Initial Reference Model based on evidence from literature (all sources are ficticious)

Clarifying Criteria, Main Questions and Hypotheses

Criteria
Research Questions and Hypotheses

The formulation of the main research questions and hypotheses at the beginning of the project is essential for conducting the research. In our case, some research questions would be: What causes product unreliability.

Figure 3.10 shows the Initial Reference Model based on the literature sources of the example and the alternative example

Selecting Type of Research

The level of understanding of the existing situation obtained from the literature (review-based DS-I) is sufficient to develop support (Comprehensive PS), and project resources allow for formal assessment of support (Comprehensive DS-II). As a result, research projects of this type include; a comprehensive study of the existing situation (Comprehensive DS-I); support development (Inclusive PS); and a formal evaluation of this support (Comprehensive DS-II).

Determining Areas of Relevance and Contribution

This representation clarifies the basis on which the research should be based and the areas of contribution of the research. Separate diagrams can be drawn to cover additional (sub)objectives, topics and research questions, or – as done here – one diagram can be drawn that encompasses the entire study.

Figure 3.13 Example: Second step of setting up the ARC diagram for the project on

Formulating Overall Research Plan

Overall Research Plan
Visualisation Exercise
Reflection on RC

It became clear that there was not enough understanding of the typical types of knowledge required to develop sports equipment and where and how these types of knowledge and the areas involved affect the process. What kind of insight into the life of the product should you have in front of you.

Figure 3.16 Aims against stages, after Eriksson (2007)

General Guidelines on Doing Research

It is especially important to have a strong tendency towards originality (Ramon y Cajal 1999); alternative explanations must always be considered and evaluated against the criterion of observation. Pretense should be avoided at all costs, as it is dishonorable and will be discovered sooner or later if the matter is interesting enough.

Main Points

Determining the type of research to be undertaken, that is, which DRM stages the research should focus on, depends on the current state of the research. Since a detailed research plan requires knowledge of the exact research questions, the initial Overall Research Plan created in the RC stage may not be very concrete.

Descriptive Study I: Understanding Design

Schools of Thought

In this section, we address two of the issues raised in these disciplines that are particularly relevant to design research. A clear representative of the second school of thought is the data-driven, Grounded Theory, approach, where theories are based on empirical data.

Types of DS-I

Everything that can be counted doesn't necessarily count; everything that counts cannot necessarily be counted.”.

DS-I Process Steps

This involves combining the results of the various studies, adjusting and completing the Reference Model and updating the Initial Impact Model. One could also choose to start with interviews to gain a detailed understanding of the topic of interest in the contexts represented by the interviewees, and then conduct a survey to verify whether the findings are true for other contexts.

Reviewing Literature

Identifying Literature
Summarising Literature

Whether or not strong evidence exists does not necessarily reflect the quality of the study – the study may have been exploratory. The principles are derived from best practice and have been found to have a positive effect on product quality.

Determining Research Focus

Identifying and Defining Factors and Links of Interest
Formulating Research Questions and Hypotheses
Techniques for Refining Research Questions and Hypotheses
Focusing the Set of Research Questions and Hypotheses

H A comprehensive list of requirements developed early in the design process minimizes changes in later stages of the design process. This technique works backwards from the research documentation to the data to be collected.

Figure 4.5 Different possible ways of presenting the results, affecting the research questions to be asked and research methods to be chosen

Developing Research Plan for DS-I

Selection of Methods
Selection of Data-collection Methods
Detailing the Research Plan
Pilot Study

Enter comments on any specific features of the method or setting that are relevant to addressing the research question or hypothesis. This method addresses both the research questions and one of the hypotheses, and the effort is not too great.

Table 4.2 Real-time data-collection methods used in design research observation (no involvement of the researcher);

Undertaking an Empirical Study

Collecting Data
Processing Data
Analysing and Interpreting Data
Verifying Results
Drawing Conclusions
Updating the Initial Reference Model
Determining Further Empirical Studies

Codes that abstract the data are intended to be used for analysis instead of the original data. Most tests require certain assumptions to be met if the results of the data analysis are to be meaningfully interpreted.

Drawing Overall Conclusions

Combining Results of Empirical Studies
Completing the Reference Model and Updating the Initial Impact Model Model
Documenting Research
Consequences and Suggestions for the Intended Support
Determining Next Stage
Determining Future Work

Figures 4.7 and 4.8 show the top and bottom of the final reference model for the reliability case. Figures 4.9 and 4.10 show the top and bottom of the initial impact model for our reliability example.

Figure 4.7 Upper part of the Reference Model resulting from the DS-I stage

Main Points

Its development, and especially its evaluation, will increase understanding and sharpen the possible questions and hypotheses that can or should be further explored. To determine the research focus, factors and relationships of interest are identified and defined, research questions and hypotheses are formulated and refined; and the final chosen set.

Prescriptive Study: Developing Design Support

Types of Design Support

By design methods we mean the sequences of activities to be followed to improve particular phases of the design process (task clarification, conceptual design, detail design, etc.) and specific tasks within these phases (eg generation, evaluation , etc. ). The design tool supports the effective and efficient use of the approach, method or instruction.

Types of PS

When a research project involves a comprehensive evaluation (DS-II) of a support (Design Research Types 4, 6 and 7 in Figure 3.12), an initial PS should be conducted to demonstrate the impact of the evaluation outcome on the support . and about the Impact Model. A Review-based PS is based on literature research and documentation about the support and, where possible, on conversations with users and developers.

A Systematic PS Process

The focus of the Actual Support must be on the core contribution of the research project, i.e. the core functionality of the Intended Support (see discussion in Section 5.7.1). These are compared and evaluated to create the impact model that represents the desired effects of the intended support (intended impact model).

Figure 5.1 Main steps in the PSstage; stars (*) indicate steps of an Initial PS

Task Clarification

D The support must be usable by individual experienced mechanical designers D The support must be easy to configure. D It should be possible to use the support in conjunction with existing support available in mechanical design offices.

Table 5.1 Partial requirements list for the support related to the Intended Impact Model in Figure 5.2 (D = demand, W = wish)

Conceptualisation

Determining Main Functions
Generating and Selecting Support Concepts
Introduction Plan

The resulting Intended Impact Model is shown in Figure 5.4 (the remainder of the model remains as shown in Figure 5.2). How difficult is it to carry out the step in the intended context, and what is involved.

Figure 5.3 Three alternative, superimposed, Intended Impact Models (partial models) representing three different Intended Support concepts covering alternative sets of functionalities, indicated by hexagon shapes

Elaboration

For example, Stephenson (see Appendix C.8) asks: “How can we improve reliability (problem) evaluation (activity) in the design phase (phase) of earth-moving equipment (application) implementation?” Chakrabarti (Appendix C.3) asks: “How can we support designers in generating (activity) a wider range of mechanical designs (application) in the conceptual phase (design phase) to achieve the intended functionality (problem) to fulfill?" The intended use of the support for a particular application necessitates the acquisition of domain knowledge. The synthesis example illustrates a case where the elaboration of the Intended Support requires a revisit of the DS-I phase, driven by specific questions dictated by the needs of the PS phase.

Realisation

Core Contributions, Support Functionalities and Outline Evaluation Plan Plan
Developing Actual Support
Actual Introduction Plan and Actual Impact Model
Tool Development

The result is a model of actual effect that is used to estimate actual support. The result of the realization phase is the actual support and the completed documentation of the actual support.

Figure 5.13 Lower part of the Actual Impact Model of the reliability example

Support Evaluation

This increases the chances of the support being useful and reduces the number of repetitions required. At the absolute minimum, app evaluation involves the researcher as a user using an interim version of the support to evaluate its utility and usability.

Main Points

The focus is on the core contribution of the research project, i.e. the core functionality of Targeted Support. The step results in the complete description of the Targeted Support, the Targeted Delivery Plan and the Targeted Impact Model.

Descriptive Study II: Evaluating Design Support

Evaluation

Importance of Evaluation
Types of Evaluation in DRM

Evaluating design support is essential, as its effects can only be predicted during the development of the support. The introduction of support will and is indeed intended to influence the factors in the Reference Model to realize the desired state.