This becknique is recommended when the project's experts cannot cuor!inate their schedules or when geographic distance separates them.
lphi technique is also appropriate when bringing experts together to venue may generate excess friction.
Inputs and Outputs
The inputs for the Delphi technique are questions or questionnaires. The questionnaire addresses the risk area(s) of concern, allowing for progressive refinement of the answers provided until general consensus is achieved.
The questionnaire should allow for sufficient focus on the areas of concern without directing the experts to specific responses.
Outputs from the process are progressively detailed because all iterations should draw the experts involved closer to consensus. The initial responses to the questionnaire will generally reflect the most intense biases of the experts. Through the iterations, the facilitator will attempt to define com- mon ground within their responses, refining the responses until consensus is achieved.
Major Steps in Applying the Technique
The technique relies heavily on the facilitator's ability both to generate the original questions to submit to the experts and to distill the information from the experts as it is received. The process is simple but is potentially time-consuming:
w Identify experts and ensure their participation. The experts need not be individuals who have already done the work or dealt with the risks under consideration, but they should be individuals who are attuned to the organization, the customer, and their mutual concerns. Experts can be defined as anyone who has an informed stake in the project and its processes. Commitments for participation should come from the experts, their direct superiors, or both.
Create the Delphi instrument. Questions asked under the Delphi tech- nique must be sufficiently specific to draw out information of value but also sufficiently general to allow for creative interpretation. Because risk management is inherently an inexact science, attempts to generate excessive precision may lead to false assumptions. The Delphi questions should avoid cultural and organizational bias and should not be direc- tive (unless there is a need to evaluate risk issues in a niche rather than across the entire project spectrum).
w Have the experts respond to the instrument. Classically, this is done remotely, allowing the experts sufficient time to ruminate over their responses. However, some organizations have supported encouraging questionnaire completion en masse during meetings to expedite the process. No matter the approach, the idea is to pursue all the key insights of the experts. The approach (e-mail, snail mail, meetings) for gathering the experts' observations will largely determine the timing for the process as a whole.
110 Chapter 10
w Review and restate the responses. The facilitator will carefully review the responses, attempting to identify common areas, issues, and concerns.
These will be documented and returned to the experts for their assess- ment and review. Again, this may happen by mail or in a meeting, although the classic approach is to conduct the Delphi method remotely.
Gather the experts' opinions and repeat. The process is repeated as many times as the facilitator deems appropriate in order to draw out the responses necessary to move forward. Three process cycles are con- sidered a minimum to allow for thoughtful review and reassessment.
Distribute and apply the data. Once sufficient cycles have been com- pleted, the facilitator should issue the final version of the documen- tation and explain how, when, and where it will be applied. This is important so that the experts can observe how their contributions will serve the project's needs and where their issues fit in the grander scheme of risks and risk issues up for discussion.
Use of Results
The Delphi technique is frequently used when there are only a handful of experts who have an understanding of the project. It is also used when certain experts have insights about a particular aspect of the project that cannot be ignored. Although some other risk identification, assessment, and response development tools have broad application, the Delphi technique is a more exacting tool, drawing out only the responses or types of responses desired. The information acquired from the Delphi technique can be used to support risk identification, qualification, quantification, or response development.
Resource Requirements
The Delphi technique requires that a project have both a skilled Delphi facilitator and experts to support the process. The facilitator must have the ability to present the premise clearly in the Delphi questionnaire and then must have the capacity to refine and distill the inputs from the
participants. The participants, in turn, must have an awareness of the area on which they are being consulted.
Reliability
The technique generates relatively reliable data (for a qualitative analysis) because multiple experts subject the information to at least three iterations of reviews. The iterative nature of the process and the requisite reviews tend to enhance accuracy, although the use of inappropriate experts or the develop- ment of poorly couched questions may produce less than optimal results.
Delphi Technique 1 1 1
Still, because there are multiple reviewers, some built-in safeguards ensure a measure of reliability.
Selection Criteria
As with each chapter on techniques, the Delphi technique is assessed using selection criteria relating to resource requirements, applications, and outputs for the technique. To compare the Delphi technique with other techniques, review Table 6.
Resource Requirements
The Delphi technique requires little more than basic office supplies. The infrastructure for the technique is minimal, as it is little more than a specially processed expert interview.
From a personnel perspective, the facilitator's greatest talent must be in distilling the information from one iteration of the approach to the next, achieving a balance of the information presented, and at the same time, not alienating the experts involved.
Participants in a Delphi technique analysis can derive comfort in the fact that their contributions, for the most part, will be anonymous because
their inputs will never be directly presented to the other experts. The facilitator will filter and distill it first. Nonetheless, the participants should be reasonably skilled at documenting their contributions, as that is where the Delphi technique generates its value.
w Cost for the Delphi technique is minimal. Because most participants can complete the questionnaire at their leisure, there is little time pressure on the participant's side. The facilitator is also generally not time-constrained in this practice and thus has some latitude to com- plete this effort when there is time to work on it. Even though the cost is minimal, the time to complete a Delphi technique process can be extensive, as it can continue for weeks if unmanaged.
w Proper facilities and equipment for the Delphi technique consist of little more than office supplies for participants to record and return their responses to the facilitator. In some more modern sessions, the Delphi technique is conducted in real time using e-mail and deadlines.
Although the facilities infrastructure for such an approach is more expensive, most organizations already have such capabilities in-house.
The time needed to implement the Delphi technique is the single most significant drawback of the approach. Despite that e-mail has created a faster way to accomplish the work, the technique still may take several days to complete. For some organizations, however, the quality of the data generated makes this trade-off worthwhile.
112 Chapter 10
w Although the ease of use for the participants is high, the facilitator has to be skilled at distilling and paraphrasing information. The facilitator must also ensure that the process stays on track. It is very easy to allow the Delphi technique to falter because of the time frames and distance involved.
The project manager's time commitment is slight, with intense, short bursts of activity each time a cycle of responses is received.
Applications
The Delphi technique has broad utility because of its use of the experts' skills and insights. The applicability of the technique is assessed on a scale of high, medium, and low.
Project status reporting is an area where the Delphi technique can provide more balanced insight than other tools can. Some projects falter because there is not a common understanding of the work accomplished, but the Delphi technique by its nature can reorient a team. Since the tool draws out consensus of the experts, it can facilitate in-depth analyses of project status. The tool's value here is medium.
w Because the experts in an organization tend to make major planning decisions, the Delphi technique can be seen as viable here. Particularly in situations where there is significant conflict over planning decisions, the Delphi technique has high applicability due to its capacity to get a common vision from a group of experts.
w Contract strategy selection is an area where experts are frequently tapped to make decisions, and likewise, conflict can be significant. As with planning decisions, the Delphi technique can serve extremely well in these situations.
w Applying the Delphi technique in milestone preparation would probably have limited use. Whereas milestone preparation is a function of needs analysis, multiple experts are normally not required to ascertain the best times for milestones.
w Design guidance is a prime application for the Delphi technique. It is a creative endeavor requiring multiple perspectives. As such, the Delphi technique is a classic tool for bringing different approaches to the fore and selecting the best possible approach.
w Source selection may be an application of the Delphi technique. If the experts in the technique are familiar with the needs of the procurement and if they are attuned to the organization's limitations, the Delphi technique may be appropriate. However, the tool's utility here is medium at best.
Delphi Technique 1 13
w Budget submittal is a quantitative process, and thus cannot take full advantage of the Delphi technique.
The Delphi technique is peerless in allowing for thoughtful review of the subject matter experts' insights. As such, organizations may be able to use this technique to establish risk responses, to identify risks, or to assess risk performance to date. However, the drawbacks associated with the timing of the process tend to limit its utility. When time is not of the essence, the Delphi technique can create some of the most thorough qualitative analyses available to the project manager.
Outputs
The outputs of the Delphi technique are sets of modified responses to the questionnaire. Although participants generate those responses, the facilitator has the ultimate responsibility to produce final outputs based on an amalgam of responses from subject matter experts to each question or issue.
w The accuracy of the Delphi technique is qualitatively rooted but is perhaps the single most accurate qualitative tool since it draws on multiple experts to establish its conclusions.
w Level of detail is a strength of the Delphi technique because there are rarely limits on the insights that the experts can share. As the process goes through multiple iterations, the level of detail can increase if the questions are expanded or the follow-up is particularly detailed or provocative.
w Utility is a subjective factor that takes into account both the effort involved and the value of the information. The Delphi technique tends to generate highly utilitarian information as it is revised several times before the outputs are finalized.
Summary
The Delphi technique is time-consuming. But it is a sound, structured practice for drawing out insights from professionals who might otherwise not contribute to the project's body of knowledge. It affords the facilitator the opportunity to review multiple perspectives before coming to grips with the middle-of-the-road perspective that Delphi tends to generate. The technique can be applied in a variety of situations, but for each, the time constraint must be given serious consideration.