CHAPTER 3 RESEARCH METHODS
3.6 Delphi Method
credibility of the interviewer; the interviewees were supplied with relevant information to give them an opportunity to consider the information. Furthermore, the interviewer explained the research to the interviewee in order to gain the interviewee’s confidence and during the interview process the interviewer maintained a neutral tone of voice, phrasing questions clearly, and using appropriate probing questions. Finally, the interviewer ensured that the attention of the interviewee was maintained at all times.
statistical analysis of the responses, which allows each participant to see where his/her opinion lies when compared to the rest of the group ; and
controlled feedback, which entails that the participants responses after each round are analysed and each respondent receives feedback during the next round (Hasson et al., 2000;
Hasson & Keeney, 2011).
The main aim of using the Delphi Method is to reach consensus (Thompson, 1995). One advantage of the Delphi Method is anonymity in answering questions which often sets the participant at ease and provides opinions that are free from peer pressure (Goodman, 1986; Snyder-Halpern, 2002).
Another advantage of the Delphi Method is that it allows people who are in different places to share their expertise without the need for a meeting (Rogers & Lopez, 2002; Murry & Hammons, 1995).
Nevertheless, some disadvantages are that its anonymity may mean that participant responses are untraceable back to the participant which can lead to a lack of accountability (Sackman, 1975).
Representativeness of the sample (Dillman et al., 1998) and low response rates (Mullen, 2003) are other limitations of the Delphi Method.
3.6.1 Delphi Panel
Several authors have criticised the use of the term “expert” since it is not easy to define the term (Beaumont, 2003; Hasson, Keeney & McKenna, 2000; Mullen, 2003). Several researchers argue that the attention should be on ensuring that the participants are able to provide relevant information based on their knowledge and experience (Beaumont, 2003; De Meyrick, 2003; Mullen, 2003; O’Loughlin
& Kelly, 2004). There are no exact guidelines for calculating the size of the panel (Mullen, 2003).
The size of the panel of a Delphi Method unlike conventional surveys does not require a statistically large number of participants to be valid (Loo, 2002; Mullen, 2003). According to Dalkey and Helmer (1963), participants should meet the following two criteria. The first recommended criterion is that experts should have knowledge and experience of the subject matter. The second criterion is that they should be representative of the profession so that suggestions may be adaptable and transferable to the general population.
On the other hand, Adler and Ziglio (1996) stated that the Delphi participants in any study should meet four requirements namely: knowledge and experience with issues under investigation; capacity and willingness to participate; sufficient time to participate in the study; and effective communication
skills. In choosing expert participants for this study, each expert was required to meet at least five of the following minimum criteria (Skulmoski, 2007):
Knowledge and experience in IS/BI.
Academic Qualification: has an earned a degree (National Diploma/B-Degree/M- Degree/PhD).
Experience: Industry experience of at least 8 years
Published articles in peer reviewed journals, books and or conferences in IS/BI.
Teaching: Has served as an instructor in the teaching of courses focusing on IS/BI or recognised related field.
professional registration with a recognised IS or ICT registration body
Capacity, willingness, and time to participate.
Panel participants were identified from a number of sources namely, various South African universities, as well as members of various computer societies in South Africa such as The Institute of Information Technology Professionals South Africa (IITPSA); The South African Institute of Computer Scientists and Information Technologists (SAICSIT). The identification of participants involved creating a list of e-mail addresses from the administrators of CSSA and ITSA. An invitation to participate was sent to the addresses. Furthermore, Google and Google Scholar search engines were also used to identify individuals who have published BI related articles. While LinkedIn was used to identify professionals who met the above requirements.
3.6.2 Delphi Survey Administration
Several researchers argue that a Delphi Method should not have more than three rounds in order to minimise time spent, cost and participant fatigue (Hasson et al., 2000; Linstone & Turoff, 1975;
Mullen, 2003; Powell, 2003). De Meyrick (2003) argued that having more than three rounds in a Delphi study may make it difficult to retain high response rates.In this study, the Delphi study was conducted in two rounds. Several other studies have used two rounds to reach consensus (Mullen, 2003).
3.6.2.1 First Round
To elicit varying ideas, views, and opinions of the participants the first round is typically open ended (Keeney, Hasson & McKenna, 2001). However, in this study the traditional first round Delphi of eliciting information from the participants was replaced with the results from the literature study and
semi structured interviews. This modified approach has been used by other researchers (Duffield, 1993; Jenkins & Smith, 1994), reduces the time of the process, as well as the likelihood of response fatigue.
To reduce the risk of bias of the limited options the participants are given an option to suggest other factors and sub factors at the end of the questionnaire (Hasson et al., 2000; Keeney et al., 2001). In this study, participants indicated their level of agreement with pre-formulated statements provided on a questionnaire (appendix E). A 4-point Likert-type scale was used for rating the factors. The scale levels were; very important (4), important (3), slightly important (2) and unimportant (1). The instruction to the participants was to rate the factors according to how important each factor is towards the success of a BI system.
Table 3.6 : Rating scale provided to participant in the Delphi Study
Scale Meaning
Very important (A most relevant factor)
First-order priority.
Has direct bearing on the success and meaningful use of BI systems.
Must be resolved or dealt with.
Important (Is relevant to the issue) Second-order priority.
Significant impact on the success and meaningful use of BI systems but not until other factors are addressed.
Does not have to be fully resolved or dealt with.
Slightly important (Insignificantly relevant)
Third-order priority.
Has little importance on the success and meaningful use of BI systems.
Not a determining factor or major issue.
Unimportant (No priority) No relevance.
No measureable effect on the success and meaningful use of
BI systems.
Should be dropped as an aspect/barrier to consider.
3.6.2.2 Second Round
To initiate the second round, an email was sent to the first round participants with the questionnaire attached. In the second round, each participant received a personalised questionnaire showing their response from the first round and a summary of the other participants’ responses. The second round
gave the participants an opportunity to change their ratings of the level of importance in light of the new information received.
3.6.3 Consensus of the Delphi Method
The Delphi Method is a research approach used to gain consensus through a series of rounds of questionnaire surveys, usually more than one, where information and results are fed back to panel informants between each round (Mullen, 2003). Holey, Feeley, Dixon and Whittaker (2007) suggest that consensus can be determined by the following: the aggregate of judgments, a move to a subjective level of central tendency or alternatively by confirming stability in responses with consistency of answers between successive rounds of the study.
On the other hand, Hsu and Sandford (2007) argue that there is no agreement in literature on how consensus can said to have been attained. Dajani, Sincoff and Talley (1979:83) suggested that consensus is assumed to have been achieved when a certain percentage of responses fall within a prescribed range for the value being estimated.