Glossary of Terms
Chapter 4: Methodology
4.7 Delphi survey
134 (represented using PowerPoint). In the case of ABMs, such representations are Unified Modelling Language diagrams, i.e. sequence diagrams and class diagrams. In the case of BNs, such representations are influence diagrams and tables of judgments. The process steps followed in interviews are:
1. Present the conceptual model representation using my own language;
2. Discuss how the framework or model would be used or operationalised; and finally 3. Invite comments and criticism; and
4. Take notes, and identify agreements or disagreements to use in further interviews.
135
• To generate qualitative data to understand the wider context of vulnerability of water services in PIC towns;
• For preliminary identification of causal relationships in relation to barriers and enablers;
• For specification of research questions in the sense that the learning that occurs as part of the process is likely to raise new questions and uncover problem areas.
The Delphi survey was email based, employing the web survey software (SurveyMonkey 2009), in order to allow remote access to stakeholders, and this was critical in enabling the survey to be carried out with such a diverse array of stakeholders.
4.7.1 Sequences of events
The steps in the survey process are, loosely as per Figure 4-6:
1. Identification of participants, as well as invitation and setting of expectation;
2. Brainstorming; with participants identifying and describing various factors;
3. Classification of responses into a number of categories / factors;
4. Validation of the classification;
5. Selection by each participant of the most important factors; and 6. Validation by participants; if ok continue, otherwise return to step 6;
7. Ranking of the most selected factors;
8. Validation by participants; if ok continue, otherwise return to step 8;
9. Re-ranking of the most selected factors;
10. Validation of rankings by participants; if ok stop, otherwise return to step 10.
136 Figure 4-6: Delphi dialogue process as it is applied in this study
This process is designed so that the outcome emerges from the interactions of the participants, but it is unavoidable that there is a bias from the researcher who facilitates the process as well as provides a categorisation. To limit this bias, the process has elements of:
• Iterative participant validation of the categorisations, and rankings;
• Participants being invited to provide feedback on the process; and the facilitator adapting the process when it is found to be appropriate.
For the survey questionnaires, email was used in conjunction with the collection of responses using designed forms in online survey software (SurveyMonkey 2009).
All written communication with the participants was based on the principles of simplicity, avoiding ambiguity and clarity of language. Clear and simple communication is critical for ensuring a high response rate as well as to reduce any risk of bias by misunderstanding. Other efforts to ensure a high response rate were:
137
• Avoiding technical terminology to the extent possible so as not to exclude people with non- technical backgrounds;
• Support by a communication expert, to make any material as accessible as possible to participants from a non-English speaking background;
• Keeping the momentum going throughout the process which occurred over several months, by updates on progress, even when there was no questionnaire sent out;
• Making every effort to simplify forms in order to reduce the time taken to fill them in;
• Inviting participants to be critical of the process, and adapt the process when it is perceived to be appropriate to do so;
• Being clear about commitments in terms of time.
4.7.2 Selection and invitation of participants
For the purpose of this Delphi survey, participants were sought that possess cross-cutting experience with issues in the PICs urban water sectors. The Delphi dialogue traditionally starts with a mapping of knowledge hubs in the area, and subsequently maps out potential participants, which are then ranked based on a number of criteria. This process was not strictly followed but three wide categories of individuals were identified that play vital roles in water development projects: Funding agency representatives, Local stakeholders, and Experts.
Naturally, there are also a number of subcategories relating to each of these Stakeholder groups, as can be seen in the Table 4-3.
The anonymity of participants in the survey needs to be ensured, but some basic statistics about them can be disclosed. Out of the 47 participants, a mere 7 were female (it is a male dominated sector); 10 came from PICs, 16 were international experts (from outside of Australia), and 21 were experts were based in Australia. 29 of the participants had extensive experience of working in the water sector in the PICs, whilst the remaining participants worked with PICs indirectly, such as for example those representatives for the Asian Development Bank, the
138 World Bank, the European Commission, the Global Water Programme, and the International Water Association; and a small number of international researchers on similar issues in other locations, such as in the Caribbean and in Africa. Experts with relatively limited experience in the PICs were put into a reference, base case group; a sub-category of the Experts group.
Table 4-3: Stakeholder panels in the Delphi study
Local stakeholders Experts Funding agencies
Water utility interests Engineers Development banks
Government departments or parliament Development specialists Government aid agencies
NGO interests Natural scientists International aid programs
Business interest Socio-cultural experts Research foundations
Household interests Process facilitators Donation based aid organisations
Landowner interests Volunteer organisations
Agricultural and subsistence interests Industry association
Ten Local stakeholder participants from PICs in the Delphi dialogue were selected on the basis of access and willingness to participate; mainly from the categories of Water utilities and Government departments. The countries represented were Kiribati, Tonga, Cook Islands, Fiji, Papua New Guinea, Tuvalu, Tonga, Samoa, Solomon Islands and the Federated States of Micronesia, with one participant from each country. Attempts were made to achieve a larger number of participants from PIC backgrounds, but this was not successful. However, given the smaller number of individuals in the world with cross-cutting experience with issues in the PICs urban water sectors, this may be acceptable.
The bias towards the two first sub-categories in the ability to identify and successfully invite participants into the process may in fact represent a common bias in the design of water development projects. The reason for having this bias in the selection in fact relies on two facts, i.e. the:
139
• Limited scope of the social networks of people asked to nominate participants into the other Local stakeholder group’s sub-categories; and
• Reliance on the computer literacy and Internet access of the participants.
There were twenty-two participants within the Expert panel with representatives from each of the sub-categories so the Expert panel is in fact rather diverse. There were a total of fifteen participants in the Funding agency panel, with at least one participant from each of the sub- categories except for from a Volunteer organisation.
Once a number of potential participants had been identified, they were invited into the process.
This was done via phone conversations and by sending a document describing the background and aim of the study, time requirements and scope, as well as the expected outcomes in realistic terms. Most of the invited participants agreed to take part in the dialogue.