Glossary of Terms
Chapter 3: Methods Review
3.1 Information collection
3.1.1 Interviews
The method of undertaking interviews is one of the most commonly applied approaches for collecting information in qualitative research and while the process is time consuming, the process provides rich and contextual descriptions, even though the results are dependent on the
62 skills and aptitude of the interviewer (Byrne 2001). Within the qualitative research areas, it is used in case studies (Flyvbjerg 2006), in action research (Carr 2006), in grounded theory studies (Glaser and Strauss 1967) and in ethnographies (Myers and Newman 2007). Myers and Newman (2007) however note that the reporting on how interviews were undertaken is often poorly described. Interviews may be undertaken in a structured, semi-structured or unstructured manner (Cohene and Easterbrook 2005). It is also noted that the interview process can be very rewarding in terms of the individual learning for the researcher (Byrne 2001).
The structured interview is carried out by using a number of pre-defined questions, and in a semi-structured interview the interviewer adapts to the situation (using what is referred to as
‘tactics’) and delivers the questions in a more exploratory and dynamic manner (Cohene and Easterbrook 2005). Tactics employed in semi-structured interviews may be the inclusion of narratives, the use of scenarios or role-playing, card sorts, free listening or paired comparisons and providing the participants with more options for communication (Cohene and Easterbrook 2005). In unstructured interviews there is no pre-defined agenda providing much more freedom in terms of the topics that may be covered (Cohene and Easterbrook 2005). It is noted however that Byrne (2001), who comes from the perspective of medical research, provides a different definition of these concepts, making the distinction only between structured and unstructured interviews, where the definition of the structured interviews stays the same but the definition for the unstructured interviews is when you ask open questions. Byrne’s (2001) definitions are clearly less encompassing of the range of interviewing options. Myers and Newman (2007) however add another category, the group interview where one or more participants are interviewed by one or more interviewers; and this may be structured, semi-structured or unstructured.
Another approach is provided by Dick (2002b) who suggests carrying out the following stages:
1. Building rapport: providing an introduction of who you are, what you are doing, who gets the information and what the purpose of the interview is.
63 2. Ask an opening question without being specific; as far as possible simply introduce the
topic and invite a response.
3. Keeping the informant talking for about 45 minutes to an hour without asking specific questions.
4. Towards the end of the interview, ask probing questions, and ask for a summary of the key points.
5. Write up the interview while it is still fresh in memory; and bullet points are adequate rather than a detailed report.
Dick (2002b) then suggests that this process is carried out for a selected number of participants and where the selection criteria provide maximum diversity and involves all interests. The interviewer also keeps track of agreements and disagreements in the responses and towards the end of the interviewing series, the interviewer can use the disagreements to challenge the interpretations made by participants. Dick (2002b) makes the point that with this process, information is coming from the participants and not the interviewer (and is hence participatory), and through probing questions, it involves the participant in the interpretation. By focussing on the disagreements in interpretations, the process is also dialectic by identifying differences in the participants’ mental models.
There have been a number of published studies using interviews in the urban water sector.
Brown (2005) interviewed 60 experts across multiple sectors to explore impediments to sustainable management in the Sydney water sector. Kayaga (2008) used interviews as a tool within a soft systems methodology to undertake performance measurement in the Uganda water sector. Who Kayaga has interviewed or how he has interviewed them is however not described in the article. Marlow and Humphries (2009) have interviewed water sector professionals in Australia to explore how sustainability issues can be operationalised within asset management.
These were recorded and carried out in a face-to-face fashion where participants could choose either a structured or semi-structured approach. Probing questions were used to seek clarity but not to invite participants to make interpretations. Dray and colleagues (2006b) used an initial survey for selecting relevant groups and in this way identifying leaders or active community
64 members who were then selected for semi-structured interviews. The semi-structured interviews had three parts: 1) inviting participants to interpret photos, 2) cognitive mapping (i.e. mind mapping), and 3) using a card game. Based on the results of the first round of interviews, an Individual Activities Survey (a structured interviewing technique) was carried out asking questions about behaviour and activities.
Myers and Newman (2007) also point out a number of pitfalls in interviewing, such as:
• Using ambiguous language providing ambiguity in interpretations and consequently in responses;
• The tendency to steer the interviews and thereby skewing the information, and sometimes even socially constructing the results in the process of interviewing;
• Providing a bias in interpretation towards responses of those participants that are articulate or perceived as important;
• It may be difficult or even impossible to gain entry to all levels of an organisation; or all relevant potential participants;
• There may be a lack of trust, meaning that the participant may not divulge sensitive information;
• There may be insufficient time, meaning that the results are often incomplete or truncated.
Myers and Newman (2007) also stress the importance of ethical standards, i.e. getting permissions, respecting people and fulfilling commitments to individuals and organisations.
They also note that structured interviews are suitable for well-defined problem areas, while unstructured interviews are preferred for topics with high levels of uncertainty and ambiguity;
and semi-structured interviews are suitable for the areas in between (Myers and Newman 2007).
65 3.1.2 Delphi method
An interesting method for group knowledge elicitation that has been applied in the natural resource management context (see Rixon et al. 2007b), among other contexts, is the Delphi method which has been described as follows:
Delphi may be characterised as a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem (Linstone and Turoff 2002: 5).
Okoli and Pawlowski (2004) describe a remotely conducted Delphi survey (e.g. web based survey) as follows:
1. Selecting panellists; which involves
2. Developing a knowledge resource map and a spreadsheet, and then;
3. Populating this spreadsheet with names as potential participants;
4. Ranking participants based on a relevant criteria; and selecting appropriate panels based on the range of participants’ backgrounds;
5. Categorising and inviting participants (target size is 10-18 per panel).
6. Collecting data using questionnaires and sending these out by mail or email
7. Brainstorming – identifying a range of issues and factors concerning the topic at hand;
8. Narrowing down – inviting the participants to select a smaller number of issues in order to reduce the number of issues to something manageable; and
9. Ranking – inviting the participants to rank the issues in order to reach a ranked list of issues for each panel.
10. Within each iteration, it is useful to ask participants to validate the process, interpretations and the results; as discussed previously this invitation to participate in the sense-making is important for extracting the mental models of the participants;
66 11. Analysing the results; identifying agreements and disagreements within each panel and
thereby providing a list of mental representations of the problem.
Historically, the Delphi method was originated by a number of studies undertaken by the RAND Corporation in the 1950s, and it has been applied within a large number of areas for such as:
forecasting; identification of a research topic; specification of research questions; identification of a theoretical perspective for the research; selection of factors of interest; preliminary identification of causal relationships; and the creation of a common language for dialogue (Linstone and Turoff 2002; Okoli and Pawlowski 2004).
Delphi survey is a qualitative research approach that generates rich datasets based on narratives and examples provided by participants. Then, a Delphi process explores patterns in responses and allows participants to validate patterns identified by researchers. The Delphi method can also be applied within the Action Research area (Dick 2002c), and in this case the expectation is that the participants decide to take action based on the understanding that has been gained. For instance, in an ideal case, the Delphi method may start a constructive discussion about possible water development projects that will address the factors identified through this study.
The Delphi method was deemed to have the following advantages over more traditional questionnaire-based approaches by providing (Okoli and Pawlowski 2004):
• A typically higher response rate;
• More opportunities for validation and follow up on responses; via
• Direct validation of the researcher’s synthesis and classifications by participants; and
• The ability to explore and clarify ambiguous responses;
• Fewer limitations because there are less concerns about statistical significance of results;
• Better ability to explore the diversity in responses and identify disagreements, which may help in focusing research questions;
67
• Providing a common language to emerge from the interactions between diverse stakeholders and this may allow for further cross-boundary discussion and discourse;
• Allowing issues to emerge instead of being prescribed, as is the case when using a questionnaire.
Okoli and Pawlowski (2004) also describe how the Delphi method is related to a word in French, bricolage, which means ‘to use whatever resources and repertoire one has to perform whatever task one faces’. The Delphi method achieves this because of its flexible nature, and is hence a good starting point for finding the information and understanding that already exists.
There are a number of different methods for undertaking this type of group communication:
conference telephone call, committee meeting, formal conference or seminar, workshop, email or internet (Linstone and Turoff 2002; Rixon et al. 2007b).