Part One
Chapter 4 Methodology
4.3 Data Collection: Triangulation 11
4.3.4 Comparing the Techniques
Table 4.3 below compares each approach, and Table 4.4 (cont.) illustrates their strengths and weaknesses. Each method has been selected to collect a specific aspect of the interviewees’ perceptions about accountability. When taken together the three methods compensate for the weaknesses in each individual method.
Research Method Why selected? Why appropriate? Data Collected Semi-structured
Interviews
Semi-structured interviews allow for open ended questions and relatively free flow of ideas (as opposed to structured interviews that require a response to pre- established questions and codes (Fontana and Frey, 1994). When such questions and codes are not immediately obvious, such as in research on values, beliefs and perceptions Morse (1994 p224) states that unstructured interviews should be used (Morse, 1994).
Interview data represents one form of ‘speech event’ (Spradley, 1979) which is an organised sequence of ideas of the world (McCracken, 1988) prompted by the researcher’s questions.
Document Analysis Documents and text represent another source of information (beyond the spoken word) about the construction of reality (Hodder, 1994). While such documents require the researcher to interpret them, they remove some of the bias potential in the interview situation. Interpretation will be guided by the themes and issues identified in the interview stage.
Documents, the official sanctioned texts of organisations that are not produced for the purposes of the research and that relate to accountability, will be coded with the interview transcripts.
Direct Observation Non participant observations provide a third alternative source about how reality is being constructed – the discussion between members, and between members and their stakeholders.
It presents the most naturalistic of the data collection methods, so removing as much as possible observer effects from the data collection (this is why participant observation has been rejected).
This will allow further understanding of the themes and issues identified in the previous two stages.
Detailed field notes and memos will be taken while observing interactions between members of the voluntary organisation when discussing stakeholders and accountability, and in meetings between members and their stakeholders.
Table 4.3: Types of Data Collection Techniques
Research Method Strengths Weaknesses Semi-structured
Interviews
Provides in-depth rich data;
Gives some idea about how respondent views the world through the responses they chose to give;
Transcripts are relatively easy to analyse and compare;
The interviewer can seek responses to questions that are of interest to the research;
Privileges the spoken word and expression, and acknowledges the importance of speech as a means of constructing realities.
Prompts during interviews can control the information given by respondents (leading questions);
Respondents give the information they think is required; and Interviewers tailor questions based on their preconceived notions of the respondent (McCracken 1988;
Kvale 1996);
Few standard techniques so reliance on the experience and judgements of the interviewer (Kvale, 1996);
Focuses on thoughts and experiences at the expense of action (Kvale, 1996);
Is an unnatural form of communication as respondents required to demonstrate competence in the role the interviewer casts them in.
Interview data represents a particular representation of the respondents’ construction of reality (Dingwall, 1997).
Success based on intangibles such as rapport, level of respondent apprehension (Spradley, 1979);
Document Analysis One form of constructed reality captured in time and space;
Are formal snapshots of an organisation’s reality (Miller, 1997);
They are enduring and can give historical insight (Hodder 1994)
Need to contextualise the document (its history, audience, author) in order to interpret it sensibly;
Documents’ meanings are not stable and will be interpreted differently by different people (Hodder, 1994; Miller, 1997);
Table 4.4: Strengths and Weaknesses of Data Collection Techniques
Research Method Strengths Weaknesses Direct Observation Able to view the subjects
interacting with each other;
Subjects do not have to appear as rational to the researcher as in an interview (Dingwall 1997);
Non-interventionalist so removes as much as possible observer effects;
Allows insight as much as possible into private settings;
Unstructured so subjects are free to reveal what is important to them (Adler and Adler, 1994);
Validity of technique questioned because observers rely on their perceptions of what is happening (in this research strategy the observations will be guided by themes and issues from the interview data and document analysis);
Reliability in question because can not verify that the events observed are not random (Adler and Adler 1994);
Potential bias as only view instances that support researchers ideas (Lofland, 1971);
As a non participant, the researcher is an ‘outsider’ so not so privileged with access to private thoughts (Spradley, 1980).
Table 4.4 (cont.) 4.3.5 Piloting
Once the interview schedule was formulated, it was tested with experienced researchers to identify poorly worded questions and bias. It was then tested with four leaders in two voluntary organisations that fulfilled the case study selection criteria.
This was to determine whether the information sought by the research would actually be obtained by the research schedule. The pilot voluntary organisation leaders were also asked to identify documents that may be of use to the research. Observations of non-verbal cues were also completed for the pilot interviews. This provided a useful starting point for gathering the three sources of data needed for triangulations.
Berg (1998) suggests that this two-step process, involving experienced researchers and those familiar with the study’s subject, should ensure the interview schedule is as well designed as possible.
The interview transcripts, documents and observation sheets were coded in order to develop a standardised process before the proper data gathering began.
The final interview schedule, and post-interview coding sheet, can be viewed in Appendix C.