CHAPTER 2: NON-VIOLENCE, THE HISTORY OF VIOLENCE AND THE ROLE OF THE CHURCH IN ZIMBABWE
5.7 Objectivity, validity and reliability in qualitative research
142
commencement of the programme. In order to establish whether participants changed their behaviour during the course of the programme, there needs to be both a pre-test and a post- test. The design involves observations and measurements before implementation of the intervention (pre-test), followed by the intervention and administering of observations and measurements again (post-test). “A ‘statistically significant difference’ means that any differences that are observed between a mean pre-test score and a mean post-test score is not due to chance factors but (most likely) indicate ‘true differences’”, they argue (2001, p348).
Pre- and post- test observations and measurements in this research are designed to meet this criterion, and are reinforced by experimental and control groups as explained under ‘classical experiment’ earlier in Section 5.4.
143 Credibility
Credibility is the compatibility between the constructed realities that exist in the minds of the respondents and those that are attributed to them, and is achieved through persistent observation and prolonged engagement in the field until ‘data saturation’ occurs. While circumstances did not allow me to stay with the participants throughout this research, enough time was spent with the participants to understand them at personal level. Also, to compensate for the time that I was not in Zimbabwe, I carried out interviews with community leaders to get independent opinion on changes they observed on the youths. Added to this were ‘conflict diaries’ that the youths themselves wrote which gave insight into how they reacted in conflict situations. The concept of conflict diaries was borrowed from Henerson et al. (1987) who call them ‘self-reports’. Self-report procedures, they posit, represent the most direct type of attitude assessment and should be employed unless one has reason to believe the people whose attitudes he or she is investigating are unable or unwilling to provide the necessary information.
Transferability
Transferability refers to the extent to which findings can be applied in other contexts or with other respondents. In qualitative research observations are defined by the specific contexts in which they occur, a situation that does not guarantee relevance in one context and time frame with a different context. Guba and Lincoln (1984), quoted by Babbie and Mouton (2001, p277- 8), assert that in qualitative research, transferability rests on those who wish to apply it to the receiving context, and this is enhanced when the researcher collects sufficiently detailed descriptions of data and report them in sufficient detail and precision to allow judgments about transferability to be made by the reader. This is also enhanced by purposive sampling, described in detail in Section 6.2 of this thesis. Transferability is one of the main aims that this study sought to achieve in two ways: first, to other youths in Zimbabwe, particularly the youth militias who are among the main perpetrators of violence in the country, and secondly, to the churches’ programmes in grooming youths that have more non-violent options when in a
144
conflict as espoused by AVP. This, it was anticipated, would enhance the churches’ efforts in building a peaceful Zimbabwe.
Confirmability
This refers to the degree of authenticity of the research to which the findings are the product of the inquiry and not the biases of the researcher. Babbie and Mouton (2001, p278), quoting Guba and Lincoln (1984) refer to a confirmability audit trail, i.e. an adequate trail should be left to enable the auditor to determine if the conclusions, interpretations, and recommendations can be traced to their sources and if they are supported by the inquiry. They categorise data into six classes: raw data such as recorded video tapes, written field notes, documents; data reduction analysis products such as summaries and condensed notes; data reconstruction and synthesis products that include research findings, conclusions and final report; materials relating to intentions and dispositions that would include inquiry proposal and expectations;
and, instrument development information such as forms, questionnaires an surveys. For this study all these aspects are provided, some as annexure.
Validity
Jupp (2006, p311) defines validity as the extent to which conclusions drawn from research provide an accurate description of what happened or a correct explanation of what happens and why. He identifies three distinctive features: first, validity of measurement which asks whether a research instrument such as a questionnaire measures what it purports to; second, validity of explanation – also known as internal validity which involves asking whether the explanations and conclusions derived from research are the correct ones for the specific subjects or contexts that have been studied; and third, validity of generalisation – also known as external validity – which involves asking whether the conclusions drawn from a particular study can be generalised to other people and other contexts. This is very similar to transferability and dependability as espoused by Babbie and Mouton above.
An important procedure to increase validity is triangulation of data. Triangulation refers to the combinations and comparisons of multiple data sources, data collection and analysis
145
procedures, research methods, investigators and inferences in order to converge and cross- validate findings from a number of sources (Brewer and Hunter, 2006; Babbie and Mouton 2001; Greene, 2007; Tashakkori and Teddlie, 2009; Flick, 2009). When simplified, this means observing the same issue from at least two different points to maximise the validity of research.
Flick is among scholars who identify four different forms of triangulation: triangulation of data which combines data drawn from different sources at different times in different places or from different people; investigator triangulation which is characterised by the use of different observers and interviewers to balance the subjective views of individuals; triangulation of theories meaning approaching data from different theoretical angles which are used side by side to assess their usefulness; and, methodological triangulation which proposes a combination of different (quantitative or quantitative) procedures in order to transcend the boundaries of both methodological approaches. Of these, this study will utilise mainly two – triangulation of data as shown by the variation of sources, and methodological triangulation in employing mixed research methods. Triangulation of data is in line with what Flick (2009) refers to as systematic triangulation of perspectives where different research perspectives in qualitative research are combined with one another in a targeted way to compliment their strong points and to illustrate their respective limitations.
Reliability and dependability
Reliability refers to whether a particular technique would yield the same result each time when applied repeatedly to the same object. Babbie and Mouton (2001, p119-22) argue that reliability is a function of consistency, and give some techniques of dealing with problems of reliability to include:
I. Test-retest method, where the same measurement is done more than once and the same response would be expected. If answers vary, the measurement methods may, to the extent of that variation, be unreliable. In the research, the pre-test post-test technique that will be done three times with the experimental group contribute will provide an important case to test validity. However, because attitudes are expected to change, the shift in attitude will prove or disapprove the effect of the stimulus. The pre-
146
test post-test in the control group will also test validity. The results of these tests are discussed in Part 4 of this thesis. A logical relationship between variables will be approximated, a system Babbie and Mouton refer to as ‘construct validity’.
II. Using established measures which have proven reliability in previous research. While there are not other measures that test attitudes towards violence, this research borrows from the Ron Kraybill Conflict Style Inventory (2005), as explained in Section 10.2.1.
III. Where unreliability is generated by deficiency on the part of research workers, training and oversight systems can be used to improve reliability. Chapters Six and Seven of this thesis provide much detail on the data collection methods and analysis used, respectively.