Cluster III. Good governance: where the strategy aims at developing structures and systems of democratic governance that are participatory, representative, accountable and inclusive and
4.9 Validity and reliability of the study
This research is a multi-case study and a mixed methods investigation, which utilised qualitative and quantitative methods. The most prevalent approach in this study is the qualitative design. The issue of the validity and reliability of the study were dealt with in cognizance of the variations in which these issues are considered in both qualitative and quantitative designs.
Validity and reliability are the major technical considerations in quantitative and qualitative research (Babbie and Mouton 2001: 119). In the broadest sense, reliability and validity addresses issues concerning the quality of the data and appropriateness of the methods used in carrying out a research project. The quality of the data and the appropriateness of the methods employed are particularly important in social sciences, because of the different philosophical and methodological approaches to the study of human activity (Cano 2005).
The broad concepts of reliability and validity are applied to all research, because the goal of finding plausible and credible explanations for outcomes is central to all research. Morse et al.
(2002) stated that the concepts of reliability and validity as overarching constructs can be appropriately used in all scientific paradigms, because to validate is to investigate, to check, to question and to theorise. Lincoln and Guba (1985) cautioned that while all research must have
“truth value”, “applicability”, “consistency” and “neutrality” in order to be considered worthwhile, the nature of knowledge within the rationalistic (quantitative) paradigm is different from the knowledge in naturalistic (qualitative) paradigm. Consequently, each paradigm requires paradigm-specific criteria for addressing “rigour” (the term most often used in the rationalistic paradigm) or “trustworthiness”, their parallel term for qualitative “rigour”.
Morse et al. (2002) elaborated that whether quantitative or qualitative methods are used, rigour is a desired goal that is met through specific verification strategies. Morse et al. (2002) warned that, without rigour, research is worthless, becomes fiction and loses its utility. A great deal of attention should therefore be applied to reliability and validity issues in all research methods.
4.9.1 Validity and reliability in quantitative research
In quantitative research validity refers to the capacity of the research technique to encapsulate the characteristics of the concepts being studied and so properly to measure what the methods were intended to measure (Payne and Payne 2004: 233). Validity is mainly concerned with the extent to which the measuring devices would yield accurate results and capture the essence of what they are intended to represent (Babbie and Mouton 2001: 122; Leedy and Ormrod 2001: 31). Joppe (2000: 1) provides the following explanation of what validity is, in quantitative research:
Validity determines whether the research truly measures that which it was intended to measure or how truthful the research results are. In other words, does the research instrument allow the researcher to hit "the bull‟s eye" of the research object?
Researchers generally determine validity by asking a series of questions, and will often look for the answers in the research of others.
In quantitative research, reliability is mainly concerned with the extent to which similar results will be obtained if the study were to be repeated (Payne and Payne 2004: 195).
Research findings are considered to be reliable if they are repeatable, to the extent that repeated measures would yield constant results (Babbie and Mouton 2001: 119; Cohen, Manion and Morrison 2000: 117; Leedy and Ormrod 2001: 31; Payne and Payne 2004). Kirk and Miller (1986: 41-42) identify three types of reliability referred to in quantitative research, which relate to: (1) the degree to which a measurement, given repeatedly, remains the same (2) the stability of a measurement over time; and (3) the similarity of measurements within a
given time period. A high degree of stability indicates a high degree of reliability, which means the results are repeatable (Golafshani 2003: 598).
4.9.2 Validity and reliability in qualitative research
Validity and reliability are two factors which any qualitative researcher should be concerned about while designing a study, analysing results and judging the quality of the study. It is important for an inquirer to be able to persuade his or her audiences that the research findings of an inquiry are worth paying attention to (Lincoln and Guba 1985: 290).
Since validity and reliability are rooted in quantitative research, there is a need for validity and reliability to be re-defined for their use in qualitative research. Robson (2002: 170) felt that there is a need to find alternative ways of operationalising these terms that are appropriate to the conditions and circumstances of qualitative enquiry. Consequently, the quality of a study in each paradigm should be judged by its own paradigm's terms. In other words, these terms, as defined in quantitative terms, may not directly apply to the qualitative research paradigm. For instance, the question of replicability in the results does not directly apply in some form of qualitative research, as it does in quantitative research (Golafshani 2003: 597;
Winter 2000).
Patton (2001: 14) pointed out that, while the credibility in quantitative research depends on instrument construction, in qualitative research “the researcher is the instrument." Thus it seems when quantitative researchers speak of research validity and reliability, they are usually referring to research that is credible while the credibility of qualitative research depends on the ability and effort of the researcher. To emphasise this point, Robson (2002: 168) said that while the essential test of validity of findings in the natural sciences is that it has been directly replicated by an independent investigator, this approach is not feasible when a qualitative design is used and is highly questionable in real-world flexible (qualitative) designs involving people, as it can be complicated to created identical circumstances for the attempt to be replicated. The same point was raised by Marshall and Rossman (1999: 195).