6.10.1 Positivist versus Post-positivist criteria for validity
The traditional criteria for ‘validity’ find their roots in a positivist tradition. “Within the positivist terminology, validity resided amongst, and was the result and culmination of other empirical conceptions: universal laws, evidence, objectivity, truth, actuality, deduction, reason, fact and mathematical data to name just a few” (Winter, 2000, pp.6-7). It is within this tradition and terminology that quantitative research is traditionally defined.
Qualitative research, arising out of the post-positivist rejection of a single, static or objective truth, has concerned itself with the meanings and personal experience of individuals, groups and sub-cultures. ‘Reality’ in qualitative research is concerned with the negotiation or ‘truths’ through a series of objective accounts. Whereas quantitative researchers attempt to disassociate themselves as much as possible from the research process, qualitative researchers have come to embrace their involvement and role within the research. For quantitative researchers this involvement would greatly reduce the validity of a test, yet for qualitative researchers denying one’s role within research also threatens the validity of the research.
Winter (2000, p.7) believes that many of the allegations of invalidity from both sides can be attributed to “a failure to recognise the different purpose to which quantitative and qualitative methodology is suited”. Winter states that the only similarity between the two research methods is that, at some, point, questions will be asked and
research information will be collected. Quantitative research limits itself to what can be measured or quantified and qualitative research attempts to ‘pick up the pieces’ of the unquantifiable, personal, in depth, descriptive and social aspects of the world.
Unlike quantitative research, there are no standardised or accepted tests within qualitative research and often the nature of the investigation is determined and adapted by the research itself. There may not be any hypothesis or even findings as such. Instead the ‘validity’ of the research resides within the representation of the actors, the purposes of the research, and the appropriateness of the processes involved.
6.10.2 Validity in inferential quantitative research 6.10.2.1 Internal validity
According to Winter (2000), internal validity relates to whether the findings or results of the research relate to and are caused by the phenomena under investigation and not other unaccounted for influences. In essence, this essentially comes down to the issue of correlation and causality. Within quantitative research, causality must be established for the test to be valid, or indeed, of any use at all.
Winter (2000) states that researchers get around this by attempting to isolate the factors under investigation away from any confounding variables. The pollution of the results by other unwanted factors does not entirely invalidate the test, especially since they cannot really be fully isolated and stripped away, but a study with notable, and yet unaccounted for, influences is said to have low internal validity. According to Winter (2000), what the quantitative researcher is attempting to do is ensure that his/her explanation for the results can be the only possible or feasible one. There are many factors that pose a threat to validity in this case, such as the maturation of the individual in a longitudinal study, previous experiences of the individuals, 'lost data,' the effects of the test itself, or regression to the mean as a result of outliers or positively or negatively skewed values.
6.10.2.2 External validity
External validity is the extent to which the results can be generalised and thus applied to other populations. A test can have very high internal validity and very low external validity at the same time. Threats to external validity are similar to those for internal validity, except that the test itself is more likely to pose a threat as an alternative explanation for similar results.
Winter (2000) questions the value of the generalisability of quantitative research.
According to him, quantitative research attempts to fragment and delimit phenomena
into measurable or ‘common’ categories that can be applied to all the subjects or wider and similar situations. Hence, quantitative research, whilst able to claim validity for wider populations and not just merely samples, is restricted to measuring those elements that, by definition and distortion, are common to all.
Winter (2000) states that this raises the question of ‘at what cost’ are we exchanging accuracy for generalisability. Within the quantitative definition, an account may be judged ‘valid’, ‘replicable’ and ‘stable’ on the merits of its generalisability. Yet, according to Winter (2000), one could argue that generalisation in itself is neither valid nor accurate. It is likely that a ‘generalisable’ statement, whilst relating to all those to whom it is applied, may not actually describe the phenomena of any single case with accuracy, in the same way that a mean average score need not be the same value as any of the numbers of which it is an average.
6.10.3 Validity in qualitative research
Winter (2000, p.7) states that “some qualitative researchers have rejected the notion of validity, in any form, as entirely inappropriate to their work”. Other qualitative researchers, although they argue that the term validity is not applicable to qualitative research, have at the same time realised the need for some kind of quality check or measure for their research. As a result many researchers have espoused their own theories of ‘validity’ and have often generated or adopted what they consider to be more appropriate terms, such as ‘trustworthiness’, ‘worthy’, ‘relevant’, ‘plausible’,
‘confirmable’, ‘credible’ or ‘representative’.
According to Osborne (1994, p.180), findings “are valid to the extent that they resonate with the experiences of others who have experienced the phenomenon in question”. Stiles (1993, p.607) states that “validity concerns whether an interpretation is internally consistent, useful, robust, generalisable, or fruitful”.
A “valid” account in qualitative research generally refers to the presentation of an account that is sound and grounded in the research information. According to Osborn and Smith (1998) the aim is not to produce a single true account of the research information. The emphasis is on understanding by people, including the readers of the study, rather than on facts.
6.10.4 Validity in the current study 6.10.4.1 The literature study
A literature study was conducted in which both psychological and medical perspectives of ADHD were reviewed. Some critics may object to the fact that the
literature study was conducted prior to the collection of research information, a procedure generally avoided by phenomenologists lest the literature should threaten validity by biasing the researcher’s perceptions. However, Shantall (1996) argues that a literature study need not lead to theoretical bias and may in fact enhance understanding and empathy, by opening the researcher’s mind and revealing existing pre-conceptions.
Moustakas (1994, p.111) concurs that “review of the professional and research literature connected with the research topic and question is necessary when preparing to conduct a phenomenological study”. According to him, the investigator
“assesses the prior relevant studies; distinguishes their designs, methodologies and findings from the investigator’s own study; and indicates what new knowledge he or she is seeking and expects to obtain”.
6.10.4.2 Misinterpretation
A possible threat to validity in qualitative research is misinterpretation. This risk is increased by a reliance on a limited range of information resources. According to Stiles (1993), the solution is to ask many questions and expose oneself to multiple perspectives. Accordingly, several participants were interviewed in this study and their perspectives were probed in depth in order to negotiate meaning.
6.10.4.3 Testimonial validity
According to Stiles (1993) one check on an interpretation’s accuracy is to ask the people whose experience it purports to represent. Stiles refers to this as testimonial validity.
In the current study, the researcher decided not to follow this route. Stiles (1993) concedes that even when participants are consulted, there are circumstances in which literal agreement may not be expected. For example, a participant may not understand him/herself (Stiles, 1993). Or the interpretation may draw on language and concepts unfamiliar to participants. Further, the original experience that produced the text (or tape) no longer exists, and the participant does not necessarily have better access to it than the investigator does.
Giorgi (1989) states that a phenomenological analysis yields an explanation that is beyond the awareness of the participants themselves. According to him, phenomenologically, there is no doubt that the meaning for the participant that must be captured, but it is equally certain that the meaning must be taken up and be re- expressed in the language of the researcher’s discipline (sociology, psychology etc.).
Giorgi (1989) states that going back to the subject for confirmation of the disciplinary
expression is not really legitimate in this instance, even though it is a frequently practiced strategy. The reason is that only other members (and perhaps not even all, if theoretical orientations vary widely) of the same disciplinary community are in a position to critique or verify the expression. Giorgi (1989) uses the example of a survey based upon many manifest variables where the use of factor analysis will reduce the many variables to a few “basic” or “common” factors, which are then labelled according to the variables so reduced. How many survey researchers go back to the research subjects and ask them to approve of the reduction? It is almost never done because it is assumed that the subjects will not follow the logic or mathematics of the reduction process. Giorgi (1989) states that the reduction process in qualitative research from the subject’s everyday meaning to disciplinary meaning ought to be equally complex, and so the subject should not be expected to be the final arbiter if he or she cannot follow the reduction process. According to Giorgi (1989), this is not a matter of superiority as a human being, but a matter of specialization as with dentists or pharmacists.
Participants in the current study were, therefore, not invited to read and comment upon the interview transcripts or the interpretations thereof.
6.10.4.4 Consensus among researchers
Consensus among researchers is also one of Stiles’ categories for validity. Stiles (1993) states that consensus among researchers involves trying out interpretations on other investigators. The researcher’s supervisors acted as critics and mentors.
This offers readers the assurance that other investigators who were familiar with the raw research information found the proposed interpretation convincing.
6.10.4.5 Revealing of the researcher’s orientation
Revealing of the researcher’s orientation can also improve validity. Stiles (1993, p.614) states that “deep personal involvement and passionate commitment to a topic can bring enmeshment, with its associated risks of distortion, but they can also motivate more thorough investigation and a deeper understanding”.
Revealing an investigator’s personal involvement and commitments and the process of the investigation allows readers to incorporate the investigator’s part in the story into their understanding and to adjust their understanding to compensate for the investigator’s biases.
As a researcher, my mind was not a blank slate with regard to this research. In fact, my values, interests, and intimate familiarity with the research problem were the origin of my motivation for this study. My interest in the research topic was mostly
derived from a personal perspective, as my son was diagnosed with ADHD a number of years ago. As a parent of a child with ADHD, I frequently interact with educational personnel and I am continually attempting to make sense out of these interactions.
As a parent, I know how daunting the interactions with neurologists, paediatricians and therapists can be. I can relate to the familial stress that occurs when parenting such children, and I have often wondered how other parents make sense of similar experiences.
Is it a good thing that I have experienced many of the feelings that my participants have? Feelings, if handled appropriately, can be an essential aid in doing qualitative research (Rosaldo, 1989). I believe my feelings were a potent indicator of participants’ feelings and provided a source for speculation. Likewise, my feelings also helped me to formulate questions to elicit deeper responses from the participants. In other words, my emotional reactions were a source for research hunches and allowed me to formulate follow-up probes. When expressed appropriately, my feelings also provided a powerful way of developing rapport with the participants.
6.10.4.6 Coherence of the report
Validity is also established by the coherence of the report, which can best be judged by the reader. Coherence refers to the internal consistency and comprehensiveness of the account. The research should enhance understanding of the phenomenon under study. This study confirmed many aspects of the literature review, but also transcended it by offering insights gleaned from entering the life-world of the sufferers themselves.