List of Acronyms
Chapter 2: Research methodology
2.5 Validity and trustworthiness
Like all research methods, the narrative/life history method is not without problems, many of them related specifically to the method itself. Webster and Mertova (2007) identify the following risks in narrative research:
1. Intersubjectivity (slipping into a narrative plot, without the appropriate reflection and analysis)
2. Smoothing (a positive response is invoked despite the data)
3. External constraints (cultural, institutional, time, state of mind of the participants).
Some of these are obviously potential problems in any research, using any method. However, narrative/life history research has its own specific risks, discussed below.
2.5.1 Memory
Life history research is about asking people to talk about, and thus recall, their lives or certain aspects of these. This raises the issue of memory: “How far can you ask people to recall their past on the basis of memory? What sort of biases will their memory be subject to?
Is recollection of the past also affected by current or present experiences and circumstances?”
(Dex, 1991, p.2). Cortazzi and Jin (2006) also raise this issue: “...remembered accounts may be shaped by distortions of time, by rationalisations from present perspectives, and by changes in the teller’s sense of identity” (p. 32).
Portelli (1991, cited in Elliottt, 2005), however, argues that although life stories may contain many factual errors, this adds meaning to the accounts: “Oral sources....are not always reliable in point of fact. Rather than being a weakness, this is however their strength: errors, inventions and myths lead us through and beyond facts to their meanings” (Portelli, 1991, p.2 quoted in Elliottt, 2005, p.26); and, after all, in the narrative method, meaning is what it’s often about; it’s a main reason for using the method in the first place.
Where accuracy of recall is really important, this can be influenced by technique - the structure of a questionnaire, the ordering of questions.
2.5.2 Bias
Bias is clearly a concern in any research undertaking. As seen in the discussion about memory, the issue of bias has frequently been raised within the field of narrative inquiry in relation to the participant. However, it relates to the researcher, as well. Narrative, by its very nature, is subjective - which stories should be included, which left out - and can be distorted to create “a happy ending”. There is also the issue of concealed/distorted messages/
perspectives (Webster & Mertova, 2007). There is a danger in treating the stories uncritically.
It is thus important to be aware of any underlying motives of the narrator. Watson (2006) emphasises that the researcher must be aware of multiple realities and the ‘political’ aspects of explanation - which versions is being privileged by the teller. Cortazzi and Jin (2006) argue that the researcher’s identity is extremely important in narrative research. Researchers can (and do) impose their own understandings or interpretations, even restructuring stories.
They give an example of researchers restructuring indigenous people’s stories to, for
example, create a chronology that was not originally there, thus missing crucial issues raised by the stories. They warn that research participants often know of this tendency of
researchers, and quote one community member as saying, “That’s not what really happened.
That’s only what we tell the anthropologists so they’ll go away and leave us in peace” (p.32).
Again, as with memory, the issue of subjectivity and bias is not always a weakness -
narratives “illuminate the real life experience of learners and at the same time are sensitive to the broader connections to the individual’s worldview” (Webster & Mertova, 2007, p.20).
Bias tells us something.
In my own study, I was not particularly concerned about the accuracy or truthfulness of what people said; although, where possible, I did check within and between stories to pick up any errors of memory or fact (and found very few). For my purpose, I was interested rather in how people interpreted various experiences, in their analysis of things. Thus I did not find
2.5.3 Trustworthiness
All researchers need, as Lather (1991) argues, to ensure that their research does not take the concept of subjective research too far, does not simply become the expression of the
researcher’s own complex internal narrative (or political project). It thus requires some kind of validation. Lather (1991) uses Guba and Lincoln’s (1981) minimum requirement of triangulation, reflexivity, and face validity/member-checks, but adds to this a fourth
requirement in testing for validity, that of catalytic validity. So although Lather (1986) makes a strong argument for openly ideological research, she insists that this be rigorous. If we are not rigorous, then our work will not be considered legitimate; but also, “we will fail to protect our work from our own passions, and our theory-building will suffer” (p.78).
Within narrative research, one of the early writers on the method, Plummer (1983), argues that because the method focuses on the subjective reality of the individual, the limitations of the method include issues of reliability and validity. Since then, many writers have addressed these issues, and there is some disagreement in the field.
Webster and Mertova (2007) argue that you cannot use the same criteria in narrative research as in other methods. Reliability, as it is generally understood, cannot be applied to narrative inquiry, since the same questions asked to different people will get different answers, because they each had different experiences. Thus reliability, as the consistency and stability of the measuring instrument, does not apply (Webster & Mertova, 2007). In narrative research, reliability thus relates to the dependability of the data, whilst validity relates to the strength of data analysis, the trustworthiness of the data, and the ease of access to the data (Polkinghorne 1988 cited in Webster & Mertova, 2007). Webb (2006), however, is critical of attempts by qualitative researchers to reconfigure reliability and validity to fit their purpose, and proposes that the underlying principles of trustworthiness, relevance and honesty are more useful tools;
as is the concept of ‘good’ (i.e. useful) research.
Polkinghorne (1988, cited in Webster & Mertova, 2007, p.22) suggests we abandon
reliability altogether, and rather judge using trustworthiness, whilst Clandinin and Connelly (2000), also arguing that reliability, validity and generalizability are not really appropriate for
narrative inquiry, suggest that good narrative has an explanatory, invitational quality, authenticity, adequacy and plausibility. Good inquiry, on the other hand, comes from wakefulness (discussed further below).
Huberman (1995, cited in Webster & Mertova, 2007) suggests the following measures:
• access (to context, process and construction of knowledge; and to research notes, transcripts and data);
• honesty (asking questions like, how can one establish confidence in the ‘truth’ of the findings? How can one determine the extent to which the findings are applicable elsewhere? Are any patterns emerging? Are the conditions of the inquiry influenced by biases/motivation/interests of the inquirer?);
• verisimilitude (the stories should resonate with the experience of the researcher; they should appear to have a level of plausibility; they should be confirmed by like and other events, in Webster and Mertova’s (2007) understanding, as discussed above);
• authenticity (the researcher has provided enough information to convince the reader that the story is told in a serious and honest way; there should be narrative coherence)
• familiarity (the familiar needs to be made strange, so that it is not ‘dulled’ and you do not see particular things).
• transferability (you must be able to compare the story with other situations); and
• economy (there is often a huge amount of data; thus one needs to be efficient and economical in a way that will not compromise the integrity of the data or findings).
I tried to use all of these measures in undertaking this study, as will hopefully be apparent in the following chapters.
2.5.3.1 Triangulation
Triangulation refers not just to multiple measures, but also to multiple data sources, methods, theoretical schemes. The point of triangulation is to look for counter-patterns as well as convergences (Lather, 1986, 1991). Babbie and Mouton (1998, p.285) suggest, as well as getting the person who told the story to read the transcript, and provide an
• comparing the data with other sources collected by other methods;
• comparing with official records;
• comparing with other informants.
Not surprisingly, some of the literature from a more deconstructivist approach rejects the applicability of triangulation to narrative (eg. Webster & Mertova, 2007), on the grounds that it is almost impossible within the method (given that one is asking individual people to tell their individual, entirely subjective, entirely unique, stories), and because triangulation is about searching for an ultimate truth that does not exist.
Given that I am working within a critical paradigm, and mindful of Lather’s warnings about not protecting my research from my passions, I have tried to use triangulation as far as possible. Although I have only used one method (in-depth interviews), I did get the participants to read and check the transcribed interviews, and checked across different people, looking for patterns (including counter-patterns). Where other interviews with the same people are publically available (as is the case with some of the participants), I also checked my interviews against these; and I checked those experiences/events recounted in the narratives against other accounts of the same experience/event. I looked at the stories from the perspective of a number of different theories (hegemony and counter-hegemony;
different learning theories; Badiou’s theory of the event); and also used multiple theories to check whether the one I’m interested in - Badiou’s - can add anything new.
2.5.3.2 Reflexive subjectivity
Reflexivity relates to the researcher, and how s/he thinks hard about his or her assumptions.
“...[R]esearcher reflexivity is...the capacity of researchers to acknowledge how their own experiences and contexts (which are usually fluid and changing) inform the process and outcomes of inquiry” (Etherington, 2006, p.81) Obviously, this is a critical issue if one accepts the notion that all research is fundamentally ideological: “Emancipatory social theory requires a ceaseless confrontation with the experiences of people in their daily lives in order to stymie the tendency to theoretical imposition which is inherent in theoretically guided empirical work” (Lather, 1986, p.67). Reflexivity must thus be built into all stages of the
research, from deciding on what to study, to how to do it, to actually doing it, and finally to its final representation (Etherington, 2006).
Many writers, across paradigms, agree that reflexivity on the part of the researcher is an essential part of doing ‘good’ (rigorous) research. Clandinin and Connelly (2000) call this
‘wakefulness’: “A language of wakefulness allows us to proceed forward with a constant, alert awareness of risks, of narcissism, of solipsism, and of simplistic plots, scenarios, and unidimensional characters” (p.182).
Part of being reflexive is, as I have already discussed, being open about our own position, and accepting the possibility that this might bias our research. It also means asking ourselves hard questions about how and why the research is being done. Lather (1991) argues that researchers needs to operate within a conscious context of theory building, and constantly probe and question theoretical assumptions, asking questions such as “Where are the weak points of the theoretical tradition we are operating from within? Are we extending theory?
Revising it? Testing it? Corroborating it?” (p.67). We have to guard against allowing our theory impose or subsume counter-patterns, so we need to empirically ground our theory:
“Empirical validation requires a critical stance regarding the inadequacies of our pet theories and an openness to counter-interpretations” (Lather, 1986, p.76).
Webster and Mertova (2007) also insist on the importance of asking ourselves questions.
They cite Strauss and Corbin (1998) as saying that there are three different kinds of questions we need to ask:
1. Sensitising questions - what is going on here? (Issues, problems, concerns). Who are the actors? How do they define the situation? What does it mean to them?
2. Theoretical questions - what is the relationship between one concept and another?
How do events and actions change over time?
3. Practical and structural questions - which concepts are well developed/which are not?
What, when, how are data gathered for an evolving theory? Is the developing theory logical?
Etherington (2006) argues that “reflexivity can create a bridge between researcher, practitioner and new knowledge, thus enhancing trustworthiness” (p.81).
I am acutely aware of where I stand, and why I am doing this research, and have tried to make this transparent from the start both in this thesis, and to the research participants (and to Abahlali. Clearly, my position has influenced how I have done the research (as discussed in the chapter); and could potentially affect my findings. However, although I am testing theory in this study (Badiou’s theory of the event, and the extent to which it can help explain
‘unlearning’), although it is a theory I am interested in, I came into this research with its usefulness as on open-ended question. This is because I also have concerns about the theory;
for example, that it does not appear to allow enough for agency. I discuss this more fully in Chapter 9. I thus looked at the issue (of whether this is a useful theory) in both considering the theory qua theory, and in looking at the data. My ultimate finding was unexpected. As discussed above, I also used other strategies to mitigate against my own bias.
I have also undertaken to present this research, including my theorising and findings to Abahlali; this will also, of course, test my research.
2.5.3.3 Face validity
Also known as ‘member-checks’, face validity involves giving the research back to at least some of the respondents, by recycling categories, emerging analysis, and conclusions back to participants (Lather, 1986). Lather herself is slightly ambivalent about this in her work. For example, in her 1986 article on openly ideological research, Lather is very critical of neo- Marxist research that assumes that the research subjects cannot interpret the data correctly because of false consciousness, and thus the researcher/intellectual needs to do this.
However, in the same article she argues that “Our common-sense ways of looking at the world are permeated with meanings that sustain our powerlessness” (Lather, 1986, p.76); and thus we cannot only use member-checks/face validity and catalytic validity. In her 1991 book, she also points out that ‘false consciousness’ can limit the extent of usefulness of this process, since participants may be unable to move beyond the hegemonic determination of their own beliefs and attitudes.
Nevertheless, most writers in the field of research methodology consider face validity/’member-checks’ as a minimum requirement for trustworthy research.
In my study, I gave the participants my transcriptions of their stories so that they could add/change/delete, and edited the transcriptions as they appear in Chapter 10 accordingly. In point of fact, these related almost entirely to typographical errors.
2.5.3.4 Catalytic validity
Lather’s (1986, 1991) final requirement for rigorous research is catalytic validity -
documenting that the research process led to insight and activism of respondents, “the degree to which the research process re-orients, focuses, and energizes participants in what Freire (1973) terms “conscientization”, knowing reality in order to better transform it” (Lather, 1986, p.67). Lather recognises that this is an extremely unorthodox guideline for validity, because it is so clearly contra the idea of researcher neutrality, and rather insists that good research must necessarily lead to emancipatory action. I am not aware of other writers
accepting this guideline, except in so far as those within an emancipatory approach argue that emancipatory research must deal with the issue of praxis (as discussed above). Whilst
Lather’s writing appeal to me on many levels, this is one area where I have a problem with her. In her argument for catalytic validity, Lather assumes that people need the researcher/me to help them ‘re-orient, focus, and energize’ them (‘conscientize’ them); and they do not. My study is premised on this assumption.