RESEARCH METHODOLOGY
Exemplar 4.2: Excerpts from mathematics history interview
4.7 ENSURING RESEARCH QUALITY
Key issues relating to the quality o f this research included validity, generalisability and ethics.
4.7.1 Validity
The challenge for research underlaboured by critical realism (and this challenge can be extended generally to realist research) is how one makes validity judgements that engage with real phenomenon, given the fallibility o f our constructions o f actual phenomena (Maxwell, 2012). In other words, the validity o f the knowledge claims researchers make, is always subject to their intersubjective judgements, choices and decisions (Danermark et al., 2002).
Shadish, Cook and Campbell (as cited in Maxwell, 2012) suggest “validity is a property of inferences. It is not a property o f designs or methods, for the same designs may contribute to more or less valid inferences under different circumstances. ... No method guarantees the validity o f an inference” (p. 130). Although rooted in work on experimental and quasi
experimental research, this suggestion is typically realist. In this view, validity is relational in that it is not found in methods or procedures, but rather in the relationship between context, purpose and method. Put differently, assessing the validity o f research is not simply a case of establishing if specific procedures and methods have been used rigorously, but reflecting on the conclusions emanating from these procedures and in relation to the specific context o f the research (Maxwell, 2012). Danermark et al. (2002) suggest that the object o f study, rationale and method should be considered concurrently and in relation to each other, in order to inform the decisions made in terms o f data generation and analysis. This view o f validity “pertains to
the accounts or conclusions reached by using a particular method in a particular context for a particular purpose, not to the method itse lf’ (Maxwell, 2012, p. 130).
There is thus no set list o f characteristics, methods or procedures for ensuring validity in realist research. W hat I considered therefore was the relationship between purpose, methods and context and the extent to which I addressed possible validity threats to the conclusions in my study. Seale (1999, as cited in Maxwell, 2012) has three criteria for judging the validity of realist research. The first focuses on the relationship between the different elements o f the research (e.g. is there coherence between the research questions, the methodology, and the methods for generating data). During the various stages o f this research, from proposal development to concluding the writing o f the PhD, alignment has always been a priority in my work. Embarking on a study that uses a methodological framework which has had limited prior application in education in general and mathematics education in particular, I had to ensure that the questions I posed were typically realist questions. These are questions that foreground ontology rather than epistemology. I presented my rationale for these questions in Chapter One.
I had to provide a convincing and sustained argument for the use o f a novel framework. In Chapter Two I examined the limitations o f current research in the field for my research. This research emphasises what we know rather than what is, and conflates structure and agency.
These two limitations provided an opportunity for me to search for a novel framework. In addition, I had to ensure alignment between my social realist framework, the data generation methods and analysis o f the data. In analysing the data, I utilised the thought processes o f abduction and retroduction. Abduction enabled me to redescribe the phenomenon (i.e. the emergence and expression o f teachers’ identities in teaching FP mathematics) by drawing on critical realist ontology and the methodological framework o f Archer’s social realism, namely the morphogenetic approach. The process o f retroduction provided the opportunity to analyse the data and explain what is, by asking transfactual questions (i.e. questions about the mechanisms at the level o f the real). By means o f an example in Chapter Five, I sought to ascertain the mechanisms that had given rise to teachers’ identities. As highlighted above, the questions asked enabled me to discern the various mechanisms that conditioned the emergence o f teachers’ identities. I asked similar questions in relation to the expression o f teachers’
identities through the teaching o f mathematics. Asking transfactual questions also required that I focus on developing a deeper understanding o f the context o f the research.
The second criterion, as illuminated by Seale (as cited in Maxwell, 2012) concerns the research context. If expressed as a question, this could be posed as: are the events being studied located within their social, economic and political context? Exploring the preconditions that gave rise to these teachers’ identities and their expression through the teaching of mathematics required that I constantly looked back historically in order to identify the basic conditions. In Chapter Five I suggest that the unionisation o f South Africa in 1910 (the moment when the South African nation state was constituted) was a key moment in history. The effects o f that moment have impacted on teachers’ identities.
The third concern, as elaborated by Seale (as cited in Maxwell, 2012), relates to how the evidence is used to generate conclusions. In other words, the key question is: do the findings emerge directly from the data? This account o f validity focuses more on the explanations, findings, conclusions and inferences rather than on the techniques and methods used to generate the data. In considering the conclusions o f research, it is necessary to consider how these could be incorrect, by reflecting on the potential validity threats. A number o f validity threats were identified earlier in this chapter. These were presented together with comments about how I managed the threats. Some o f the potential validity threats included language, transcription, challenge with the stimulated recall interviews, observer bias, positionality and working with a meta-theory and methodological frameworks that have not been used much in research on teacher identity and mathematics education. I have highlighted each o f these possible validity threats except for positionality. I expand on the issue o f positionality later in the chapter as this requires further explanation.
In support o f Seale’s (1999, as cited in Maxwell, 2012) criterion for validity, Brinberg and McGrath (as cited in Maxwell, 2012) state that “validity is not a commodity that can be purchased with techniques. ... Rather validity is like integrity, character and quality, to be assessed relative to purposes and circumstances” (pp. 280-281). However, there were a number o f techniques employed to ensure the validity o f the data and conclusions. These included amongst others, techniques employed in positivist and interpretive research, triangulation, spending a sustained period in the field, peer review, member checking, and keeping an “audit trail”28 (Bassey, 1999, p.75).
28 All my empirical data, that is interviews and observations, have been kept both as raw data, coded data, and data formatted specifically for use in the text of this thesis and in the appendices. All video- and tape-recordings
Triangulation o f data was implemented through the use o f three different data generating methods (i.e. interview, observation and field notes) and through the use o f multiple interview formats. In addition to the day-to-day conversations I had with teachers when I was in the field with them, there were four to five scheduled interviews conducted with each teacher. While the nature o f the interviews differed (life history, mathematics history and practices) I used these to probe further, clarify misunderstandings and verify my interpretations o f the data and the emerging themes.
Verifying the transcriptions and translations was complex. While I provided each teacher with copies o f the transcripts o f all the interviews and requested that they confirm whether the transcriptions represented what they had said, none o f the teachers provided me with any feedback despite my asking numerous times.
During the research process, I had regular meetings with peers, colleagues and my first supervisor to assess my progress. As explained earlier I was part o f a broader research project, the EU:SFTE; this provided many opportunities for ‘peer debriefing’ (Carspecken, 1996) and peer review where fellow researchers were able to interrogate methods and interpretations.
However, both member checking and peer debriefing are not without their problems. For Maxwell (2012) these include firstly, the extent to which the participants and peers are interested in the actual study, secondly, the challenge o f “juxtaposing their own understanding to that o f the researcher” (p. 259), and thirdly, managing one’s own research trajectories and interests.
In this research, I transcribed all the interviews verbatim directly after each interview. I (re)listened to the audio recordings o f the interviews while reading the transcripts to ensure that I had produced as accurate an account as possible. Wolcott (2009) emphasises “description provides the foundation upon which qualitative research rests” (p. 27). In my field notes, there were phrases written directly in isiXhosa so as to ensure the accuracy o f the recording. I used these particularly when I was not sure o f the translation.
have been stored on various research-related hard-drives. I have also kept copies of the various translations of the transcriptions of the various video-recorded lessons.
In this section I have explained how I addressed observer bias, particularly in relation to my previous and current designations, and how I sought to overcome such bias. However, observer bias is intertwined with positionality more broadly, and this is a potential ‘validity threat’ that I address further, later in this chapter. Furthermore, I attempted throughout the research thesis to ensure that my philosophical assumptions (Chapter Three), role as researcher (Chapter Four) and findings (Chapter Five, Six and Seven) were surfaced, explicitly stated and interrogated.
4.7.2 Generalisability
One o f the criticisms o f case study research is the perceived inability to generalise out of research that is singular, context-dependent and context-specific. Flyvbjerg (2011) in his seminal critique o f common misunderstandings o f case study research, suggests
one can often generalise on the basis o f a single case, and the case study may be central to scientific development bias generalisation as supplement or alternative to other methods. But formal generalisation is overvalued as a source o f scientific development whereas ‘the force o f example’ and transferability are underestimated. (p. 305)
Danermark et al. (2002) concur and drawing on the work o f Ervin Goffman, suggest that there are numerous examples o f researchers generalising out o f qualitative research and even qualitative research framed by a case study methodology. Flyvbjerg (2011) builds his argument based on the nature o f the case and in particular, how the case is chosen; suggesting that some cases lend themselves to generalising and others not. My case was chosen both for convenience but also for its representivity o f the original sample.
Bhaskar (1978) claims “scientifically significant generality does not lie on the face o f the world, but in the hidden essence of things” (p. 227). In other words, he argues that it is by surfacing structural and cultural mechanisms at the level o f the real that researchers are able to make claims o f generality. Generalisation in this sense is built on transfactual conditions (i.e.
conditions that enable something to be so, rather than something different) and the constituent mechanisms and properties o f social structures and cultural systems. These mechanisms (i.e.
the transfactual conditions and fundamental structures) are attained though transfactual arguments, that is, arguments that are based on retroductive inference. Generalisation in realist research is thus different from generalisation in empirical research which focuses on regularities (i.e. constant conjunctions) as opposed to transfactual conditions (Chapter 3).
Flyvbjerg (2011) further suggests that case studies can be utilised both to test hypotheses and
theories, and to undertake “detailed explorations o f hyphothesised causal mechanisms” (p.
306). In this research, my case tested the use o f social realism in researching the emergence and expression o f teachers’ identities in teaching FP mathematics. In particular, my research uncovered the structural, cultural and agential mechanisms that condition teachers’ identities and the expression thereof in the classroom.
4.7.3 Ethics
My ethical deliberations can be described in two broad categories. The first concerned procedural ethics, which is the ethics o f obtaining consent to conduct research. The second is referred to as process ethics. This is about the ethical decisions made during the research process. In many respects, it is more complicated than procedural ethics because it requires thinking on one’s feet in some instances (Guillemin & Gillam, 2004).
Before explaining how I addressed these different dimensions o f ethics in my research process, it is necessary to acknowledge the power relations that existed in my research. There was an unequal relationship between the research participants and myself. As I will discuss in the next section, my privilege - particularly in relation to race, class and language - has and still does, position me in certain ways, even long after the ‘end’ o f apartheid and the advent o f democracy in South Africa. However, as Foucault (1978) suggests, where there is power, there is also resistance. This was evident in my research. For example, I had to continually negotiate access with one o f the teachers. On numerous occasions she asked me to remind her what the purpose o f the research was. On two other occasions, when I arrived at the school, she informed me that she would not be teaching mathematics that day, yet the next day I saw mathematics work that the children had completed in their jotters the previous day.
Oakley (as cited in Sayer, 1992) states:
Like any activity, research is a social process; and adopting the traditional academic conception does not render the research process innocent or ethically neutral: on the contrary, the belief that it does may permit insensitivity and political naivety. While these general warnings about ethical problems must be heeded, actual decisions must be made in the light o f an evaluation o f the particular politics (including one’s own
‘personal politics’) o f the situation under study, with all its conflicting interests and imbalances o f power. (p. 256)
Central to Oakley’s words, for me, is the importance o f relationship building and recognition that this process continues throughout the research and beyond. In addition, acting ethically in
the context o f research requires that the researcher make explicit one’s assumptions in relation to the research. In this thesis, I have endeavoured to make my theoretical and methodological perspective transparent, to problematise my assumptions, to provide detail o f the ethical deliberations that occurred in negotiating access to the research sites and during the research process in schools, and to share my ethical decisions during the write-up o f this PhD.
4.7.3.1 Procedural ethics
Ethical clearance for this PhD research was obtained through a variety o f mechanisms and in different ways:
• The Education Higher Degrees Committee validated my PhD proposal with the institutional ethical permission granted in May 2012;
• Permission from the Provincial Department o f Education (Eastern Cape) was obtained through the broader research project (EU:SFTE) for working in schools in May in 2012 (Appendix 5) and from the District Office in June in 2012;
• A meeting with all the school principals, organised through the District Office, to address them about the research and request permission to work in the schools was held in June 2012 at the District Office;
• Meetings were organised with all the FP teachers in the three schools in Lwandle Township. These meeting were to address the teachers about the project and to ascertain initial willingness to be participants in the research. There were three PhD scholars working in Lwandle Township simultaneously, hence we decided to meet with all the FP teachers in the schools in Lwandle Township first. This meeting took place on Tuesday the 13th o f June 2012.
Towards the end o f June and in August 2012, I met with the teachers o f the three schools individually to obtain permission for my PhD research. In these meetings, I further explained the purpose o f my study, the research process, and my expectations o f the participants in my research. The teachers were encouraged to think about their willingness to be involved in this research, and given time in this regard, before being requested to sign consent forms (Appendix 6). Teachers were told that they could withdraw from the research at any stage o f the research process.
Informed consent was thus obtained at various levels within the education system. Once the formal ethical procedures had been finalised and the teachers had given me their respective consent to spend time with them in their classrooms, the ethical concerns shifted to immediate issues that emerged during the research process in the field. I found that it was during the research process, “ethics in practice”, that competence in recognising and dealing with ethical dilemmas was required (Guillemin & Gillam, 2004, p. 262).
4.7.3.2 Ethics in process
Two issues emerged from the assumption that informed consent would ensure participation;
firstly, a realisation that access to the classrooms and teachers would have to be continuously negotiated, and secondly, a quandary about the extent to which the teachers who participated in this research were research participants or subjects.
As highlighted at the beginning o f this section, negotiating access to classrooms in the Eastern Cape Province is a multi-layered and on-going process. Having negotiated access with all the necessary authorities prior to approaching the teachers, there were times that I doubted the extent to which the teachers in this study believed they actually had a choice in the decision to participate. I experienced discomfort at these times. The education system in South Africa, like in other parts o f the world, is hierarchical. I began thinking that because I had obtained permission from the national, provincial and district structures before meeting with the teachers, I had reduced their discretion about participating in my research. To mitigate this, I worked hard to build relationships and trust.
Negotiating access to the classrooms o f the Grade 3 teachers at both schools was an on-going process. There were a number o f events that restricted access to the classrooms:
• Getting teachers to agree to participate in interviews after hours, as opposed to during school, had to be continually negotiated;
• The Annual National Assessments (ANA) written in September took a week and during this time no teaching occurred. Prior to the ANA, the learners were given an exemplar to complete and post-ANA a few days were spent revising errors children had made;
• Unexpected events took various forms:
o a natural disaster (viz. flooding) resulted in road closures and I was not able to access the schools;
o an unanticipated arrival o f DBE officials to check that the teachers were using the national workbooks;
o a farewell to Grade 7 learners resulted in no teaching for a day, as the teachers were required to cook for the farewell, and
o principals and district officials requiring the teachers in my study to perform tasks that took them out o f the classroom (e.g. writing up an agenda for a staff meeting, photocopying or attending a mini-cricket workshop).
Such disruptions to the data generation process are part and parcel o f research in South Africa (Vithal, 1998).
The second issue that I grappled with in this research, related to respect for persons (Bassey, 1999, p. 15), particularly the distinction between research participants and research subjects.
The research process began with an ethical tension for me. As the researcher, I asked the teachers to consent to my research interest and research process. None o f the teachers had sought me out. Furthermore, as this research attempted to expose the social, cultural and agential mechanisms that had given rise to teachers’ identities and their expression through the teaching o f mathematics, this was not a research project that was o f immediate benefit or value to the teachers I was researching. For me, the question was how could I enact respect for persons in this situation? Guillemin and Gillam (2004) suggest that this tension can be resolved if the teachers ‘take up ’ the research project and process as their own, but this was not likely to occur in my research study. I thus chose to deal with the enactment o f informed consent and respect for persons primarily through establishing collegial relationships with the research participants, by being respectful to the teachers, interested in them as persons and their practices, and considerate when they intimated that they did not want me in their classrooms. I was sensitive to body language and asked if it was convenient for me to spend time with them in the class each day and always indicated my willingness to return on another day should they wish. I also committed m yself to assisting teachers in other ways, for example, marking the children’s work and doing their photocopying. However, I was not able to escape the fact that these teachers saw me as an authority on mathematics education due to my position as university lecturer and researcher; someone who ‘knows more’ and who could ‘authorise’ their work. The teachers constantly sought confirmation from me in relation to their teaching o f mathematics.