CHAPTER 4: RESEARCH DESIGN AND METHODOLOGY
4.6 D ATA A NALYSIS , S YNTHESIS AND R EPORTING
Since a range of data types were collected in this study, a variety of techniques and tools were used in its analysis. Data was analysed using both qualitative and quantitative methodologies, and occurred largely inductively but sometimes also deductively. Data analysis began early in the data collection and proceeded throughout the research period. Emergent findings were fed continually into the data gathering process. The analysis of data is discussed below in relation to the phases of research and the sources of data.
4.6.1 Qualitative analysis of texts in the classrooms
In keeping with the participatory nature of data collection about texts in the classroom, the participating teachers were themselves involved in the categorization of the photographs of texts in their own classrooms. These analytical activities were chosen with the intention of involving the teachers in thinking critically about their classroom practice. The hands-on sorting activities gave the teachers the power to arrange and rearrange the photographs of the artefacts in their classrooms, if necessary without explanation and verbal analysis. I expected this participatory activity to triangulate data from questionnaires and interviews.
The methodologies employed for this participatory analysis involved grouping or ranking photographs of texts taken during the transect walk. Each participating teacher separately analysed the photographs from her own classroom (duplicates and unclear photographs were removed from the set), approximately a week after the transect walk. This happened in three ways. First, undirected or open grouping of photographs; second, importance ranking; and finally, time-line ranking. The purpose of ranking exercises is to reveal priorities and preferences (Mukherjee, 1993, p. 57). In the open grouping activity participating teachers were asked to group the photographs in any way they wished. Teachers explained their grouping rationale. Once this had been done, they were given the opportunity to repeat the exercise in any other way they wished. In the importance ranking exercise, teachers were asked to choose the most important photographs in terms of teaching reading in Grade 1, and to rank these in order of importance. In the time ranking exercise, teachers were asked to arrange the photographs on a rough time line indicating at which stages of the year different
texts were most useful in teaching reading. A schedule of the process which I used in facilitating these participatory analysis activities is included in Appendix 9.
The participating teachers had varied responses to these participatory exercises. One teacher was particularly unenthusiastic and reticent to be involved in these activities. This situation is analysed in Section 5.3.4 in the next chapter.
In conducting my analysis, I was aware of a project at the University of Texas (Austin) in the USA which had developed an inventory of text types in classrooms, called the TEX-IN3 inventory (Hoffman et al. 2004; Sailors, 2002). Drawing on a social practice view of literacy (Barton, Hamilton & Ivanic, 2000), this is a three-fold observation system involving a text inventory, a ―text-in-use‖ observation and a series of teacher and student interviews. While I developed my own categorisation of the texts in this study, comparison with the TEX-IN3 categories was instructive, at least in highlighting some texts which were missing in this case.
Therefore I refer to the TEX-IN3 categories at various points in my analysis, particularly in Table 4.
I did, however, borrow two rubrics of analysis from the TEX-IN3 project as an additional means of overall assessment of the data I had collected. I reasoned that the rubrics would be useful as a way of understanding the large amount of data collected from teachers through teacher interviews and the transect walk in this study. One rubric (see Appendix 11) was used to provide a holistic assessment of the texts used in the classroom and the other (see Appendix 12) was used to assess the teacher‘s critical understanding of the texts in the classroom.
A rubric is an instrument for organising and interpreting data gathered through observation.
Rubrics set out clearly what would be considered excellent performance in a given field, as well as what one would expect to see in any number of less than excellent performances.
Rubrics are often given to learners before assessment tasks are conducted, thus making transparent the assessment process and serving as a powerful motivator. However, I chose to use the rubrics simply as an analytical tool and did not share the rubrics with the teachers at any stage. However, in concluding the research process towards the end of 2007, I did discuss with teachers how texts could optimally be used in literacy teaching, describing the
―excellent‖ level of these rubrics and discussing ways in they might use texts differently in
their literacy lessons. I was aware, as discussed further in Chapter 5, of the fact that these rubrics were designed in the context of American schools, where there are quite different expectations of teachers and different norms in terms of teaching and learning materials. In using the rubrics I assessed which criteria had been met on the basis of my data collection in order to arrive at an overall score. This judgement was necessarily subjective, and could have been more valid had co-researchers been present to provide parallel judgements.
4.6.2 Quantitative analysis of texts in the classroom
In addition to the participatory analysis of texts in the classroom, I analysed photographs of texts quantitatively according to categories derived from the theoretical orientations towards reading reported in Chapter 2. As already indicated, an inventory of classroom texts devised in the USA called the TEX-1N3 inventory (Hoffman et al., 2004; Sailors et al., 2002) provided some useful comparisons with my analysis. The categories of text in the TEX-IN3 inventory are included in Appendix 10, and are cross-referenced with my own analysis in Table 4 in the next chapter.
For the purposes of my analysis, each photograph was allocated a category and a tally made of numbers of items in each category across the three classrooms. Simple statistical analysis enabled the comparison and graphic representation of types of text. Although the spirit in which this study was conducted emphasised the commitment to reveal and report what is rather than what is not, this quantitative analysis of texts did indicate the kinds of texts which were not present in the classrooms. This created some challenges in terms of reporting in the spirit of the study.
4.6.3 Qualitative analysis of data from classroom observation, interviews and group discussion
A great deal of research material was generated through the observations and interviews.
Although most of this was audiotaped, I did not complete detailed literal transcriptions of all of these tapes. Wanting to avoid what Walford refers to as ―the fetish of transcription‖
(Walford, 2001, p. 92, cited in Cohen et al., p. 462), I engaged in data reduction and selection which gathered data ―with a wide-angle lens‖ and ―funnelled‖ this through a process of
sifting, sorting, reviewing and reflecting (Cohen et al., 2007, p. 462). I listened to the audio- tapes of classroom observations and teacher interviews a number of times before beginning transcription, referring at the same time to my field notes. In the process, I started to identify patterns, themes, underlying meanings and concerns, and was able to choose segments of the tapes for close attention, translation of all isiZulu words and detailed transcription. I was concerned not to select data for transcription which supported some preconceived idea, but selected segments which fitted with the emerging ―meaning fields‖ (Carspecken, 1996, p.
93). Out of concern not to miss important details, I returned to the original tapes a few times during the data analysis process, most recently while writing this dissertation.
The transcript segments from the taped interviews and classroom observations were analysed by means of content analysis. Cohen et al. say that content analysis ―takes texts and analyses, reduces and interrogates them into summary form through the use of both pre-existing categories and emergent themes in order to generate or test a theory‖ (2007, p. 476). Other possible data analysis methods would have been discourse analysis (which is similar to content analysis in that it looks for themes by categorising and coding data, but examines specifically how language is used by participants to make sense of their reality), grounded theory (an extension of content analysis which ―theorises reality according to a set of empirically organised categories‖ (Henning, 2004, p. 115), consciously looking for relationships and themes) or narrative analysis.
My analysis of data involved reading of data and judgement through a process of constant comparison (comparing newly acquired data with existing data), and analytical induction and enumeration (counting the frequencies of categories) to identify emerging themes. I chose to do this manually, coding and sorting data, rather than using electronic means. I felt that this gave me a greater sense of connection to the data and to the emerging themes and categories.
Data analysis not using technology is very useful for small bodies of data, as long as rigorous coding is practiced. In the initial coding process I marked the data to describe:
Teachers‘ beliefs about teaching reading;
Teachers‘ ways of thinking about learners;
Activities undertaken in the classroom;
Relationships and social structure;
How literacy develops;
Texts used to teach literacy.
Subsequently, I re-categorised the items coded for texts used to teach literacy into sub- categories indicating individual word work and work with extended text. The items coded for teacher‘s beliefs about teaching reading were re-categorised into top-down, bottom-up and balanced groups (see Chapter 2). In the process of going through data more than once I was able to check the consistency of this coding. I next constructed statements which described what these teachers did when teaching literacy (carefully avoiding stating what they did not do), and data from different sources were categorised in terms of these statements. Examples of these statements are:
The three Grade 1 teachers in the school follow common lesson plans each day.
Teachers focus on teaching word recognition out of the context of extended text.
These and other statements are summarised in Chapter 5 and further discussed in Chapter 6.
4.6.4 Quantitative analysis of data generated from classroom observation
During classroom observation, a checklist of metalanguage use was employed. During lesson observations I noted all instances of oral or written use of language about language on this checklist. Very few examples of use of metalanguage were recorded, and quantitative analysis was simple and straightforward and very easily summarised in words.
4.6.5 Analysis of questionnaire
Teacher responses to the questionnaire were collated. Average scores were calculated more to give a clearer indication of trends than to prove a statistical point, since with such a small sample of teachers, statistical analysis could not yield useful results, and since the case in this study was in any event not the individual teachers but the three Grade 1 classrooms. The intention of the teacher questionnaire was to enable the correlation between teacher beliefs and teaching practice to ascertain the relationship between beliefs and practice. Once again, statistical correlation analysis was not employed due to the size of the sample. At the level of individual teachers it was easily possible to analyse the link between their practice and beliefs without engaging in statistical calculation. Although individual teachers were not intended to be the unit of analysis in this study, in presentation of the results of the questionnaire the responses given by individual teachers were reported in such a way that it would be possible
to build up a picture of each individual teacher‘s perspectives in relation to the beliefs and practices parts of the questionnaire.
4.6.6 Analysis of children’s drawings
Children‘s drawings were scrutinised carefully, using content analysis. Bell points out that content analysis of visual images ―is quite a technical procedure... and is seldom able to support statements about the significance, effect or interpreted meaning of a domain of representation‖ (2001, p.13). Such content analysis is most useful to test comparative hypotheses by making quantitative generalisations of categories. Content analysis of images can show what is given priority (ibid, p.26). As such, it is most useful if used as one of a number of other methods of data collection and analysis. This type of analysis, as opposed to psychoanalytic analysis, does not analyse the individual. I am not qualified to analyse children psychoanalytically, and therefore chose to tally incidences of themes in the drawings. The themes identified were categorised in terms of texts, attitudes/feelings and teacher and child activities (see Table 6 for more details).