CHAPTER 2: LITERATURE REVIEW
3.8 The Use of Qualitative & Quantitative Research Methods
58 f) Stage 6: Evaluation of the Process and Re-implementation
The benefit of the cyclical nature of action research allows for evaluation of the implementation and effects on R2L both during the period of study and afterwards (as the intention of using the R2L approach is to continue to apply it for the benefit of future students in the BCom4 access programme and beyond). For example, it is important that R2L, which was originally developed in Australia, should be adapted to South African conditions. This may mean, for example, the collection of culturally relevant texts, or adaptation to the specific inadequacies of the current schooling system, such as the application of R2L at tertiary level.
59 a) Initial Writing Task
An initial writing task served as a baseline test to determine the initial level of student writing skills. The text consisted of an editorial letter written in response to a contemporary news topic. The students were then asked to write a short comment on the text. It was the intention of the researcher to measure this initial written task against the final written task at the end of the study period. Both tasks were marked according to the R2L assessment criteria.
b) Questionnaire
The questionnaire was intended to give the researcher an idea of the literacy history of the students and their previous interaction with print. Although the questions are mainly closed questions, a few were designed to be open-ended in order to obtain deeper insights into some areas for which closed questions may not have given enough scope, for example, when asking students if they have any current problems reading in English or their L1.
c) Tracked Students
Ten students were closely tracked during the research period to facilitate the detailed recording and analyses of quantitative data, such as assessment results and qualitative data, such as recorded observations and anecdotes. As well as the writing tasks assigned to the entire group of 46 students, the tracked students completed pre- and post-phase reading assessments. All questionnaire responses were recorded but those of the tracked students under-went deeper analysis.
Tracking students in this manner allowed the researcher to probe answers to questions more deeply.
60 d) Focus Group Meeting
The focus group meeting consisted of one semi-structured session held during the 2nd semester. The purpose was to ascertain attitudes and feelings towards R2L and gain an extent of its effectiveness in teaching academic literacy skills. The researcher made use of informal notes and questions recorded before-hand for the interview in order to guide the questions but wanted to create a forum in which the students felt the freedom to express their feelings and concerns openly and confidently.
e) Reading Assessments
The reading assessments, both –re- and post-intervention, utilised a reading test rubric contained in the R2L training manual (Rose, 2009). Use was made of graded texts, counting pronunciation errors and hesitations. When one text was read well, the student was moved up to another level in order to determine the grade level she was reading at. The researcher counted hesitations, mis-pronunciations and errors and then divided by the total number of words to determine the reading level of the student, who was then asked inferential and referential questions about the text to ascertain comprehension. The purpose of these assessments was to measure the initial reading level of the student and then to retest to see if R2L had had an effect on the reading grade level by the end of the 2nd semester.
f) Writing Assessments
The written tasks consisted of differing genres. Those in the 1st semester were a narrative, an explanation and an exposition, whilst those in the 2nd semester consisted of an editorial followed by an academic argument. The writing tasks were marked according to 14 standard points in R2L, i.e. purpose; presentation;
61 stages; phases; field; tenor; mode; lexis; appraisal; grammar; conjunction;
referencing; spelling; and punctuation. In addition, an extra point was added for content. This is to allow for any extra relevant content the student may have decided to add, including all relevant information from the text, and to allow marks for an overall excellent assignment. Each criteria was assigned a value as of between 1-5 points. This was initially 1 - 3 but was felt to be inadequate as most marks were falling into the middle category. 0 = absent, 1 = weak, 5 = strongest (approximating native speaker). T
These written tasks formed stage 6 of the R2L teaching cycle, namely ―Independent writing‖. The written tasks were then marked to measure progress and approximation to genre. Results of all written assignments were recorded, but those from the tracked students were recorded and collated for analysis and inclusion in this research paper.
g) Observation and Field Notes
The researcher kept an on-going journal (see Appendix 18) consisting of field notes throughout the year. This consisted of observations of the perceived attitudes and responses of the students during the implementation of the R2L approach, as well as the practical day to day planning of the project. This included, for example, errors made during the implementation of R2L, such as the incorrect choice of text, and steps that could be taken to remedy such situations. Also recorded in the journal were solicited and unsolicited comments from other lecturers as to the possible cross-over effects of the R2L implementation in other BCom4 modules.
Recording the qualitative data in a journal provided a systematic way of collecting potentially valuable information during the research and allowed the researcher to revisit the implementation process for analysis during the post-intervention phase.
62 The instruments above were used during three specific phases of the study project, namely: pre-intervention, intervention; and post-intervention phase. Use of each instrument will be discussed within the context of the relevant phase.