EAL speakers in SLT caseloads
Process 3: Assessment
9. Are there factors operating within the primary settings that can inhibit or promote performance on the competencies assessed?
4.1 ORIENTATION
CHAPTER 4
RESEARCH METHODOLOGY
4.1.2 Data collection and analysis
Table 3: Table showing the research data collection methods and analysis of data Data description and
sequence
Method and data collection Tool
Data Analysis PREPARATORY PHASE Descriptive survey via a
nationally posted questionnaire.
Survey with SLTs.
Questionnaire
Collected data was analysed using aquantitative computer programme, SPSS 18. The data was organised into simple frequencies and presented in tables and graphs (see chapter 2).
PHASE ONE Data description and
sequence
Method and data collection Tool
Data Analysis Step 1:Test administered
RAPT administered in English to all children who met the selection criteria
Administering of test to children through individual presentation, using language instrument (RAPT), video and audio recording.
Content analysis was used. The responses were analysed qualitatively and interpreted using a personal / experiential understanding and knowledge of the African (and specifically isiZulu) cultural background and worldview to guide the interpretation.
Step 2:Focus group 1 Conducted in mother- tongue i.e. isiZulu using pictures in RAPT as stimulus for discussion
Focus group ( children) Video and audio recording
The responses were transcribed and analysed descriptively and consecutively, while the emerging patterns were recorded, organised and coded thematically. Discourse and narrative analysis was used to focus on the emergent themes and input.
Step 3:Focus group 2 Conducted with adult isi- Zulu speakers regarding their perceptions of the RAPT content.
Focus Group (community and parent voices)
Core question schedule Video and audio recording
The responses were transcribed and analysed descriptively and consecutively, while the emerging patterns were recorded, organised and coded thematically.
Step 4
Interview with academics as language professionals
Interviews (Presenting RAPT to academics)
Interview schedule
The responses were transcribed and analysed descriptively and consecutively, while the emerging patterns were recorded, organised and coded thematically. The informants were asked to check that what was recorded by the researcher was an accurate reflection of their input.
After all the data in the various steps of Phase 1 were transcribed and qualitatively analysed, the content and pictures of the RAPT, with the help of an artist, were adapted on the basis of the responses from all the participants in steps 1 to 4 above. The final Phase (2) as described below was then implemented.
PHASE TWO Data description and
sequence
Method and data collection Tool
Data Analysis The adapted RAPT was
presented to SLTs
Delphi technique (Creating test guidelines using SLT
feedback)
Adapted version of test and schedule to analyse the changes
Highlighted the importance of a comprehensive approach totest adaptation
The approach depicted in Table 3 was selected for the interpretation of the research to:
• gain a better perspective on how language and cultural issues impact upon the interpretation of the test
• identify variations in participants’ responses and their perspectives on the questions asked
• Identify emerging themes that can stimulate discussion around the issue of using standardised tests with EAL speakers from indigenous linguistic and cultural backgrounds.
The framework used for the above approach is based on Ulin et al.'s (2002) principles for guiding qualitative research (2002) discussed below:
Principle 1: A social phenomenon cannot be understood outside of its context
The historical, social and political context is extensively discussed in the research because, in South Africa, language and culture are very sensitive and are linked to the history and politics of the country (Ramphele, 2008; Ratele, 2006).
Principle 2:Theory both guides qualitative research and is a result of it
The researcher has, for guidance, drawn on Critical Theory as a lens for interpreting the data, and Taylor’s cultural framework for organizing the data and understanding communication pathology. These theoretical and conceptual frameworks, respectively, were used to guide interpretation.
Principle 3:Exceptional cases may yield insight into a problem or provide new leads for further inquiry
Although it is important for the organization and structure of the research that there are guidelines, there were situations that arose during data collection such as obtaining the
return of consent forms, numbers and/ or availability of participants who posed challenges to these criteria, resulting in a review of these criteria. For example, in lieu of the exceptional cases encountered with gathering academics as a group, the data collection was changed from a group to individual interviews. This review, however, added value to the understanding of the research problem.
Principle 4:Understanding of human behaviour emerges slowly and non-linearly As the primary participants were children, their behaviour and responses could potentially have been highly unpredictable (Greig, et al., 2007). Despite the planning of each stage of their involvement, at times they refused to actively participate, resulting in unplanned changes. Establishing rapport with them also took longer than anticipated at times and impacted on the planned time frame.
Although the processes of data gathering and data analysis are described separately in this study, they were not mutually exclusive processes, for example, while the data gathering was occurring, there was already a degree of analysis that was taking place (Lincoln & Denzin, 1998; Maree, 2007). Decisions that led to modifications to the data gathering were influenced by this analysis. For example, during the piloting of a questionnaire, it was found that some of the questions were confusing and they were changed to be more easily understood. The emerging themes and perspectives also guided the direction that the data collection followed and helped to ascertain whether there was a need for further sessions for the focus groups.
4.1.3 RAPT: as a research instrument/example of other tests
In this study, the Renfrew Action Picture Test (RAPT) is the research instrument used as a case study to exemplify the essential problems of using standardised tests with EAL speakers, and acts as an example of other tests.
• The RAPT is a standardised screening test for language development (extensively described in previous chapters) that has guidelines on how it is to be quantitatively scored. It serves multiple methodological purposes in this study, as reflected in steps 1, 2 and 3 in Table 2. It is used for individual testing of the children to demonstrate differences in interpretation and as an example for discussion with the various focus groups, namely the children, parents/community, academics and Speech-Language Therapists (SLTs).
• The aim of this test is to evaluate spoken language i.e. the ability to convey information and use of grammar which is age-appropriate for children who are between 3 and 8 years.
• The SLT sits opposite the child, shows the child a picture, asks the child to listen to the question about the picture and then answer it. The child's answer is recorded. Each of the ten pictures has expected responses. There are points allocated for information and the grammar of the responses given by the child. Each of these expected responses has corresponding points. All these points are to be tallied under the categories of information conveyed and the grammar used. The scores are then compared against norms that are arranged in terms of age, based on scores that were derived by the developer of the test. These norms were developed in the United Kingdom. The comparison of the child's score with the norms is used as a criterion to assess the child's ability to convey information or use grammar which is age-appropriate.
• The only prompting permitted is to facilitate a more complete response and should be indirect. The SLT is cautioned to be aware of and allow for deviations in
pronunciation and dialectal variations, and to use her judgement on unusual responses (Renfrew 1997a). Typical responses to each picture are provided and are allocated
points. Norms on the state of the child's spoken language are given, based on the points scored.
4.1.4 RAPT as applied in this research
Although the administration and scoring manual of the research instrument requires that it be scored quantitatively, as explained above, in this particular research it was scored
qualitatively to provide accurate information for the purposes and population for which it was applied in the study. The purpose of the RAPT was as a case study to demonstrate the effects of culture and different language upon interpretation of test stimuli, so quantitative assessment would not have been appropriate.
The outcomes of the individual feedback on the test and those of the different focus groups were analysed, interpreted and discussed. The participants’ comments are the foundation for the themes, such as African vs. Western worldview and differences in perception about animals that emerge in the research. An understanding of these themes necessitates firstly, an interrogation of how these tests are used in their current form with populations from an indigenous language and cultural background and secondly, the guidelines for SLTs that will emerge from that interrogation.
Due to the typicality of the features, history, development and structure of the RAPT that it shares with other standardised tests, it was possible to use it in this study as an exemplar for other tests. Conceptually, it thus fulfils a role that can be compared with that of a case study i.e. it is used in the same way that the principles and findings emanating from a case study are generalised to other contexts. The RAPT was used in this study to determine whether there were consistent deviations in the responses from EAL isiZulu-speaking children to the questions on the various picture stimuli. It was applied to each child exactly as stipulated
above and the responses were recorded verbatim, as indicated in the test administration manual. These were then qualitatively interpreted.