Excerpt 6 June 1988)
4.8 Data Collection: Sources of Evidence
Yin (2003a) lists six sources of evidence commonly used in case studies:
Documentation Archival records Interviews
In Duff et al. (2000), the highly regarded coordinator of an ESL/healthcare skills program in which we were doing research sent letters, on our behalf, to former students whom we had not previously worked with, and this recruitment strategy proved very effective.
•
•
•
Direct observations Participant observation Physical artifacts
Using multiple sources of data allows researchers to “corroborate and augment evidence from other sources” (Yin, 2003a, p. 87). Not all case studies involve all of these sources of data, though. Wolcott (1994) pro- vides the mnemonic of three E’s in (ethnographic) qualitative data collec- tion: experiencing (participant observation), enquiring (interviewing), and examining (studying documents).
In applied linguistics, data collection often includes a number of instru- ments or techniques (see Mackey & Gass, 2005, for examples). Many studies focus on documents, archives, or artifacts (e.g., in policy studies or text stud- ies), or on observation and interview data (e.g., in SLA or classroom stud- ies). Document analysis might involve relevant paperwork and artifacts, such as textbooks, newspaper articles, students’ writing samples or assignments, course outlines, and research journals kept by participants or researchers. In addition to the possible data sources already listed are tests, elicited responses in the form of grammaticality judgments, stimulated recall, verbal reports (e.g., introspective or retrospective reports), and questionnaires (especially for surveying a larger group from whom cases are selected).
Data collection is determined by the underlying research questions and the forms of evidence deemed necessary to answer those questions.
Data collection decisions also depend on what the researcher plans to do with the data. The study must be feasible or doable, and this consideration should factor into the research design. Collecting masses of data that will never be analyzed or that would take an inordinate amount of time to tran- scribe, for example, is wasteful in terms of time, energy, other costs, and goodwill, and some would say it is unethical as well.
In the remainder of this section, I provide examples of data collection strategies in some published case studies. I also include relevant data col- lection information in some of the summaries of studies in Chapter 3.
In Morita’s (2004) multiple-case study of six Japanese students’ par- ticipation and perceptions of their own participation in whole-class and small-group discussions in Canadian university classrooms, data collection and analysis proceeded in the manner shown in Table 4.2, over an eight- month period. She employed a variety of methods: students’ self-reports
•
•
•
(by e-mail, in journals, etc.), classroom observations with fieldnotes taken, interviews (with the focal students and their instructors, separately), and documents of various types related to courses.
Achiba’s (2003) longitudinal single-case SLA study examined the English L2 developmental patterns and requesting strategies of her child.
The unit of analysis was a request. She collected two types of data:
(1) audio- and video-recordings of the child during playtime interactions at 14 different times over a 75-week period, during which the child inter- acted, in turn, with a peer, a teenager, and an adult; and (2) diary notes of the child’s spontaneous use of English with her, primarily. Because Achiba
Table 4.2 Summary of Database
Methods Data Collection Period
(Sept. 1999–April 2000) Data
Weekly self-reports by students
Ongoing
1–3 times per week, per student
•
•
E-mail messages Audiotaped face-to-face or telephone conversations Written journals
283 reports total, about 16 dif- ferent courses
•
•
•
• Interviews with [6]
students Interview 1: Beginning
of academic year Interview 2: End of Term 1
Interview 3: End of Term 2
•
•
•
Audiotaped and transcribed interviews
18 interviews total Average 1.7 hours each
•
•
•
Classroom observations • Ongoing Field notes on 59 lessons in 5 courses (151 hours of observation)
•
Interviews with
instructors Once with each instruc-
tor toward the end of the courses
• Audiotaped and transcribed
interviews 10 interviews total Average 1.2 hours each
•
•
•
Documents • Ongoing Course outlines
Handouts for presentations Self-evaluations of class participation
•
•
•
Source: From Morita, N., TESOL Quarterly, 38, p. 581, 2004. Published by Teachers of English to Speakers of Other Languages, Inc. This work is protected by copyright and it is being used with the permission of Access Copyright. Any alteration of its content or further copying in any form whatsoever is strictly prohibited.
was more concerned than Morita about exact linguistic structures produced (as opposed to more general participation patterns of the type Morita inves- tigated), audio- and video-recordings of language use were important.
Finally, Jarvis’s (2003) case study employed a combination of natu- ralistic and more laboratory-like experimental procedures. He examined the effects of an adult immigrant’s acquisition of an L2 (English) on her L1 (Finnish). His “natural use” data included Finnish constructions pro- duced in informal Finnish oral interactions over more than a year, based on which Jarvis detected 15 “unconventional” or “deviant” patterns (primarily lexico-semantics and idiomatic usage). His “clinical elicitation” data collec- tion, on the other hand, included showing short film clips devised to elicit narratives containing the marked structures, eliciting metalinguistic judg- ments regarding the same set of constructions, and gathering self-report data from a debriefing interview with the Finnish woman about her judgments.
In contrast to the preceding studies, my study of Jim (Chapter 1) involved only structured oral interview data, elicited by several types of tasks dur- ing each interview: small talk, picture description, picture sequence nar- ration, and personal or folktale narratives. Thus, in different types of case studies, different types of data will be collected and analyzed, according to the evidentiary needs and traditions of the subdiscipline, and often based on ethical and logistical considerations as well (i.e., whether permissions have been granted).