Corrective Feedback in Dialogue-based Computer
Assisted Language Learning
Halizah Binti Basiron
Department of Computer Science
University of Otago
Dunedin, New Zealand
halizahb@cs.otago.ac.nz
ABSTRACT
Computer assisted language learning (CALL) systems are used by people to learn a language. CALL systems have provided a number of advantages for language learning such as their ability to provide consistent and flexible corrective feedback during the learning process. This feedback is ex-pressed as information about what is ungrammatical or un-acceptable in a target language. This paper presents a lit-erature study on the field of corrective feedback and CALL and describes the future plan for my PhD research.
Categories and Subject Descriptors
I.2.7 [Artificial Intelligence]: Natural Language Process-ing—language parsing and understanding; H.5.2 [Information Interfaces and Representation (HCI)]: User Interfaces—
natural language
General Terms
computer assisted language learning, dialogue-based CALL
Keywords
corrective feedback, metalinguistic, recast
1.
INTRODUCTION
A computer assisted language learning (CALL) system is a computer application used as a tool in language teaching and learning. From the year 2000 to 2005, developed CALL systems have focused on language skills such as grammar and vocabulary learning, pronunciation, reading, writing, and listening [15]. There have been only a few systems de-veloped for the speaking skill, which is the most crucial one to master. The reading, writing and listening skills can be mastered easily through attending classes or doing exercises. As a result, the demand for a CALL system which caters to the speaking skill has increased, especially after the intro-duction of on-line chatting technology. However, when using a chatting system, the presence of a tutor is vital and con-versation can only be established when the tutor is available on-line. Alternatively, there is a need to develop a system
where a student can interact with a computer at any time. Such a system is called a dialogue-based CALL system. One of the main goals in any dialogue-based system is to allow the learner to train her or his communicative competence in natural, conversation-style environment.
In a normal language learning classroom, teachers usually provide positive evidence and negative evidence to learners in response to the erroneous sentences. Positive evidence consists of examples of what is grammatical or acceptable in a target language. Negative evidence, often known as corrective feedback, is information about what is ungram-matical or unacceptable in a target language.
This paper describes my proposed research to enhance a dialogue-based CALL (DB-CALL) system. Improvements will focus on corrective feedback provided by the system when dealing with a learner’s erroneous input. Some back-ground studies on corrective feedback and DB-CALL sys-tems are explained in Section 2. Then in Section 3, I discuss some of the outstanding issues from the literature. Finally, my proposed work is outlined in Section 4.
2.
LITERATURE REVIEW
This section starts with an introduction to corrective feed-back. Then investigation studies the efficacy of corrective feedback in different language learning environments are dis-cussed in Section 2.2.
2.1
Corrective Feedback
A piece of corrective feedback is a response from an ad-dressee to a speaker, where the adad-dressee’s intention is to correct the speaker’s erroneous utterance. The incorrect ut-terance can consist of grammatical errors, meaning errors or inappropriate use of lexical items. According to [3], correc-tive feedback is a response to a learner’s erroneous utterance by: i) indicating where the error has occurred; ii) providing the correct structure of the erroneous utterance; or, iii) pro-viding metalinguistic information describing the nature of the error, or any combination of these.
All corrective feedback is classified either as explicit or im-plicit in form [7, 3]. Exim-plicit corrective feedback tells overtly that an error has occurred whereas implicit feedback does not. Studies identified eight different types of corrective feedback employed by language teachers [8, 10]. Table 1 shows the various types of corrective feedback.
This paper was published in the proceedings of the New Zealand Computer Science Research Student Conference 2008. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.
Corrective Feedback in Dialogue-based Computer Assisted Language Learning
193
Table 1: Various types of corrective feedback Corrective Feedback Explanation
(Explicit Type)
Explicit Correction A teacher’s feedback in which he/she overtly corrects a student’s erroneous utterance by providing the correct form of the utterance.
Metalinguistic Feedback An explanation of any errors that occurred in a student’s erroneous utterance without providing the correct answer.
Elicitation A teacher obtains correct forms by asking questions. Corrective Feedback Explanation
(Implicit Type)
Recast A teacher reformulates a student’s utterance wholly or partly in a correct form. Clarification Request A teacher asks a student to rephrase the utterance.
Repetition A teacher repeats a student’s incorrect utterance and raises her voice to highlight the error. Translation A teacher translates a learner’s native language utterance into a target language.
Paralinguistic Sign A non-verbal corrective feedback such as facial expression, gesture cues, and high voice intonation
2.2
Research on Corrective Feedback
Studies on corrective feedback have focused on its nature and its role in language teaching and learning [10]. This section reviews a few studies which have been conducted to investigate the effectiveness of corrective feedback in various language learning environments.
An experiment investigating the effects of recast and met-alinguistic corrective feedback on the acquisition of past tense-edwas conducted in a normal classroom of language learning [3]. The participants were 34 English-as-second-language (ESL) learners and their proficiency level was lower intermediate. The participants were divided into three groups: two experimental groups and a control group. Group 1 and Group 2 completed two communicative tasks for two con-secutive days. During the tasks, Group 1 received metalin-guistic feedback and Group 2 received recast feedback, in response to past tense -ed errors. Table 2 (first column) shows a sample of conversation. The control group did not complete the tasks and did not receive any feedback on the errors. Testing was done in three stages: a pre-test, a post-test and a delayed post-test. The pre-post-test was conducted be-fore the instructional tasks. The post-test and the delayed test were conducted a day and two weeks after the instruc-tions, respectively. During the communicative tasks, Group 1 elicited more correct target forms (56% of total target forms elicited) compared to Group 2 (48%). There was a significant difference between the two groups’ pre-test versus the delayed test results. Overall, the results showed Group 1 performed better than Group 2.
Using on-line chatting technology, [6] replicated the experi-ment done by [3]. The participants were 31 ESL eleexperi-mentary- elementary-level learners divided into three groups: Group 1 received metalinguistic feedback, Group 2 received recast feedback, and a control group did not participate in the chatting. Test-ing was done in the same three stages. The pre-test was given before the chatting, while the post-test and the de-layed test were conducted a day and two weeks after the chatting respectively. Although Group 1 produced more cor-rect target forms (52%) than Group 2 (46%), results showed no significant difference between Group 1 and Group 2 nei-ther from the pre-test to the post-test results nor from the pre-test to the delayed test results.
One experimental study investigated which one of three types of corrective feedback (metalinguistic,metalinguistic + elic-itation, andrepetition + elicitation) is the most effective on learners’ uptake while using a CALL system [5]. Learner’s uptake means the student’s response to correct his or her mistake(s). The CALL system known as E-Tutor was de-veloped for the learners who wanted to practise various ex-ercises on German vocabulary and grammar. The exper-iment was done in four stages: a pre-test, the system us-age sessions, a post-test, and questionnaires. The pre-test was taken by the participants before using the system. The participants, who were university students, used the system for an entire semester (15 weeks) and completed exercises provided in each chapter. The post-test and questionnaires were given to the participants at the end of the semester. Results yielded that the participants were most likely to cor-rect their errors (87.4%) when provided withmetalinguistic + elicitation, compared to the provision of metalinguistic
(86.9%) andrepetition + elicitation (81.7%). Overall, the
metalinguistic + elicitationis the most effective method for the learners’ uptake.
194
H.B. Basiron
Table 2: Sample of conversation and corrective feedback Sample of Dialogue in [3] Sample of Dialogue in [9] Learner: He kiss her SPELL: What food does Katie like? Researcher: Kiss-you need past tense. [metalinguistic] Learner: Umm Katie like pizza.
Learner: He kissed SPELL: That’s right. Katie likes pizza. [Recast]
Learner: ...they saw and they follow follow [sic] him SPELL: What drink does Katie like? Researcher: Followed[Recast] Learner: [Silent]
Learner: Followed him and attacked him. SPELL: What drink does Katie like? [Slower]. Learner: Umm-drink ...
SPELL: John likes red wine.[Reformulation]
What drink does Katie like?
2.3
Dialogue-based CALL (DB-CALL) systems
This section describes some existing DB-CALL systems and what types of corrective feedback they provide. First and foremost, L2Tutor is a text-based dialogue system for learn-ing English by French students [11]. It implemented key-words and scenario-based methodologies, and the scenario used is ordering meals at a restaurant. The objective of the system is to show that it can have a natural and robust conversation with a learner. The L2Tutor provides metalin-guistic feedback in response to the learner’s erroneous input.
The Let’s Chat conversational system, developed by [14], is a DB-CALL especially for beginners or intermediate second language learners to practise their social conversations skills. There are several simple topics such as Friends, Food, Hol-idays and Sports which the learner may select to converse with the system. While communicating with the system, the learner is given a set of possible model responses. The developer uses this technique as an alternative to the limita-tions of natural language processing technology. Let’s Chat implements a simple error correction. For example, the sys-tem issues a response“That sounds odd. Perhaps you should choose again”if a learner answers a question by selecting a response which belongs to a different topic.
Another DB-CALL system is the Spoken Electronic Lan-guage Learning (SPELL) system [9] which incorporates re-cent technologies such as pedagogical animated agents, speech recognition and synthesis, and a virtual environment. Types of scenarios chosen in SPELL include ordering food at a restaurant, buying tickets at a railway station, asking for di-rections in a town centre, and asking about a virtual agent’s family. Recast feedback is provided if the learner’s response is semantically correct but has some grammatical errors. The system reformulates its question if the learner has re-mained silent or given an incorrect response. Table 2 (second column) shows a sample conversation.
3.
DISCUSSION
In general, most studies have been conducted to investigate the effectiveness of corrective feedback in different language learning environments, such as normal classroom and labo-ratory settings. Only a few studies have been done on on-line text-chatting or CALL system environments. Results have shown the benefits of explicit feedback [3, 4, 5]. How-ever, some other studies have shown that implicit feedback is better than or as effective as explicit feedback [6].
The experiment with on-line chatting [6] cannot prove that explicit corrective feedback (metalinguistic) is more efficient
than implicit feedback (recast). The results show no sig-nificant difference between the two groups. The learners’ proficiency level is mentioned as one of factors in why such results were produced. The proficiency level of the partici-pants is lower than participartici-pants in [3]. However, no known experiment has been conducted to replicate [6] with more proficient participants.
Most experimental studies of effective corrective feedback strategies conducted on CALL use mixed types of corrective feedback [4, 5]. With the existence of natural language pro-cessing technology, many CALL systems implement various styles of metalinguistic feedback. As far as I am concerned, only the SPELL system [9] provides recast and reformula-tion as corrective feedback. However, there is no menreformula-tion of any acceptance testing on such feedback. To the best of my knowledge, no experiment as similar [6, 3] has been conducted in the environment of a DB-CALL system.
Research shows that teachers prefer to provide recast (im-plicit feedback) as corrective feedback over other feedback types [8, 16], especially for beginners. Seedhouse [2] men-tioned that explicit feedback may result in embarrassment and demotivation of the learners.
Why do language teachers in classrooms prefer to use recast? Why do CALL systems use metalinguistic feedback? Since learning language using CALL is a different environment with different modes of instruction, independent research is needed [5]. Moreover, the outcomes will likely vary. There-fore, I would like to investigate the effectiveness of recast and metalinguistic corrective feedback on a DB-CALL sys-tem, as described more in the following section.
4.
RESEARCH PLAN AND CONCLUSIONS
This section first explains an existing DB-CALL system which I am working on. Next my research plan is outlined.
Corrective Feedback in Dialogue-based Computer Assisted Language Learning
195
Table 3: Corrective feedback provided in Te Kaitito Te Kaitito: How are you?
Learner: I fine.
Te Kaitito: You mean “Iamfine”. Let’s try again. How are you?
Learner: I am fine.
provide metalinguistic and recast feedback in response to learner’s erroneous utterances. My research questions are as follows:
1. Which type of corrective feedback system leads to less errors made by students during their interaction with the system?
2. Which type of corrective feedback improves a learner’s progress?
3. Does the explicit or implicit system better raise stu-dents’ awareness of their errors?
4. Does the explicit or implicit system better facilitate learners in correcting their errors?
5. Does the explicit or implicit system better motivate students in language learning?
In order to answer my research questions, two versions of the Te Kaitito system will be developed. The first version provides metalinguistic feedback (System 1) and the sec-ond version provides recast feedback (System 2). Since I am focusing on an English grammar and lexicon, the sys-tem is assumed to be used by ESL learners. Three groups of participants (Group 1, Group 2, Group 3) are required and the participants will be Malaysian secondary schools pupils in form 2 (age 14). Grammatical items focused on are past tense, present tense and future tense. The con-tent of lessons comes from a Malaysian textbook for Form 2 secondary schools [12]. All participants will be given a pre-test to determine their proficiency level. Group 1 will use System 1 and Group 2 will use System 2 respectively. During the system usage, Group 3 will do some other ac-tivities. A post-test will be given to all groups after system usage. The pre-test and post-test consist of questions on dialogues covered during the system usage session. During the experiment, the keyed interaction between the student and the system will be logged. Finally, the interaction and results from pre-test and post-test will be analysed to get the research findings.
5.
REFERENCES
[1] A. Copestake and D. Flickinger. An Open Source Grammar Development Environment and
Broad-coverage English Grammar Using HPSG. In
Proceedings of the 2nd International Conference on Language Resources and Evaluation, Athens, Greece, 2000.
[2] R. Ellis. Instructed Second Language Acquisition A Literature Review. Technical report, The University of Auckland, 2005.
[3] R. Ellis, S. Loewen, and R. Erlam. Implicit and Explicit Corrective Feedback and The Acquisition of
L2 Grammar.Studies in Second Language Acquisition, 28:339–368, 2006.
[4] A. Ferreira. An Experimental Study of Effective Feedback Strategies for Intelligent Tutorial Systems for Foreign Language. In J. S. Sichman, H. Coelho, and S. O. Rezende, editors,IBERAMIA-SBIA, volume 4140 ofLecture Notes in Computer Science, pages 27–36. Springer, 2006.
[5] T. Heift. Corrective Feedback and Learner Uptake in CALL.ReCALL, 16(2):416–431, 2004.
[6] S. Loewen and R. Erlam. Corrective Feedback in the Chatroom: An experimental study.Computer Assisted Language Learning, 19(1):1–14, 2006.
[7] M. H. Long. The Role of Linguistic Environment in Second Language Acquisition. In W. Ritchie and T. Bhatia, editors,Handbook of Second Language Acquisition, pages 413–468. San Diego:Academic Press, 1996.
[8] R. Lyster and L. Ranta. Corrective Feedback and Learner Uptake Negotiation of Form in
Communication Classrooms.Studies in Second Language Acquisition, 20:37–66, 1997.
[9] H. Morton and M. A. Jack. Scenario-Based Spoken Interaction with Virtual Agents.Computer Assisted Language Learning, 18(3):171–191, 2005.
[10] I. Panova and R. Lyster. Patterns of Corrective Feedback and Uptake in Adult ESL Classroom.
TESOL Quarterly, 36:573–595, 2002.
[11] C. Price, G. McCalla, and A. Bunt. L2tutor: A Mixed-Initiative Dialogue System for Improving Fluency.Computer Assisted Language Learning, 12(2):83–112, 1999.
[12] R. Shanta, U. Kurup, and S. Lorenz.Kurikulum Bersepadu Sekolah Menengah English Form 2. Mega Setia Emas Sdn Bhd, 2003.
[13] N. Slabbers. A system for generating teaching initiatives in a computer-aided language learning dialogue. Technical Report OUCS-2005-02, Department of Computer Science, University of Otago, 2005.
[14] I. A. D. Stewart and P. File. Let’s Chat: A
Conversational Dialogue System for Second Language Practice.Computer Assisted Language Learning, 20:97–116, 2007.
[15] G. Stockwell. A review of technology choice for teaching language skills and areas in the CALL literature.ReCALL, 19(2):105–120, 2007.
[16] M. Suzuki. Corrective Feedback and Learner Uptake in Adult ESL Classrooms.Teachers College Columbia University Working Papers in TESOL and Applied Linguistics, 4(2):56–77, 2004.
[17] P. Vlugter and A. Knott. A multi-speaker dialogue system for computer-aided language learning. In
