International Journal of Education and Pedagogy (IJEAP) eISSN: 2682-8464 | Vol. 5 No. 1 [March 2023]
Journal website: http://myjms.mohe.gov.my/index.php/ijeap
END-OF-TERM SPEAKING ASSESSMENT RUBRIC FOR GRADE 6 ENGLISH TEACHERS TEACHING NEW
ENGLISH TEXTBOOKS IN VIETNAM
Nguyen Huong Thao1*
1 Faculty of English Language Teacher Education, University of Languages and International Studies, Hanoi, VIETNAM
*Corresponding author: thaonh.ulis.vnu@gmail.com
Article Information:
Article history:
Received date : 5 March 2023 Revised date : 17 March 2023 Accepted date : 23 March 2023 Published date : 30 March 2023 To cite this document:
Nguyen, H. T. (2023). END-OF- TERM SPEAKING ASSESSMENT RUBRIC FOR GRADE 6 ENGLISH TEACHERS TEACHING NEW ENGLISH TEXTBOOKS IN VIETNAM. International Journal of Education and Pedagogy, 5(1), 127- 144.
Abstract: Since 2008, Vietnam Ministry of Education has released the new project on studying and learning English in Vietnam called National Foreign Language 2020 Project. The new English textbooks are built base on communicative language teaching enhancing students’ communicative competence. Schools are required to organize end-of-term speaking assessment for the first time. Therefore, this research aims at identifying how end-of-term English speaking assessment for grade 6 students studying new English textbooks organized in lower secondary schools in Hanoi, Vietnam. Moreover, the research investigates teachers’ perspective on the importance and effectiveness of speaking assessment. From that findings, suggestions of the assessment format and criteria that should be included in the scoring rubric will be given.
The mix-method is employed to collect data through questionnaire and interview. The findings illustrate a lack of consistency in the organizing of end-of-term speaking assessment but positive teachers’ attitude toward the assessment process. Finally, based on the collected data, the model of a scoring rubric is suggested.
Keywords: end-of-term speaking assessment, speaking rubric, English teaching assessment.
1. Introduction
Vietnamese government has been urged to pay more attention to the development of English and its education due to globalization. Since the 1960s, English has entered the school curriculum as mandatory in both general and graduate education (Hoang, 2015). The authorities continuously took action by innovating textbooks. The progress can be seen in the shift from teaching English without material in the 1970s to teaching with two textbooks series which are incorporated in the content and different in the course length: 3 years in the North and 7 years in the South in late 1970s and early 1980s. The system then changed again for a 7- year unified English curriculum throughout the country using two kinds of English textbooks (the standard and the advanced one) which follow the guidelines from the Ministry of Education and Training (MOET, 2008; Hoang, 2015). Despite the attempt of innovating English teaching, the textbooks still focused on grammar-translation method and aimed at enhancing students’ reading skills (Denham, 1992). Consequently, there was poor requirement of spoken English for both teachers and students which led to the fact of 98% of Vietnamese students cannot use English for basic communication even though they have already studied the language for 7 years from grade 6 to grade 12 (Denham, 1992; Nhan, 2013).
Being aware of the shortcomings of the system and affected by the trend of English as a global language, since 2008 the Ministry of Education and Training (MOET) introduced the national project called “Teaching and Learning Foreign Languages in the National Education System, Period 2008 – 2020” or also known as “The National Foreign Language 2020 Project”. The purpose of this project is to provide a thoroughly English program with the balance of four skills in which the communicative competence is the priority outcome. The project made English a compulsory subject from grade 3 and targets by 2015 students who graduate from high schools, vocational schools, colleges and universities can use English independently and actively.
Along with this project is the establishment of new English textbook series which are based on Communicative Language Teaching approach; hence, assessment of speaking is an inevitable part of the outcome requirements (Hoang, 2018). Any innovation takes attention from researchers; thus, some studies have been conducted exploring the new textbook series. On one hand, Hoang (2016) illustrates the renovation of new syllabus design and course book development. N. T. Nguyen (2017) and T. Nguyen (2017), on the other hand, criticized the effectiveness of the books. Dang and Seals (2018) shares the same interest with the two mentioned researchers in analysing the books but focus on its sociolinguistic aspect especially at the primary level. Hoang (2018) shows his concern on the cross-cultural issue of the books since they are the products of the collaboration between Vietnam and MacMillan and Pearson experts. Although one of the main purposes of the National Foreign Language 2020 Project is enhancing communicative competence, hardly can we find the research investigating speaking education and its assessment. Le’s (2015) project is the minority of the field when its focus is the students’ anxiety in oral exam within the new English programme in a Vietnamese university as a part of the National Foreign Language 2020 Project. In general, the establishment of the new English textbooks plays an important role in the development of Vietnamese students’ English competence which takes much attention from experts and researchers. Inspire of the wide range in research, there has not yet a study targeting on speaking assessment in lower secondary school, particularly at grade 6. Moreover, some studies claims that the shortage of competent English teachers in the educational system is one of the obstacles for the achievement of the 2020 project (Dang, 2012; N. T. Nguyen, 2017; T. Nguyen,
2017). Therefore, the motivation for me to deliver the research is the desire to fulfil the gap of the field and to reduce English teachers’ burden in assessing speaking for students at grade 6 studying new English textbooks in Hanoi, Vietnam.
The research aims at identifying the perspective of grade-six English teachers in Vietnam on the importance of speaking assessment and how they test students’ speaking skills. From that finding, suggestions of the criteria and a sample rubric will be given for assessing speaking students at grade 6 with the new English textbooks in Hanoi, Vietnam. These aims could be achieved by finding answers to the following three research questions:
1. What is grade 6 English teachers’ attitudes to speaking assessment in teaching English in Hanoi, Vietnam?
2. How do grade 6 English teacher test students’ speaking skills in Hanoi, Vietnam?
3. What criteria should be included in a scoring rubric for speaking assessment of grade 6 students studying new English textbook in Hanoi, Vietnam?
2. Literature Review
Purposes of Speaking Assessment
Common purposes of speaking assessment listed by Luoma (2011) are providing feedback for further learning, giving course final results, for training or employment selection. Le (2012) added that the objectives of speaking assessment normally go along with the positive washback effect in which he based on to divide assessment purposes into four types that providing diagnostic information, screening of individuals for educational or commercial purposes, selection and placement. Relying on another perspective of the shift in language teaching from conventional theories to constructivist learning paradigm, other researchers classified purposes of assessment as three main types, namely assessment for learning (Stiggins, 2008); assessment as learning (Bennet & Gitomer, 2009); and assessment of learning (Biggs, 1995). The types are equivalent to formative, metacognitive and summative purposes of assessment.
Issues About Developing Speaking Assessment
The cycle of assessing speaking involves many interrelated components which are students, assessors, assessing tools and scoring rubric (Luoma, 2011). Accordingly, in this section all aspects of speaking assessment will be discussed.
Firstly, in terms of students, the description of students which are the object of the study has been presented in the first chapter as young learners at secondary school with starting level at A1 CEFR and expected outcome level at A2.1 CEFR.
Secondly, as regards assessors, the term is defined by Underhill (1999, p. 78) that “an assessor is a person who listens to a learner speaking in an oral test and makes an evaluative judgement of what he/she hears”. Since English is not original language of Asian countries, the debate of whether native or non-native assessors better in evaluating oral skills still under controversy.
Recent research projects conducted by Zhang and Elder (2014) in the Chinese context and Ekmekci (2016) in the context of Turkey have compared the results given by native and non- native assessors for learners about the speaking skills. The findings illustrated that either native or non-native raters/assessors produce the same or do not have much different in the result of speaking assessment.
Thirdly, one of the main concerns of developing assessment is assessing tools or tasks. Tasks in assessment refers to “a specific test item, topic, problem, question, prompt or assignment”
(Hakuta & Jack, 2009, p. 4). A more detail definition defines speaking tasks include activities which involve speakers in using language for the purpose of achieving a particular target in a particular speaking context (Bachman & Palmer, 1996). Bachman and Palmer (2010) in their model of communicative competence suggested that students should be encourage to involve in performance test in order to demonstrate their language proficiency. They added that the tasks in such assessment should enable students to talk, participate in real life and authentic conversation. In terms of suitable tasks for young learners, McKay (2006) introduced two types of pure speaking activities and activities that combine both listening and speaking. He also suggested some examples of each types such as news telling, storytelling, picture talks, interview, question-and-answer tasks, etc. Luoma (2011) also presented various types of tasks associated with their frameworks. Linguistically-oriented concepts targets in assessing vocabulary, grammar and pronunciation. Communication-oriented concepts guide developers to involve communication activities in the task such as description, narrative, instruction, comparing and contrasting, explaining and predicting, decision making. Finally, situation- oriented concepts focus on the specific-purpose test like in vocational or professional education. Among various types of tasks, the choice of task base on the intention of what to be measure (Le, 2012).
Regarding rating scale, rubric is described as a set of “criteria for a piece of what counts and articulates gradations of quality for each criterion” (Andrade, 2005, p.27). Hakuta and Jack (2009, p.4) defined scoring rubric in a simpler way that it includes “scoring guide, rating scale and descriptors or other framework used to evaluate responses”. Rubric is claimed to be a useful tool in assessment process as it helps maintain consistency (Arter & McTighe, 2001). However, Weir (1993) noticed that a rubric need to be designed or modified to be appropriate with the quality and functions of a particular test format/setting.
Therefore, in designing scoring rubric some aspects should be concerned, namely purposes of the rubric, types of rubric, level (band score), criteria (Ayhan & Türkyılmaz, 2013). Firstly, three common purposes of using scoring rubric are user-oriented, assessor-oriented, constructor-oriented and diagnosis-oriented (Alderson, 1991; Pollitt & Murray, 1996). In the context of the study, the scoring rubric is definitely built for assessor-oriented purpose in order to provide assessors information of what and how to assess speaking. Secondly, in terms of rubric types, for years, two major types of scoring rubric have been employed which are holistic and analytic (Fulcher, 2003; Luoma, 2011). Holistic rubric provides a single scale of score band combining with descriptions of language ability at different levels, whereas analytic rubric demonstrates subscale of criteria in which markers give score by evaluating test takers’
performance on each dimension (McKay, 2006). Each type has its advantages and disadvantages. On one hand, although holistic rubric is preferred for its convenience and time- saving in scoring due to its uni-dimensional in level (Mertler, 2001; Nitko, 2001), Thornbury (2005) stated that it lacks of feedback. On the other hand, analytic rubric is employed over holistic as it clearer and more precise in descriptors, it informs test takers information about their strengths and weaknesses, encourages students to improve their oral performances and learning, provide consistent scoring and offer fairness, validity and reliability in conducting speaking assessment (McNamara, 1996; Moskal, 2000; Andrade, 2000; Mertler, 2001; Arter
& McTighe, 2001; Thornbury, 2005). Even though there is a tendency in using analytic over holistic rubric in language assessment, analytic is claimed to be relatively time-consuming and
if a rubric contains too much scales or criteria, the assessors can be easily distracted and confused in assessing (Humphry & Heldsinger, 2014). Thirdly, as regards to number of level (band score) including in a scoring rubric, Nilson (2014) and Humphry and Heldsinger (2014) indicated that the scoring rubric with too many levels, especially more than 4, would make it difficult for assessors to have consistent decisions. Final aspects should be considered is criteria in a scoring rubric. It can be seen different types of criteria were assessed in previous research about speaking assessment. In his study of analytic rating scale, Sawaki (2007) investigated the rubric with five criteria, namely pronunciation, vocabulary, cohesion, organization and grammar. Alternatively, Council of Europe (2001) includes five aspects of speaking skills in its rubric: range, accuracy, fluency, interaction and coherence. Within the field of assessing young learners, Efthymiou’s (2012) rubric included grammatical accuracy, fluency, scope of vocabulary, pronunciation and content, whereas Westeryeld and Moran (2011) assessed four criteria of verbal productivity, syntactic complexity, grammatical accuracy and verbal fluency.
Srikaew, Tangdhanakanond and Kanjanawasee’s (2015) work under the similar field to this study about assessing grade 6 students in Thailand comprised six criteria in their rubric, namely vocabulary, syntax, cohesion, pronunciation, ideational function and fluency. Different selection in terms of criteria raises the question of the plausible number of criteria as Council of Europe (2001) warned that scoring rubric with more than 4 or 5 categories cause cognitive overload and 7 categories would reach the psychological limitation. Thornbury (2005), hence, states that 4 to 5 should be the maximum number for criteria in a scoring rubric for assessing speaking, whilst Luoma (2011) accept 5 to 6 as the manageable range.
Otherwise, assessment types and format also play an important role in assessment design.
Regarding assessment types, it is divided into formal and informal assessment. The former is an assessment planned and designed systematically to get information about students’
competence and achievement in limited time (Ketabi & Ketabi, 2014), while the latter is considered the occasional activities and unplanned comments or feedbacks (Brown, 2004).
Although formal assessment is claimed to affect students’ competence as it raises students’
affective filter (Cohen, 1996, as cited in Celce-Murcia, 2001), it is considered more valid and reliable than informal assessment. Informal assessment is not effective since the results are daily utterance and the assessment process does not base on any criteria (Harris & McCam, 1994). Additionally, the focus on designing authentic speaking tasks has engendered new formats of assessing speaking, namely pair and group assessment as the alternative for the traditional one-to-one assessment (Sun, 2014). These assessment formats allow students to interact with each other and demonstrate the tasks while being observed by one or more assessors to evaluate their language proficiency (Van Moere, 2013). Due to the advantages of new formats assessment, some large-scale tests have adopted them. Paired speaking format was first applied in the First Certificate of English in 1996 (Saville & Hargeaves, 1999), while group oral assessment was applied in English A/S level Examination in Hong Kong from 1994 (Swain, 2001). The formats soon get criticism for not allow enough time for students to produce an appropriate amount of linguistic output in comparison to traditional one-to-one interview (Foot, 1999). However, their benefits are undeniable. It is said that pair and group assessment lower the candidates’ anxiety and stress, hence, their language proficiency is best demonstrated (Ikeda, 1998; Saville & Hargreaves, 1999; Norton, 2005).
3. Method
In this research, a combination of two instruments including questionnaire and interview applied for well-rounded results. While quantitative approach allows a large amount of data is collected at the same time (Bryman, 2016), it is claimed to be limited as only numerical description and unchanging data are provided (Manheim, Rich, Willnat, Brians, & Babb, 2011). Meanwhile, qualitative research approach is proved to express participants’ feelings, opinions and experiences (Denzin, 2001). Therefore, combining both methods, mixed method, would reduce the disadvantages of qualitative and quantitative approach (Creswell & Plano Clark, 2011), as well as increasing the validity of the research results.
Sampling
Both probability and non-probability sampling methods were applied in choosing participants for the research project. On one hand, probability method, cluster sampling technique was employed to collect data from lower secondary schools in Hanoi. The schools were divided into 12 districts and questionnaires were passed to teachers of each group. The choice of cluster sampling base on its ability in generalising the collected data and time saving (Cohen, Manion
& Morrison, 2007). On the other hand, non-probability method, purposive sampling technique involved in selecting participants for interview as it provides the chance for researchers to make justification and have insight understanding of particular cases (Sharma, 2017). After collecting data from questionnaire, teachers with special answers were asked to have further interview.
The choice base on the differences in their responses in terms of organizing and attitude of speaking assessment. The expected number of participants is from 50 to 70 teachers.
Questionnaire
The questionnaire will be delivered to about 20 grade 6 English teachers teaching new textbooks from 4 different schools in Hanoi, Vietnam. The questionnaire will be designed by Google survey tool and the link will be sent to the participants. The choice of participant based on their experience in teaching the new English textbook for students at grade 6, hence, as long as they have taught English for students at grade 6, they are eligible to participate the research.
The questionnaire is expected to answer the first and second research questions asking about teachers’ attitudes on the importance of speaking assessment and how they assess their students’ speaking skills. Since likert scale questions are stated by Likert (1932) that they serve best to measure attitudes or opinions, they will be used to identify the first research question.
Besides, close-ended and open-ended questions will also be used in questionnaire design because on one hand, close-ended questions limit the variety of answers which allow us to collect the precise and direct data (Le, 2012); on the other hand, open-ended questions allow respondents to express an opinion without being influenced by the researchers (Foddy, 1993, p. 127). Therefore, the combination of both close-ended and open-ended questions would assist in not only collecting exact data but also avoid missing items. Using two kind of questions, the participants would find it comfortable to share their experience in assessing students’ speaking skills which is the focus of the second research question.
Interview
After collecting data from questionnaire, about 4 or 5 particular participants with different perspectives on the importance of speaking assessment or distinct technique in testing students’
speaking skills will be invited for further interview. The interviews will be conducted via Skype.
Interview is expected to answer the third research questions of what criteria should be included in a speaking assessment. Semi-structured interview is an appropriate type of interview for this research. As Mackey and Gass (2005) stated semi-structured questions can help in expanding the incomplete or ambiguous answers of participants, thus, there are more interaction between interviewer and interviewees which play important role in eliciting additional data.
Additionally, the fusion of both structured and unstructured questions would enhance the nature of conversation which make interviewees more comfortable in sharing opinions, ideas and attitudes in details but interviewers still can direct the conversation focusing on the main topic.
4. Data Analysis
The data analysis procedure includes two phases of analysing data from questionnaire and analysing data from interview.
In terms of questionnaire, participants’ and schools’ name were encoded to keep the identities anonymous. As receiving 32 responses from 21 lower secondary schools in 7 districts and 1 subdistrict in Hanoi, schools’ names were encoded after areas such as, 3 schools from the same district A were called A1, A2, A3 respectively. Moreover, participants were named P1, P2, etc.
as the abbreviation of participant number X. Collected data from open-ended and close-ended questions were presented in graphs, table to answer the first and third research question of how end-of-term speaking assessments are organized for grade 6 students and the use of scoring rubric. During the analysis of data from likert scale, 4 points was assigned to Strongly agree, 3 was assigned to Agree, 2 was assigned to Disagree and 1 was assigned for Strongly disagree.
Since likert scales were used to ask about teachers’ attitude on the importance and effectiveness of speaking assessment, the results were calculated to find the mean index in order to find the agreement among answers.
In terms of interview, participants’ identities were anonymous. Recording was transcribed for the important part which contain the answer for research questions. The collected data was then brought to compare, contrast and supplement for the ambiguous information in questionnaire.
5. Results and Discussion
5.1 Research question 1: How is end-of-term speaking assessment for grade 6 students studying new English textbooks delivered in Hanoi, Vietnam?
The collected data was from 21 lower secondary schools of 7 districts and 1 subdistrict in Hanoi. The number of schools and teachers involve in the project were different in each district which is presented in the following table 4.1. It can be seen that district C and D had the most schools participating in the project, whereas district D and G had the most responses for the questionnaire. All the participated schools have at least one year experience in applying new English textbooks in which district B has the longest time and district D has the shortest time in using new textbooks.
Table 4.1: Information About Schools District Number of
schools
Number of responses
Number of years applied new textbooks
Average number of years applying new textbooks
A 3 3 4-5 4.3
B 2 2 5 5
C 4 6 2-3 2.8
D 4 8 1-4 2.25
E 2 2 4-5 4.5
F 2 2 3-5 4
G 3 8 1-4 3
H 1 1 3 3
The result of the first question about whether a school have an end-of-term speaking assessment for grade 6 students showed that 2 out of 21 schools did not organize that activity. 7 teachers shared their reasons for that fact due to the time limitation, the lack of facility, human resource and the focus on writing assessment. Especially, teacher P29 expressed that her school was teaching two English curriculums at the same time in which half of the school was studying with the 7-year curriculum and the other half was studying the new 10-year curriculum. This situation led to the irrelevance in teaching syllabus and she believed it also made it hard for her school to carry a speaking assessment for grade 6 students studying new English textbooks separately. In order to have deeper understanding of these cases, question about the alternative ways to assess speaking skills revealed that instead of summative assessment, those teachers used formative assessment to assess students’ communicative competences. The formative assessment was in the form of reading aloud, question and answer, oral presentation when doing projects and oral checkpoints in class. Furthermore, teacher P21 stated that his school marked speaking skills through reading test which can be seen as a form of indirect testing.
This format is considered the format of the precommunicative eras (O’Loughlin, 2001) and not accurate and reliable enough to reflect students’ communicative competence (Qian, 2009).
However, it is remarkable to find that 2 out of 7 teachers did not deliver any form of speaking assessment. The data reported here seems to contrast with the direction of MOET’s NFL 2020 Project that end-of-term speaking assessment is compulsory. Teachers P25 and P29 explained in the interview that they do believe that speaking assessment is still optional and school year 2018-2019 is the official year that old English curriculum must be replaced. Consequently, they think it is not a big problem not having speaking assessment.
The findings on how speaking assessment was organized for grade 6 students studying new English textbooks can be found from data provided by 25 English teachers from 19 lower secondary schools. This section presents the organization of speaking assessment in different aspects suggested by Luoma (2011) and Ayhan and Türkyılmaz (2013), namely assessors, assessment format, tasks, outcome level and scoring rubric.
Chart 4.1: Assessors in Grade 6 End-of-Term Speaking Assessment
Firstly, the data about assessors is demonstrated in the chart 4.1. It can be seen from the graph that 11 schools claimed to use classroom teachers, whereas 8 schools used outside classroom teachers in assessing speaking. All classroom teachers were reported non-native speakers, whilst outside classroom teachers included two types of native and non-native speakers of English. Schools which used non-native outside classroom teachers indicated their purposes were bringing objective evaluation and emphasizing the fairness in assessment. On the other hand, teachers P16 and P23 considered native speakers to be more accurate in assessing speaking skills than non-native speakers. In addition, teachers of school D1 indicated that since their school collaborate with an English center to have native speakers teach extra lessons focusing on speaking and listening skills, they think using those native teachers in assessing speaking would bring a more authentic environment and reduce the workload for non-native teachers.
Chart 4.2: Assessment Types and Formats
Secondly, in terms of assessment types and formats, chart 4.2 illustrated that 12 out of 19 schools delivered speaking assessment in a formal setting which all students have particular candidate numbers and the assessment occurs in one day. The other 7 schools delivered informal speaking assessment which classroom teachers decide on their own how to assess and what to assess speaking skills. In addition, graph 4.2 showed that all three speaking formats were applied in speaking assessment for grade 6 in Hanoi, Vietnam. Among those, one-to-one interview was the most popular in both formal and informal setting, followed by pair and group interview respectively. According to MOET’s Decision N01475 (MOET, 2016), part 2 of speaking assessment comprises activities requiring students to interact with each other, therefore, pair or group interview are most suitable format for speaking assessment.
Nevertheless, in reality, one-to-one interview was applied for most of assessment. Teacher P11 explained her choice of one-to-one interview due the fact that it can reduce the risk of feeling being judged or embarrassed in front of classmate of students. She added that this format also
0 5 10 15
Classroom teachers Outside classroom teachers
ASSESSORS
Non-native speakers Native speakers
0 2 4 6 8
Formal setting Informal setting
ASSESSMENT TYPES AND FORMATS
One-to-one Interview Pair Interview Group Interview
allowed her to assess students more thoroughly and be flexible in helping students overcome nervous to express themselves. Otherwise, teacher P6 considered pair interview is better for its time-saving and interactive feature. In summary, speaking assessments were organized in two settings of formal and informal with three different formats, namely one-to-one, pair and group interview in which formal setting and one-to-one format were more preferred.
Chart 4.3: Tasks Types Used in Grade 6 Speaking Assessment
Thirdly, regarding tasks and tasks design, questions number 10, 11 and 12 of questionnaire revealed findings of this area. Corresponding to the quantity of assessment parts, number of tasks used in each speaking assessment were ranged from 1 to 3 tasks. The following chart 4.3 presented various tasks types used by schools and teachers in assessing grade 6 students’
speaking skills. It can be seen that description tasks were the most used by schools and teachers with 10 out of 19 schools which was estimated for 52.63%. Role-play and simulation tasks took the second place with 8 responses which counts for 42.1%. Other tasks types were moderately used range from 5 to 7 choices, whereas explaining and predicting tasks have the least choice for just 1. Besides given tasks types, 3 teachers expressed that their schools’
speaking assessment also included group presentation and questions about students’ personal information. The application of questions about personal information was relevant to introductory activity which was recommended by McKay (2006) as the vital features of tasks for young learners. He believed that this kind of activity help children to be ready before the assessment.
Chart 4.4: Tasks Framework
3 777 8
1 55 7 10
0% 20% 40% 60% 80% 100%
Others Reacting in situation Decision making Comparing & contrasting Narrative
TASKS TYPES
Number of schools No response
13 13
3 0
5 10 15
Linguistically - oriented
Communication - oriented
Situation - based
TASKS FRAMEWORK
Number of schools
Additionally, chart 4.4 demonstrated the choice of tasks framework. According to the data from chart, linguistically-oriented and communication-oriented concepts were chosen with the same number of schools of 13, while situation-based concepts was the least used with 3 choices from schools. The finding corresponded to the choices of tasks types above since the two most used tasks were parts of communication-oriented concepts. The statistic in both chart 4.3 and 4.4 showed that although communicative tasks were added more in the speaking assessment, linguistically-oriented concepts still had the same concern. Teacher P6 also comments that it was not a positive trend as she thinks those tasks do not reflect students’ communicative competence.
Chart 4.5: Expected Level for Grade 6 Speaking Assessment
As regards expected level, three most common levels were set by 19 lower secondary schools for grade 6 speaking assessment which are A1, A2.1 and A2.2 CEFR. The above chart 4.5 provided the proportion of each level. In general, level A2.1 was chosen for most of the school and level A2.2 was the least chosen level. A likely explanation for this situation is lying under the aims set by MOET in NFL 2020 Project that after finishing grade 6, students would reach level A2.1 CEFR (MOET, 2008). Therefore, most of schools set A2.1 level as the outcome for grade 6 students. However, it is also considerable that level A1 took 42% out of the whole statistic. The explanation can be found through interview with teacher P11. The participant claimed that the choice of A1 level based on students’ current competence which was weak as they have not studied 10-year curriculum in primary schools or even had never learned English before.
Finally, the using or scoring rubric was investigated through questions number 14 to 19 in questionnaire. The majority of schools (17 out of 19) indicated that they used scoring rubric in grade 6 speaking assessment. In contrast, 2 schools B2 and G2 did not use scoring rubric to assess their students. However, when being asked about the alternative way to mark students’
speaking skills, they stated that they assessed speaking without any rubric or other tools. The details of how scoring rubrics were applied and criteria should be included from teachers’
perspective would be discussed in research question 3.
42%
53%
5%
Expected Level
A1 A2.1 A2.2
5.2 Research question 2: What is grade 6 English teachers’ attitude to end-of-term speaking assessment for students studying new English textbooks in Hanoi, Vietnam?
The answer for second research question was gained from 32 responses of English teachers for questionnaire and 5 special cases in the semi-interview. In this section, teachers’ attitude on the importance and effectiveness of organizing end-of-term speaking assessment for grade 6 students studying new English textbooks will be demonstrated through aspects of assessors, assessment process and the usage of scoring rubric.
Table 4.2: Teachers’ Attitude on Speaking Assessment
Statements Mean SD
1. It is necessary to assess grade 6 students’ speaking skills. 3.44 .67
2. I prefer formal assessment to informal assessment. 2.91 .86
3. One-to-one interview is useful in assessing grade 6 students’ speaking skills. 2.66 .65 4. Pair interview is useful in assessing grade 6 students’ speaking skills. 3.06 .50 5. Group interview is useful in assessing grade 6 students’ speaking skills. 2.34 .83 6. Linguistically oriented tasks are useful in assessing grade 6 students’ speaking
skills.
2.91 .59
7. Communication-oriented tasks are useful in assessing grade 6 students’
speaking skills.
3.16 .57
8. Situation-based tasks are useful in assessing grade 6 students’ speaking skills. 2.63 .61 9. My school’s grade 6 speaking assessment follows the Ministry of Education
and Training’s instructions.
2.84 .81
Firstly, table 4.2 provides information about teachers’ perspective on the assessment format and the need of assessing speaking skills. Overall, there was a great consensus on the necessary of assessing grade 6 students’ speaking skills which was expressed through the highest mean score (mean=3.44, SD= .67). As a result, teacher P25 whose school does not organize speaking assessment also agreed that speaking assessment is necessary for motivating both students and teachers in learning and teaching speaking skills. In terms of assessment types and format, there was a tendency that formal assessment and pair interview were preferred rather than informal and other assessment formats with the mean at 2.91 and 3.06 respectively.
Additionally, positive attitude was given to communication-oriented concepts of tasks design (mean=3.16, SD= .57). In contrast, linguistically-oriented concepts received less approval (mean=2.91, SD= .59). The finding was interesting that there was a big difference in comparison between teachers’ attitude and the organizing of speaking assessment in real life.
When teachers seemed preferred communicative to linguistic tasks, both of them were applied with the same amount in reality (13 schools applied both types).
Table 4.3: Teachers’ Attitude on Assessors
Statements Mean SD
1. Classroom assessors are more accurate in accessing grade 6 students’ speaking skills than outside classroom assessors.
2.56 .76 2. Outside classroom assessors (both Native and Non-native speakers) are more accurate in
accessing grade 6 students’ speaking skills than classroom assessors.
2.69 .69 3. Native speakers give more accurate results than Non-native speakers in assessing grade 6
students’ speaking skills.
2.66 .60 4. Non-native speakers give more accurate results than Native speakers in assessing grade 6
students’ speaking skills.
2.28 .46
Secondly, teachers shared their opinions on the issues of assessors as illustrated in the table 4.3. In general, there was not much difference in teachers’ attitude to all four statements since mean score was stable at 2.5. As regards the question of whether classroom or outside classroom assessors were more accurate in accessing speaking, the result was slightly prone to outside classroom assessors (mean=2.69, SD= .69). On one hand teacher P15 indicated that the involvement of outside classroom assessors makes the assessment more objective. On the other hand, teacher P11 agreed with P15 but added that the content of the assessment should be designed by classroom teachers. In terms of first language of assessors, native English speakers received more positive view than non-native ones with the mean score at 2.66 and 2.28 respectively. Teacher P15 even seen native speakers as the perfect choice for assessors of speaking assessment.
Table 4.4: Teachers’ Attitude on Usage of Scoring Rubric
Statements Mean SD
1. It is necessary to have a scoring rubric/rating scale to assess students’ speaking skills. 3.50 .51 2. The scoring rubric/rating scale is useful for evaluating students’ speaking performance. 3.44 .50 3. 16. The more levels (score bands) in scoring rubric/rating scale the more consistent the
result is. 2.94 .56
4. 17. I would prefer to give score according to a single band score.
2.44 .67 5. 18. I would prefer to give separate scores for each element of speaking performance and
the final score is the combination of all elements. 2.81 .40
Finally, table 4.4 demonstrated the results about teachers’ attitude on the usage of scoring rubric. The majority of participants agreed with the need of using scoring rubric in speaking assessment (mean=3.5, SD= .51) and its usefulness in evaluating students’ speaking performance (mean=3.44, SD= .5). Teachers shared positive attitude to the statement of “the more levels in a scoring rubric, the more band scores the more consistency of the result”
(mean=2.94, SD= .56). Nevertheless, this finding was against the results from empirical research that too many levels included in a scoring rubric make it difficult for assessors to give the consistent evaluation (Weigle, 2002; Nilson, 2014; Humphry & Heldsinger, 2014). In terms of rubric types, analytic rubric (mean= 2.81, SD= .4) received more positive attitude than holistic rubric (mean=2.44, SD= .67). Teacher P11 expressed that she preferred analytic rubric since it provides the descriptors in more detail which support her better in evaluating students’
speaking skills thoroughly. This result shares the idea with researcher such as Sawaki (2007) and Tuan (2012) in their attempt to develop analytic scoring rubric for speaking assessment in the context of Japan and Vietnam.
5.
5.3 Research question 3: What criteria should be included in a scoring rubric for end-of- term speaking assessment of grade 6 students studying new English textbook in Hanoi, Vietnam?
Although the usage of scoring rubric is not the relevant answer for the third research question, it is still included in this section in order to have a comprehensive perspective of the issue.
As reported in the first research question, 17 out of 19 schools who organize speaking assessment for grade 6 students, declared that they use scoring rubric. Therefore, the findings about its usage were limited within the scope of 17 schools. Firstly, in the case of rubric types, the respondent took 71% for analytic and 29% for holistic rubric. The result was corresponded to the result of teachers’ preference in choosing rubric types found in the second research
question. Secondly, regarding the number of levels used in rubric, most of schools claimed that their scoring rubric comprised of 1-5 score bands, 2 schools used 6-8 score bands, while school E1 was the only one had 9-10 score bands and school C3 even did not divide their scoring rubric into different bands. Thirdly, with respect to the number of criteria, it was reported that 11 schools used the range of 4-5 criteria in their scoring rubric, the range of 1-3 and 6-7 criteria were applied by the other 6 schools, 3 schools for each type. In general, the majority of schools had scoring rubric with 1-5 score bands and 4-5 criteria which were the appropriate amount according to Weigle (2002), Nilson (2014) and Humphry and Heldsinger (2014) with the notion of no more than 4 levels and Thornbury (2005) with the idea of maximum criteria range from 4-5.
Turning to the specific criteria applied in the scoring rubric, chart 4.6 illustrated types of criteria that were currently used in grade 6 speaking assessment. Generally, criteria of pronunciation, fluency and grammatical accuracy were the most popular which were included in 15, 14 and 13 schools’ scoring rubric respectively. Coherence and vocabulary accuracy were followed criteria used by well over half of the schools with 10 and 9 schools. In addition, intonation, relevance of content of topic and task completion received the same concern by 8 schools comprised them in their scoring rubric. Cohesion, grammatical range and vocabulary range appeared in 5 and 6 schools’ scoring rubric. The least used criteria were overall impression chosen by 3 schools. Furthermore, two participants claimed that they also considered interactive skills as a criterion of scoring rubric.
Chart 4.6: Criteria in Scoring Rubric
In comparison to the criteria used in current speaking assessment, the following table 4.5 illustrated the importance of each criteria rated by teachers. The results were presented in the descending order of mean. It is clear that relevance of content to the topic, task completion and pronunciation were three most important criteria under teachers’ perspective with the mean score ranged from 3.66 to 3.41. Three least important criteria ranked by teachers were cohesion, grammatical range and vocabulary range. The results showed that teachers apparently expected to assess communicative skills rather than linguistic knowledge. Moreover, it was significant to find that there four out of six most important criteria voted by teachers coincided with four out of five most used criteria in reality. These criteria were pronunciation, fluency, coherence and vocabulary accuracy. Therefore, these criteria could play the core role in building suggested scoring rubric.
2 3 55 6 888 9 10 1314 15 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Other Vocabulary accuracy Relevance of content of topic Overal impression Grammatical range Fluency Coherence
CRITERIA IN SCORING RUBRIC
Number of schools No response
Table 4.5: Teachers’ Attitude on the Importance of Criteria
Criteria Mean SD
Relevance of content to the topic 3.66 .48
Task completion 3.47 .67
Pronunciation 3.41 .67
Fluency 3.31 .64
Coherence 3.28 .73
Vocabulary accuracy 3.16 .92
Overall impression 3.09 .78
Grammatical accuracy 3.00 .76
Intonation 2.91 .89
Vocabulary range 2.84 .77
Grammatical range 2.78 .91
Cohesion 2.75 .88
5. Conclusion
The aims of the research project are identifying the current situation of organizing of end-of- term grade 6 speaking assessment for students studying new English textbooks in Vietnam.
The teachers’ attitude on the importance and effectiveness of the assessment was also under research.
Basing on the current situation and teachers’ expectation, a scoring rubric will be suggested.
Some of the key findings are listed as below.
Firstly, in terms of how end-of term grade 6 speaking assessment are delivered for students studying new English textbooks in Hanoi, Vietnam. The findings illustrated that the majority of lower secondary schools have organized speaking assessment. Formal assessment was the main assessment type and one-to-one interview was the most common assessment format. The assessors of the assessment mostly were outside classroom teachers who were both native and non-native English speakers. The most popular tasks were description and role-play or simulation. Although these tasks types were communicative tasks, linguistically concepts of tasks design were still equal to that of communication-oriented concepts. Expected outcome level was set at A2.1 and A1 depending on the purpose of teachers or assessment developer.
Although the organizing of speaking assessment seems to be complete, it still lacked of consistency in every aspect.
Secondly, corresponding to the actual organization of speaking assessment, teachers’
perspective on the issues was studied. There was a great consensus on the necessary of assessing grade 6 students’ speaking skills. Additionally, teachers showed their preference to formal assessment types and pair interview assessment format. Outside classroom native speakers were considered more accurate in assessing speaking. There was a tendency that teachers wanted to use communication-oriented task framework in tasks design.
Last but not least, regarding the usage of scoring rubric. Almost every participated schools claimed that they used scoring rubric during the assessing procedure, nevertheless, the minority of 2 schools did not apply it. Even though both holistic and analytic were used, analytic scoring rubrics were more employed in the assessment and normally contained 1-5 levels/band score and 4-5 criteria. The collected data also revealed that teachers preferred analytic rubric and the criteria of pronunciation, fluency, grammatical accuracy and vocabulary accuracy were rated most important in assessing speaking.
6. Acknowledgement
The preparation of this paper was supported by all the participants: English teachers in lower secondary schools in Hanoi, Vietnam. Without their willingness to participate in answering questionnaire and interview, this study could not have been conducted.
References
Alderson, J. C. (1991) ‘Bands and scores. In Alderson, C. and North, B. (eds.) Language testing in the 1990s. MEP: British Council.
Andrade, H. (2000). Using rubrics to promote thinking and learning. Educational Leadership, 57(5), 13-18.
Andrade, H. g. (2005). Teaching with rubrics: The good, the bad, and the ugly. College Teaching, 53(1), 27–30.
Arter, J., & McTighe, J. (2001). Scoring rubrics in the classroom: Using performance criteria for assessing and improving student performance. New York: Corwin Press.
Ayhan, Ü., & Türkyılmaz, M. U. (2013). Key of Language Assessment: Rubrics and Rubric Design. In 3rd International Conference on Foreign Language Teaching and Applied Linguistics. IBU Publishing.
Bachman, L. F., & Palmer, A. (1996). Language Testing in Practice. Oxford: Oxford University Press.
Bachman, L. F., & Palmer, A. (2010). Language Assessment in practice: Developing language assessments and justifying their use in the real world. Oxford, UK: Oxford University Press.
Bennett, R. E., & Gitomer, D. H. (2009). Transforming K-12 assessment: Integrating accountability testing, formative assessment, and professional support. In C. Wyatt Smith
& J. Cumming (Eds.), Educational Assessment in the 21st Century (pp. 43–61). New York:
Springer.
Biggs, J. (1995). Assessing for learning: some dimensions underlying new approaches to educational assessment. The Alberta Journal of Educational Research, 41 (1), 1-17.
Brown, H. D. (2004). Language assessment: principles and classroom practices. New York:
Pearson Education.
Bryman, A. (2016). Social research methods. Oxford university press.
Cohen, A. D. (1996). Second Language Assessment. In Celce-Murcia, M. (Ed.). (2001).
Teaching English as a second or foreign language (3rd ed.). New York: Heinle & Heinle.
Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education. Routledge.
Council of Europe. (2001). Common European Framework of Reference for Languages:
Learning, teaching, assessment. Cambridge: Press Syndicate of the University of Cambridge.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Los Angeles: SAGE Publications.
Dang, T. C. (2012). National Foreign Languages Project 2020 [PowerPoint slides]. Retrieved from http://www.seiservices.com/APEC/WikiFiles/7.20.pdf
Dang, T. C. T., & Seals, C. (2018). An Evaluation of Primary English Textbooks in Vietnam:
A Sociolinguistic Perspective. TESOL Journal, 9(1), 93-113.
Denham, P. A. (1992). English in Vietnam. World Englishes, 11(1), 61-69.
Denzin, N. K. (2001). Interpretive interactionism (16). Sage.
Ekmekçi, E. (2016). Comparison of Native and Non-Native English Language Teachers' Evaluation of EFL Learners' Speaking Skills: Conflicting or Identical Rating Behaviour? English Language Teaching, 9(5), 98-105.
Efthymiou, G. (2012). Portfolio Assessment of Speaking Skills in English as a Foreign Language in Primary Education. Research Papers in Language Teaching and Learning, 3 (1), 200–224.
Foddy, W. (1993). Constructing Questions for Interviews and Questionnaires: Theory and Practice in Social Research. Cambridge: Cambridge University Press.
Foot, M. C. (1999). Relaxing in pairs. ELT Journal, 53(1), 70-76.
Fulcher, G. (2003). Testing second language speaking. New York, NY: Pearson/Longman.
Hakuta, K., & Jack, L. L. (2009). Guidelines for the assessment of English language learners. Educational testing service.
Harris, M. & McCann, P. (1994). Assessment (Handbook for the English classroom). Oxford:
Heinemann Publishers.
Hoang, V. V. (2015). The Development of the Ten-year English Textbook Series for Vietnamese Schools under the National Foreign Language 2020 Project: A Cross-cultural Collaborative Experience. VNU Journal of Science: Foreign Studies, 31(3).
Hoang, V. V. (2016). Renovation in curriculum design and textbook development: An effective solution to improving the quality of English teaching in Vietnamese schools in the context of integration and globalization. VNU Journal of Science: Education Research, 32(4).
Hoang, V. V. (2018). MOET's three pilot English language communicational curricula for schools in Vietnam: Rationale, design and implementation. VNU Journal of Foreign Studies, 34 (2), 1-25. doi: https://doi.org/10.25073/2525-2445/vnufs.4258.
Humphry, S. M., & Heldsinger, S. A. (2014). Common structural design features of rubrics may
represent a threat to validity. Educational Researcher, 43, 253-263.
doi:10.3102/0013189X14542154
Ikeda, K. (1998). The paired learner interview: A preliminary investigation applying Vygotsky an insight. Language, Culture and Curriculum, 11(1), 71-96
Ketabi, S., & Ketabi, S. (2014). Classroom and Formative Assessment in Second/Foreign Language Teaching and Learning. Theory & Practice in Language Studies, 4(2).
Likert, R. (1932). A technique for the measurement of attitudes. Archives of psychology.
Luoma, S. (2011). Assessing speaking (6th ed.). New York: Cambridge University Press.
Mackey, A., & Gass, S. M. (2015). Second language research: Methodology and design.
Routledge.
Manheim, J.B., Rich, R.C., Willnat, L., Brians, C.L., & Babb, J. (2011). Empirical Political Analysis. Boston: MA: Longman.
McKay, P. (2006). Assessing Young Language Learners. UK: Cambridge University Press.
McNamara, T. F. (1996). Measuring second language performance. London: Longman.
Mertler, C. A. (2001). Designing Scoring Rubrics for your Classroom. In Practical Assessment, Research & Evaluation, 7(25).
Ministry of Education and Training. (2008). Teaching and Learning Foreign Languages in the National Education System, Period 2008-2020. (1400/QĐ-TTg). Hanoi: MOET
Ministry of Education and Training. (2016). Assessment format assessing English proficiency equivalent to A2 level of CEFR for Vietnam (for lower secondary students). (Decision No 1475/QĐ-BGDĐT). Hanoi. MoET
Moskal, B. M. (2000). Scoring rubrics: What, when and how? Practical Assessment, Research
& Evaluation,7(3). Retrieved from http://PAREonline.net/getvn.asp?v=7&n=3
Nguyen, N. T. (2017). Thirty Years of English Language and English Education in Vietnam:
Current reflections on English as the most important foreign language in Vietnam, and key issues for English education in the Vietnamese context. English Today, 33(1), 33-35.
Nguyen, T. (2017). Vietnam’s National Foreign Language 2020 Project after 9 years: A Difficult Stage. International Economic.
Nhan, T. (2013). Promoting content and language integrated learning in gifted high schools in Vietnam: Challenges and impacts. Internet Journal of Language, Culture and Society, 38, 146-53.
Nilson, L. B. (2014). Specification’s grading. Sterling, VA: Stylus.
Nitko, A. J. (2001). Educational assessment of students (3rd ed.). Upper Saddle River, NJ:
Merrill.
Norton, J. (2005). The paired format in the Cambridge speaking tests. ELT, 59(4), 287-297.
O’Loughlin, K. J. (2001). The equivalence of direct and semi-direct speaking tests. Cambridge, UK: Cambridge University Press.
Le, T. P. A (Ed.). (2012). An introduction to research methodology in foreign language education. Ha Noi: University of Languages and International Studies, Vietnam National University.
Le, Y. T. K. (2015). Students’ perspective about anxiety in oral exam in ESP Department, University of Danang (MA Dissertation). University of Huddersfield, Huddersfield
Pollitt, A. and Murray, N. L. (1996) ‘What raters really pay attention to’. In Milanovic, M. and Saville, N. (eds.) Studies in language testing 3: Performance testing, cognition and an Assessment. Cambridge: CUP.
Qian, D. D. (2009). Comparing direct and semi-direct modes for speaking assessment:
Affective effects on test takers. Language Assessment Quarterly, 6(2), 113-125.
Saville, N., & Hargreaves, P. (1999). Assessing speaking in the revised FCE. ELT Journal, 53(1),
42-51
Sawaki, Y. (2007). Construct validation of analytic rating scales in a speaking assessment: Reporting a score profile and a composite. Language Testing, 24 (3), 355–390.
Sharma, G. (2017). Pros and cons of different sampling techniques. International Journal of Applied Research, 3(7), 749-752.
Srikaew, D., Tangdhanakanond, K., & Kanjanawasee, S. (2015). English speaking skills assessment for grade 6 Thai students: an application of multivariate generalizability theory. International Journal of Psychology: A Biopsychosocial Approach, (16), 47-66.
Stiggins, R.J. (2008). An Introduction to Student-Involved Assessment FOR Learning. New Jersey: Pearson Merrill Prentice Hall.
Sun, H. (2014). Paired and group oral assessment. Working Papers in TESOL & Applied Linguistics, 14(2), 68-83.
Thornbury, S. (2005). How to Teach Speaking. England: Pearson Education Limited Longman.
Underhill, N. (1999). Oral testing: Theoretical perspectives, current methodologies and new technology. Cahiers de l'APLIUT, 19(2), 77-86.
Van Moere, A. (2013). Paired and group oral assessment. In C. A. Chapelle (Ed.), The encyclopedia of applied linguistics (pp. 1-4). Oxford, England: Wiley-Blackwell.
Weir, C. J. (1993) Understanding and developing language tests. Hemel Hempstead:
Prentice Hall.
Westervelf, F. M., Moran, A. C. (2011). Expository language skills of young schoolage children. Language, Speech, and Hearing Services in Schools, 42, 182–193.
Zhang, Y., & Elder, C. (2014). Investigating native and non-native English-speaking teacher raters’ judgments of oral proficiency in the College English Test-Spoken English Test (CET-SET). Assessment in Education: Principles, Policy & Practice, 21(3), 306-325.
