The Studies and Educational Context
Study 1: CLIL Practitioners’ Assessment Practices
Teachers’ Professional Background and Interview Procedures
We interviewed 10 teachers to solicit their methods of assessing students’
essays and presentations in their CLIL courses (Table 2). They were special- ised in fields related to second language education. According to their self- reports, two teachers were extremely familiar with CLIL; seven teachers were moderately familiar with CLIL; and one teacher was somewhat familiar with CLIL. They learned about CLIL by attending seminars held at the university and by reading books such as Coyle et al. (2010) or Dale et al. (2011). They have implemented CLIL in the university and their previous workplaces for 1.5 years (T7) to 14 years (T1). During the interview, we primarily asked about (a) what essay and presentation tasks they assigned in their CLIL courses and (b) what aspects of language and content they assessed through essay and presentation tasks. Prior to the interviews, they were asked to provide infor- mation about the tasks and criteria as well as their demographic information through a questionnaire. The interviews were held based on the questionnaire to further examine their assessment approaches.
Essay and Presentation Tasks
Table 3 presents the essay and presentation tasks that the 10 teachers assigned to their students in their CLIL courses. The numbers indicate how many teachers employed the tasks.
The most popular tasks were research papers and research presentations. In these tasks, students were given a theme related to the content covered in the course and asked to collect information about it to present the findings in the form of essay and presentation. For example, in T3’s course, students learned about interlanguage pragmatics and speech acts (request, apology, and refusal).
As an assignment, they were asked to interview international students on campus to investigate how English speakers express these speech acts.
Subsequently, students gave presentations on the findings and wrote a paper summarising their research project. In addition, in T4’s course about
Table 2 Participants and their CLIL courses Gender Nationality
Years of
teaching Content
Students’
level
T1 Female Japanese 41– World Englishes 2
T2 Male Japanese 11–20 Positive psychology 4
T3 Female Japanese 11–20 Speech acts 2
T4 Male British 11–20 Countries, culture, science and
technology 1
T5 Female American 11–20 American studies, Japanese history
3 T6 Male British 11–20 Olympics, UK culture, arts and
entertainment, modern lifestyle
5
T7 Female Japanese 11–20 Globalisation, history of English 3
T8 Male Japanese 11–20 SLA 3
T9 Female Japanese 11–20 Life and technology 1
T10 Female Japanese 1–10 SLA 2
Note: Students’ proficiency: 1 = Elementary; 2 = Lower Intermediate; 3 = Upper Intermediate; 4 = Lower Advanced; 5 = Upper Advanced
SLA second language acquisition
Table 3 Essay and presentation tasks in the teachers’ CLIL courses
Essay task # Presentation task #
Research paper 8 Research presentation 8
Reflective journal/essay 2 Debate 1
Reflective presentation 1 Explaining a learnt concept 1 Note: Two and three teachers used the same essay and presentation task in their EAP courses, respectively. T8 used two presentation tasks in his CLIL course
countries/cities, culture, and science and technology, students were required to select a subtopic (e.g., smartphones), formulate research questions, carry out research, and present the findings in their essay and presentation. These tasks tap into the ability to research and extend the content knowledge beyond what the teacher presented (Coyle et al., 2010). Furthermore, research papers and presentations require the cognitive process related to Create, where stu- dents synthesise sources into a novel structure (Anderson & Krathwohl, 2001). The majority of the teachers who incorporated research assigned this type of task.
A variation of this were tasks that make students state their own opinions based on research. An example is an essay task given in T6’s course, where students learned about evidence that people are sceptical of, in relation to international organisations. The essay task was to choose one international organisation, collect information, and argue how worthwhile and sustainable it is. This argumentative research task does not only address the ability to research but also the ability to manipulate the content using higher-level thinking skills, including analysing and evaluating (Coyle et al., 2010), because students need to critically engage in particular information and con- struct their own arguments. This type of research was assigned by T6 and T7.
While some teachers assigned research papers and presentations in their EAP courses, others incorporated research tasks only in their CLIL courses.
The reasons for assigning research tasks in CLIL were that research taps into higher-order thinking skills and is more challenging, academic, and CLIL-like (i.e., being associated with CLIL) than other kinds of task. This indicates that the teachers chose to assign essay and presentation tasks aiming to assess stu- dents’ use of higher-order cognition and critical thinking, realising that it is a key element of CLIL. They appeared to emphasise these skills more than in their EAP courses.
In his course on second language acquisition (SLA), T8 is the only teacher who assigned tasks in which students need to directly demonstrate their con- tent knowledge. The presentation task assigned to his students was to explain one of the four SLA concepts covered in class. This task, which is similar to one requiring students to write a summary of a learnt concept (Genesee &
Hamayan, 2016), is different from the abovementioned research tasks, as stu- dents cannot successfully perform the tasks without an accurate understand- ing of the content. In this sense, his essay task examines general understanding of the content (Coyle et al., 2010). T8 also assigned a unique essay task in which students are required to apply the knowledge gained in the course. His students were asked to reflect on their own English learning experience and explain their successes and struggles using SLA concepts. Similar to Lin’s
(2016) application assessment task, students do not only demonstrate their knowledge but also apply the knowledge to explain a new phenomenon.
Assessment Criteria
Table 4 shows assessment criteria for the essay and presentation tasks used in their CLIL courses. The numbers indicate how many teachers (out of 10 teachers) used the assessment criteria.
Most teachers explicitly assessed language and technical aspects of students’
essays and presentations in their CLIL courses. In the assessment of students’
essays, three to five teachers focused on rhetorical organisation (e.g., presence of a thesis statement and a concluding paragraph), lexicogrammatical accu- racy, paragraph structure, and cohesion (e.g., the use of discourse markers and pronouns). Similarly, in the assessment of presentations, four to five teacher(s) assessed rhetorical organisation and lexicogrammatical and phonological accuracy. In addition to these linguistic criteria, six and eight teachers evalu- ated the use of required format in essays and delivery of presentations (e.g., eye contact, gesture, voice volume, fluency), respectively. The abovemen- tioned criteria were not unique to the assessment in CLIL as they were also applied in the teachers’ EAP courses. This is not surprising because the partici- pants are language teachers and these features have been ubiquitous in writing and speaking tests (Davies et al., 1999). In fact, three teachers (T1, T4, and T8) stated that they emphasised the assessment of language in CLIL courses with lower-proficiency students, rather than or not solely the content of ideas and content mastery. Students’ language proficiency may be a factor in how much teachers focus on language.
The logical development of ideas or coherence was a criterion related to the content of essays and presentations. Seven teachers incorporated this criterion
Table 4 Essay and presentation assessment criteria in the teachers’ CLIL courses
Essay criteria # Presentation criteria #
Format (e.g., convention) 6 Content 9
Organisation 5 Delivery 8
Language (e.g., grammar) 5 Organisation 5
Coherence (e.g., logicality) 5 Language 4
Cohesion 4 Teamwork 3
Content 3 Preparation (e.g., practice) 3
Paragraph structure 3 Format 2
Others (e.g., clarity) 3 Logicality 2
Others (e.g., enthusiasm) 2
into the assessment. This feature, which is related to the sequencing of ideas in a text, is frequently included in assessment criteria for writing tests in gen- eral (Davies et al., 1999). However, in research-based papers and presenta- tions, some teachers attended to features that general performance tests cannot elicit. For example, T1 and T5 evaluated whether resources obtained from research logically support students’ arguments or ideas, focusing on the rele- vancy of selected reference information or data. In particular, T1 stressed the importance of logicality of ideas in advanced-level CLIL courses more than in EAP courses. T3 also assessed the degree to which students analysed results of research logically and clearly.
In the assessment of presentations, in particular, teachers assessed the con- tent focusing on the amount of information, inclusion of required elements, interestingness of ideas, and informativeness of theme. For example, T2 focused on whether the message and content in students’ research presenta- tions on positive psychology was mature and intriguing. T4 and T6, who had their students decide a research topic and give presentations about research findings, also assessed these points and explained that presenting information that the audience already know (e.g., Japanese food or festivals) is not interest- ing or useful. Thus, it seems that interestingness and informativeness appear to concern the novelty of the research findings, which resonates with the Content and originality component of assessment criteria used by Bae et al.
(2016) and Cloud et al. (2000). However, some teachers raised concerns about assessing the quality of content focusing on interestingness and infor- mativeness. T3 stated that she did not assess the quality of research because the judgements would be too subjective. Similarly, T7 did not examine the quality of data and the content of in-text citations in detail because she was not an expert on the subject that students researched. Furthermore, even though T4 focused on interestingness of the content, he mentioned that say- ing “your ideas are not good” would demotivate students. These concerns need to be addressed if content quality is included as an assessment criterion for essays and presentations. It is necessary to carefully consider whether lan- guage teachers who are not necessarily experts of the subject can judge the degree to which the content is original, intriguing, and informative.
Finally, a criterion that may be unique to assessment in CLIL among approaches to language learning was content accuracy or content mastery (see Llinares et al., 2012). This aspect of content was not assessed through research papers and presentations. Only T8 examined this aspect of essays and presen- tations, in which students described their successes and struggles using SLA concepts (essay) and explained an SLA concept they learned in the course (presentation). T8 intended to examine students’ understanding of SLA,
which is an objective of his CLIL course, through these assessments. Content accuracy was assessed not only in a task requiring students to demonstrate understanding of the content but also in a task in which students needed to apply conceptual knowledge in explaining a new phenomenon. Other teach- ers examined content mastery using different assessment instruments, such as paper-and-pencil tests (tests in which students read questions and respond to them by writing or selecting the correct answer).
To summarise, in their CLIL courses, the majority of the teachers employed essay and presentation tasks that required students to research and demon- strate findings or argue their opinions. Through these tasks, teachers assessed not only language aspects of texts but also higher-order thinking skills includ- ing logicality or interestingness of ideas. However, some teachers voiced their concerns about assessing the quality of content, including the quality of research findings, ideas, and information from research, in essays and presen- tations. Moreover, content accuracy and mastery of the content learnt in the course was seldom assessed through research papers and presentations.
Although Study 1 identified a limited range of assessment tasks implemented by the teacher participants in their CLIL courses, it must be noted that a wider range of types of tasks to assess language, content knowledge, and criti- cal thinking does exist, as discussed in the previous section, Assessing Content and Critical Thinking.