In order to illustrate the application of the proposed framework, the rubric was used to assess Year 7 students’ history knowledge in a state school follow- ing a CLIL programme, as part of the bilingual education programme in Madrid, Spain. The implementation of bilingual programmes in the Madrid region started at the end of the 1990s (Llinares & Dafouz, 2010) in an attempt to improve not only students’ acquisition of foreign languages (mainly English) but also the learning of content subjects. The school selected for the present study was considered an optimal setting for the implementation of an innovative methodology in the teaching of content and language using a CLIL approach, since it had developed a CLIL programme for six school years and it was an example of close collaboration between English language and CLIL content teachers.
The target individuals in the study were 25 students from the same Year 7 secondary school (12 years old). 88% percent of the participating students were Spanish and 12% were Moroccan. All of them had followed an English–
Spanish bilingual programme during their primary education (from 3–12 years old), prior to their access to secondary education. At the moment of the study, most participants had an A2—three of them had a B1—CEFR level of English.
The CLIL subjects they had in this particular school were English language, history, geography, science, arts, technology and physical education.
In addition to the analysis of students’ written essays in response to the prompt, using the proposed rubric, a 10-statement true–false test was also provided (see Appendix B). These types of tests are more frequently used as assessment instruments by history teachers in the Spanish context than the essay format. A true–false test is easier for teachers to mark and students to answer. Even though these tests limit critical thinking analysis, they constrict the responses and make the answers more accurate, avoiding misinterpreta- tions. The statements are both ordered chronologically and from the easiest to the most complex, so all students have the opportunity to answer some of the statements and prove their historical knowledge.
Although a written essay on the topic contributes better to deepen students’
knowledge of the contents studied, it cannot be ignored that an essay is more difficult to code and analyse. It depends largely on how students manage to express what they know about the subject they are learning (historical literacy), expressions which may generate misunderstandings or other factors that may affect the quality of students’ responses (May, 2011). For this purpose, a rubric (Appendix A) was designed following the CDFs mentioned above in order to evaluate students’ responses as effectively as possible. The rubric assesses the students’ historical knowledge and language abilities to express the content.
Both types of assignments, a more traditional true-–false (T/F) test and a more open task requiring students’ production (essay), were supplied, in order to compare students’ performance in both types of assessment tasks, once objectivity in the assessment of the essay was guaranteed by means of the rubric. The results of the scores obtained by the students in both assignments (test and essay) are presented in Table 1.
The mean in the test scores was 5.36, lower than the one in the essay: 6.60 (Table 1). This result could be tentatively related to some experts’ agreement that developing writing skills in any topic contributes to activate the neurons and this benefit is expected to have a positive effect on both the language used to write and the subject content (Costa, Calabria, & Baus, 2019). In this study, the median was equivalent to the mean (5 in the test, 6 in the essay), which proves that the dataset was more or less distributed from the lowest to the highest values. The low standard deviation confirms this: most of the
scores are next to the mean. The mode in the essay showed that the value that appeared more often was 8, higher than the one of the test: 5. These results seem to indicate that students’ writing in history helps them to show content knowledge and, thus, reveal the key role that language plays in the expression of that knowledge. The results, thus, tentatively support the positive effect of an assessment instrument that allows students to explain what they have learned and enhances their reasoning, beyond the mere learning of historical events as illustrated in a T/F test.
As shown in Fig. 1 and Table 1, more than half of the students obtained better results in the essay than in the T/F test (56%). This proves the effectiveness of assessing linguistic and historical performance following a CDF model (Fig. 1).
There was also a correlation between the results in both tests and the length of the essays: students who got good marks in both wrote longer essays (Student 14, 94 words; Student 20, 98 words; Student 23, 144 words) and those whose
Table 1 Students’ scores in the T/F test and essay
Students’ scores T/F test Essay Diff
Student 1 6 8.33 2.33
Student 2 7 4.83 −2.17
Student 3 7 4.16 −2.84
Student 4 5 7.5 2.5
Student 5 5 9.16 4.16
Student 6 5 4.16 −0.84
Student 7 2 8.33 6.33
Student 8 7 8.33 1.33
Student 9 8 5 −3
Student 10 5 5.33 0.33
Student 11 3 4.63 1.63
Student 12 5 3.33 −1.67
Student 13 6 3.33 −2.67
Student 14 10 8 −2
Student 15 3 6.6 3.6
Student 16 5 3.33 −1.67
Student 17 5 3.33 −1.67
Student 18 7 8 1
Student 19 2 3.33 1.33
Student 20 9 6.63 −2.37
Student 21 3 6 3
Student 22 3 7.33 4.33
Student 23 7 9.33 2.33
Student 24 6 6 0
Student 25 3 3.33 0.33
Mean 5.36 6.60
Median 5 6
Mode 5 8
St Deviation 2.12 1.34
Variance 4.49 1.80
marks were lower produced shorter essays (Student 19, 52 words; Student 25, 37 words). Most students used the correct format and suggested vocabulary for the essay, which activated recalling the historical contents previously studied.
However, the lowest scoring students often used simple and coordinated clauses:
Columbus discovered America but he was looking to another way to reach Asia.
He had 3 caravels, the Santa Maria, the Pinta and the Niña. He arrived to America the 12th of October, to San Salvador. (Example 1, Student 25)
Students’ use of subordinate clauses with time, place, cause and conse- quence linkers showed how the learning of history contents through a foreign language stimulates their report and explanation skills (e.g., explanations through because):
The discovery took place the 12th of October, 1492. Transport of Columbus were 3 ships: the Santa María, the Pinta and the Niña. The 3 ships sailed on August 3, 1492 from the port of Palos (Huelva). After many days, sailors began to get impatient because they saw nothing but water. The October 11 began to appear in the sea grasses and wood chips indicating that the land had to be very close. (Example 2, Student 20)
When and where events happened is an important issue in history learning.
Not all students remembered dates and that was not a requirement in the essay, but many knew where to locate the expeditions to America, the direc- tion of the trips and one of them even set the starting point (Palos de la Frontera), as in the example from Student 20 above.
Ten students (40%) got better marks in the test than in the essay. However, it needs to be taken into account that a test is more prone to random answers.
0 2 4 6 8 10 12
student1 student2 student3 student4 student5 student6 student7 student8 student9 student10 student11 student12 student13 student14 student15 student16 student17 student18 student19 student20 student21 student22 student23 student24 student25
Test Essay
Fig. 1 Students’ score distribution in the T/F test and the essay
In any case, only in half of them the difference was higher than 2 points. In contrast, in the case of the students who scored higher in the essay, the differ- ence was higher than 2 points in most of the cases (8 out of 14). There were two students (Student 19 and 25) whose scores were very low in both the test and the essay—between 2 and 3 points; their essays were also short (30–50 words), unfinished or inconsistent.
It seems, then, that students with good writing skills are able to formulate arguments supported by reasoning and evidence better than in a T/F test, where they can get confused by the small piece of information, often unim- portant, that makes the whole statement false. These results suggest the inter- est in combining assessment approaches to give students a variety of formats in which they can show their history knowledge.