This report evaluates a development project designed by School 21 and the University of Cambridge to improve the oral skills of Year 7 students. What further development of the Oracy Curriculum, Culture and Assessment Toolkit is needed and would allow for a more robust evaluation of its impact.
Introduction
What was the impact of the pilot on the development of students' word skills. The Language and Literacy Research Group at Sheffield Hallam University contributed to the review of the Voice 21 website.
Methodology
A key feature of School 21's approach to public speaking is the development of the entire school public speaking culture. In addition, the assessment team met with the University of Cambridge team to review the use of the Oracy assessment toolkit and the work being done by the team to assess the validity and reliability of the assessment tasks.
Findings: Impact
There is a less clear, but positive, indication of progress at school 21 in relation to the pair teaching task. Ratings for School 21 were higher for all follow-up assessments than for the comparison school, where rates of progress were lower than for School 21.
Findings: Oracy Skills Framework
For example, School 21 told us that they initially focused on the cognitive part of the frame, but teachers identified a need to develop the physical strand (pitch, eye contact, etc.) and this led to a focus on performance poetry. The framework is also used explicitly with students to support self- and peer assessment. We have tried to get the 4 threads to permeate all aspects of the work, when they prepare [the Ignite speech], reformulate it, then they train each other, the children are able to explicitly say the area in which they think they are strongest in and the area they think they are weakest in.
The framework is used to inform approaches that encourage students to think about what might be appropriate in different contexts. The framework is designed to support a recursive curriculum, revisiting different skills in relation to increasingly challenging content and contexts, understanding the context in terms of the purpose of the conversation and the type and size of the audience . The framework appears well designed as a basis for developing speaking curricula that will support progress in speaking skills.
At School 21, the framework appears to have supported staff in developing a broad and exciting curriculum for their students to provide a range of targeted contexts for conversation. Furthermore, there is a risk that Skole 21's emphasis on providing different contexts for proverbs may be missed when foregrounding the framework for the presentation of the curriculum.
Findings: Dedicated Year 7 curriculum
Support for the development of conversation for formal presentational purposes appears to be a strong point of the curriculum. During the student focus groups, students spoke confidently about the different strands of the framework, but did so primarily in terms of introductory speech. Teachers talked about the value of group discussion, for example, and we saw many examples of opportunities for discussion and teachers providing support for students to engage in group discussion.
While these are elements worth exploring, this suggests that the cognitive strand is understood in terms of knowledge communication with less emphasis on exploratory talk. Given the developmental nature of the project, we cannot come to a final judgment on the effectiveness of a dedicated Year 7 public speaking curriculum. A distinctive quality of a purposeful curriculum for Year 7 is the recognition of a range of skills that can be used in different contexts, with an emphasis on making appropriate decisions about how to draw on them appropriately in these different contexts.
This would require clarifying the value of both formal and informal opportunities for conversation and supporting schools to provide opportunities for students to use conversation for learning, ie. Addressing this will involve a deeper focus on the cognitive strand of the framework.
Findings: Oracy in every lesson
Creating a climate in which conversation is valued would certainly appear to be an important condition for exploratory conversations to flourish. While these reflections were helpful, the emphasis on presentation raised questions for the evaluation team about the place of exploratory conversation. They may have been less supportive of the freer investigative speech described in the opening of this report.
It is therefore not possible to draw any conclusions here about the quality of inquiry conversations across the school, but it is worth noting that this may be an area for further exploration, particularly in light of the comments about the curriculum in section 5.1. The emphasis on meaningful contexts for conversation matches the focus on meaningful contexts for learning across the school curriculum. Teacher interviews and lesson observations suggested that a commitment to promoting orality across the curriculum in every lesson is a strong feature of the School 21 approach.
Although students are keen to participate in the discussion and debate with the evaluation team, we cannot draw any conclusions about the quality of the exploratory conversations (as we were only able to conduct a small number of observations). It appears that the development of exploratory conversations is worth further investigation, especially in light of the results of the Raven's Matrices tests and our evaluation of the curriculum described in Chapter 5.
Findings: Whole school oracy culture
The development of oratory is closely related to other aspects of the school's curriculum and pedagogy. Other emphases in School 21 on well-being, leadership and research, for example, are seen as reinforcing and benefiting from the oral curriculum. It is also acknowledged that the size of School 21 and the school's whole culture of word-of-mouth makes positive engagement much more likely.
It is also worth noting the reciprocal relationship between the public speaking curriculum and other provisions in School 21, where the development of public speaking is closely related to other aspects of public speaking. School 21's position as a new school with a small population may also mean that positive engagement is more likely as staff and students join a school where speaking is emphasized from the start. These school-specific factors raise questions about the extent to which the School 21 approach can be transferred to other settings.
However, it may be that its success depends on many aspects of the very specific conditions at School 21. Any further evaluation of the scale-up of the intervention will need to consider how School 21's orality approach is interpreted in schools. others and in the opportunities and barriers that arise during the implementation of these approaches in other countries.
Findings: Oracy Assessment Toolkit
The reliability of the Oracy Assessment Toolkit was independently assessed at Sheffield Hallam University through a teacher workshop. For each task, teachers were given an overview of the task and instructions produced by the University of Cambridge team and were also used in their testing of the Oracy Assessment Toolkit. Data from the Cambridge team's evaluation of the Oracy Assessment Toolkit suggests that there is variability in validity across specific tasks.
There is also a link to a document containing a glossary of the skills highlighted in the Oracy Skills Framework. This will aid the understanding of the function and usefulness of the Oracy Assessment Toolkit. The following section of the website provides an overview of the Oracy Assessment Toolkit tasks.
There is good evidence from the University of Cambridge team that they take the reliability and validity of the Oracy Assessment Toolkit very seriously. The team also recognizes that proper training is vital to the reliability of the Oracy Assessment Toolkit.
Supporting CPD and resources
The structure of the site reflects the elements of the Oracy Curriculum, Culture and Assessment Toolkit ie. the use of the Oracy Skills Framework to guide the curriculum and assessment could also be usefully highlighted. Drawing on some of the rationales presented in School 21's final report (Fidoe, 2014) would begin to address this issue.
It would be useful to more clearly foreground and justify the key elements of the Oracy Curriculum, Culture and Assessment Toolkit, i.e. in this regard it would be useful to explicitly identify the sources that informed the development of the curriculum make. It is currently unclear why these sources were chosen and whether (and if so how) they informed the development of the curriculum.
The website includes sections related to each component of the Oracy Curriculum, Culture and Assessment Toolkit, although the relationships between these components could be more effectively marked. The effectiveness of the proposed CPD package needs to be evaluated when it is rolled out to other schools.
Conclusions
Below we draw conclusions about the main components of the Oracy Curriculum, Culture and Assessment Instruments, it is important to note that the components are highly interrelated and when used in combination, they are mutually reinforcing in terms of oratory development. Our qualitative research findings indicate that the Oracy Skills Framework has provided an appropriate and effective structure to support curriculum design and support the development of the Oracy Assessment Toolkit. The commitment to promoting orality in every lesson is a strength of the School 21 approach, as is the emphasis on providing meaningful contexts for speaking across the curriculum.
Further piloting of the Oracy Assessment Toolkit with the suite of support materials produced in this pilot project would allow the suitability of the 7-point scale to be explored along with further testing of interrater reliability. The evaluation highlighted the need for teachers to receive training before using the toolkit and for some reorganization of the Oracy Assessment Toolkit website. The proposed CPD package to support other schools in the implementation of the Oracy Curriculum, Culture and Assessment Toolkit has not yet been tested.
Setting the boundaries of the intervention took time and it was only towards the end of the project that clarity emerged regarding the definition of what is now called the Oracy Curriculum, Culture and Assessment Toolkit. The piloting would also provide an opportunity to assess the effectiveness of the proposed CPD package and the Voice 21 website, and further refine and test the Oracy Assessment Toolkit to establish reliability and validity.
School 21 and the University of Cambridge Oracy Skills Framework
School 21's mapping of the dedicated oracy curriculum to the Oracy Skills
Parental information sheet (intervention)
Opt-in parental consent form (intervention)
Information sheet control (school)
Information sheet control (parents)
Opt-out control school consent (parent)
Control school consent (Senior leader)
University of Cambridge initial and end assessments rating schemes
University of Cambridge assessment task grading conversion to scores
