Although the district itself has an evaluation department, the team is small and time limited. Due to the district's preferred focus and existing knowledge, our project focuses exclusively on the district's eight middle schools and examines the middle school's purposes, practices, and products.
AREA OF INQUIRY IN THE ORGANIZATION
This is evidenced by the general sentiment among district and school leaders who question whether the program has any value anymore and by the lack of robust academic and other data to determine the program's effectiveness. Second, there is currently no formal evaluation process to measure the desired outcomes that the program produces.
LITERATURE REVIEW Problem Context
Knowledge and Design
However, evidence of the impact of summer programs on student achievement is currently inconclusive, partly due to program variability. The consensus on which of the best practices described above are non-negotiable remains elusive and contextual, but it is clear that summer programming has the potential to be transformative.
CONCEPTUAL FRAMEWORK Purposes, Practices, and Products
Even more consistently pressing have been issues of funding, as tight budgets, high costs, and bureaucratic red tape cause headaches and make administrators question the payoff of summer learning (McCombs, Augustine, & Schwartz, 2011). CCPS's summer learning approach aligns with the conceptual framework's logical model, in which summer learning loss is reduced for low-achieving and at-risk learners by.
QUESTIONS
What is the nature and impact of middle school summer programming within the district?
Is there a relationship between summer programming and student performance?
PROJECT DESIGN AND DATA Evaluation Design and Approach
The student dataset was divided into samples of summer program participants and non-participants. Additionally, we disaggregated and analyzed recent trends in summer school attendance and non-attendance to understand the demographics of summer programming.
WHAT IS THE NATURE AND IMPACT OF MIDDLE SCHOOL SUMMER PROGRAMMING WITHIN THE DISTRICT?
Question 1A: Purposes
What are understood to be the purposes of summer programming?
Findings
34; Academic focus and goals were the main purpose; we wanted to close the gaps between students and start students next year." (teacher). However, it also seems to stem from the role of the district and schools in planning summer programs. 34; The goals were never clearly communicated either to the school administration or to the district level, and I believe nothing formal or official has ever been announced to date." (district manager).
The goals are to stay connected with our children and to extend or supplement their learning from the previous year, knowing that they will need support moving into the next year." (teacher). There were secondary goals, But that [goal] was so monumental that this is all it has become." (lecturer). It is a combination of leading and honestly supporting the leadership in others to move in a positive, forward direction.”
Question 1B: Practices
What practical and pedagogical elements define summer programming?
While the overall structure of summer school remains similar at each of the eight schools, nuanced differences in the program's purpose, curriculum, instructional strategies, and evaluation may challenge the district to determine the overall effectiveness of the program. The goal was to keep parents abreast of their child's attention, effort, and work completion, although none of the reports were grade-based. With very little accountability present during the summer, parent involvement would help make summer school more impactful.” (teacher).
Teachers and administrators across all schools cited several other components of the program — class sizes, food, transportation and student engagement — that are essential to summer school's current functioning, but that need to be improved to make summer school what it should be. Citing background knowledge from many of the families of students who attended summer programming, teachers view the food supply as a potential gap in the summer. Several teachers also noted the impact of the stigma of summer school: "It was very clear that students knew they were the low learners, given their consistent participation in summer [school] over the years, so it could be discouraging" especially for students attending multiple years.
Question 1C: Products
Most agreed that this was due to declining interest and motivation, especially as the older groups were often made up of students who had participated for several years.
What are understood to be the outcomes – intentional and unintentional – of summer programming?
Several teachers discussed the secondary academic benefits of keeping students engaged in the academic environment during the summer. There was definitely a warmer SEL feeling and connection with the people in the building and the high school as a whole over the next year.” (teacher). They were almost like a brand ambassador or an influencer kind of thing, where they were able to act in a leadership role [the next year].” (teacher).
This study was conducted during the 2020-2021 school year – a year that was significantly impacted by the COVID-19 pandemic – and although it focused heavily on the years before the pandemic, several points of interest emerged during our interviews with teachers and principals. . The teachers interviewed, who also taught during the summer of 2020, noted an almost universal shift toward content and real-world project-based style learning. Teachers also felt that they were able to create more fun opportunities because of the open field of planning it afforded them.
IS THERE A RELATIONSHIP BETWEEN SUMMER PROGRAMMING AND STUDENT PERFORMANCE?
Indicative of ever-present racial opportunity and achievement gaps, students of color are more likely to attend summer programming, possibly because they are more likely to have lower standardized achievement scores that prompt referral to summer school. Black and Hispanic students comprised 78% of the total students of color represented within the test score data and are the primary focus of divergent analyses. The same meaning suggested in the tables above for all students of color is evident when specifically considering black and Hispanic students.
Question 2A: Do students who attend summer programs exhibit stronger academic achievement compared to students who attend?
Question 2A: Do students who attend summer programming exhibit stronger academic achievement gains in comparison to students who do
Conceptualizing learning loss as a negative scaled score change from fall to spring, we calculated differences between participants and nonparticipants (see Appendix E for all learning loss tables). In 2016, a smaller proportion of participants showed learning loss, while the opposite was true in 2017 and 2018. Across at-risk subpopulations, differences between participants and non-participants were only significant in 2018, with each subpopulation showing a higher percentage of students with learning loss in the participant group (see Appendix E).
It is possible that this learning loss in the participant group may reflect the typically lower performance of remedial students attending summer programs rather than the quality of the summer programs themselves. While the total sample of students in 2018 showed a significantly lower percentage of students with a learning loss in the participant group, there were no subpopulations. Mitigating learning loss is not enough to close gaps in opportunity and achievement.
Question 2B: Do students of color exhibit academic gains after attending summer programming?
Hispanic student attendees and non-attendees did not show statistically different scores on any year of the STAR exam. This is again representative of the overall data, which showed significant growth gains for program participants on the MAP assessment. Our deleted data did not provide information about which—if any—of the Hispanic students in our data set were also English language learners, but a future evaluation of summer programming may choose to differentiate between Hispanic ELLs and non-ELL Hispanic students. e is not, especially on the STAR exam, which is based on language (reading) skills.
Question 2C: Does attending for multiple summers enhance the program’s impact on achievement?
We looked at the quantitative data from multiple angles and didn't come up with a clear picture of what happened in summer programming between 2016 and 2019. While there are some potential bright spots — especially in 2016 — there are no obvious trends or predictive conclusions to be made.
Analytical Approach Results
- Clarity: Provide a Clear and Consistent Program Purpose
- Recommendations
- Capacity: Adopt Best Practices and Expand Resources
- Community: Engage Parents and Community Partners
- Continuous Improvement: Formalize Data Collection and Program Evaluation
First, we recommend that the district provide a clear and consistent vision, purpose, and set of goals to guide the implementation of the school program. Third, we propose that the district engage with the community through parent engagement and external partnerships. We recommend that the district use its existing leadership structures, including frequent principal meetings and the LOD team, to develop a clear and comprehensive vision and mission for.
The district already provides transportation to incoming sixth graders for the Jump Start orientation program, and we recommend expanding this resource to all middle school students for summer programs. Well before summer, teachers from across the district should also come together to make plans together. The specific selection of these data points should be directly tied to the district's and schools' established vision and SMART goals, allowing for a clear, quantitative snapshot of the programs each year.
CONCLUSION
In addition to the data noted above, this evaluation process should include qualitative input from all stakeholders (students, parents/families, teachers, administrators, .community/external partners), allowing shared experiences and potential changes to emerge. Despite these current shortcomings, nearly all stakeholders communicated value in summer programming and advocated for its permanence. High-quality summer school programs require a sustained and significant financial investment and require ample planning long before the end of the year.
Successful programs require continuous improvement—evaluating what works and being willing to try new ideas. Our proposed intervention model incorporates the most prominent research-based strategies, which we have considered the four Cs—clarity of program purpose, capacity building through adoption of best practices, and parent and partner engagement. of the community and developing a cycle of continuous improvement. Perhaps even more than specific interventions, this assessment provides an opportunity to shift another element of the district's educational work toward a culture of continuous improvement.
Resources
Longitudinal achievement effects of multiyear summer school: Evidence from the Teach Baltimore randomized field trial. Impacts of a Summer Learning Program: A Randomized Assignment Study of Building Educated Leaders for Life (BELL). The COVID-19 slide: What summer learning loss can tell us about the potential impact of school closings on student academic performance.
Effects of high school summer academic programs on high school test scores, course taking, and college majors. National Summer Learning Association (NSLA). 2012) Best Practices in Summer Learning Programs for Middle and High School Youth. The case for summer learning: Why supporting students and families throughout the year is vital.
APPENDICES Appendix A
Are there any effects of summer programming that are not explicitly stated in the objectives? Have you noticed the impact of summer programs on student attendance in the next school year? If the summer program were to be eliminated, what impact do you think it would have on the students who would otherwise attend.
What other capacity building resources do you provide to schools to help with summer planning? How the district determines which summer program changes to make and when. Are you aware of any changes to summer programs that were planned prior to COVID?
