I am pleased to be writing the foreword to this report on the national pilot evaluation of KidsMatter: The Australian Primary School Mental Health Initiative. It requires a whole-school approach and has four key components – a positive school community; social and emotional learning for students; parental support and education and early intervention for students with mental health difficulties.
KidsMatter Executive Summary
The KidsMatter Initiative
KidsMatter impact overview
Background to the KidsMatter Evaluation
Research covered student mental health, KM inclusion and implementation, and impacts on schools, teachers, parents, and students. The findings of the analysis of the qualitative data are also presented at the appropriate points after the analysis in relation to the main topics and requirements of the evaluation.
Impact of KidsMatter on schools and teachers
Positive school community. Parents and teachers provided high ratings for their school’s performance on
However, at the end of the trial, 10% more teachers strongly agreed that their school was effective in providing early intervention. 16% more teachers strongly agreed that their curricula helped students develop social and emotional competencies.
Impact of KidsMatter on family context
14% more teachers strongly agreed that they knew how to help their students develop social and emotional competencies. 8% more teachers strongly agreed that school staff as a whole acted to help students develop social and emotional competencies.
Impact of KidsMatter on student competencies
Impact of KidsMatter on student mental health
This represents a positive change for approximately 1 in 5 students who were initially in the abnormal and borderline range. This represents a positive change for approximately 1 in 3 students who were initially in the abnormal and borderline range.
Conclusions
In particular, the classification of students using the extended criteria showed a greater impact on students who were at the borderline than suggested in the SDQ difficulty analysis, possibly because KM was able to build on students' existing strengths while simultaneously reduce the difficulties. Furthermore, it is possible that the classification using the extensive criteria has resulted in a more targeted recognition of students in the deviating range.
Recommendations
Be aware of the different manifestations of student mental health at home and school and the implications of these attitudinal differences for students, teachers, and parents. This may include supplementing existing advice on ways parents and teachers can share their concerns and strategies to help students at risk of or experiencing mental health problems, so that compatible approaches can be implemented at home and at school.
Development and Background to KidsMatter
- What is KidsMatter?
- Who developed KidsMatter?
- The KidsMatter framework
- Positive school community: This component encourages schools to engender a sense of belonging and inclusion in members of their communities, by providing a welcoming
- Social and Emotional Learning for students: This component is designed to help schools select and enact a clearly structured Social and Emotional Learning curriculum
- Parenting support and education: This component focuses on the school as an access point for families to learn about parenting, child development and children’s mental
- Early intervention for students experiencing mental health difficulties
- Who conducted the evaluation?
- The KidsMatter Pilot Phase
- What is contained in this report?
These outcomes are expected to include changes in both levels of risk and protective factors and students' mental health. Similarly, providing parenting information and support is designed to have a positive impact on parenting abilities, which may also help students' competencies and mental health.
Evaluating KidsMatter
- The three-pronged approach to the evaluation
- KidsMatter outcomes
- KidsMatter processes
- Multiple stakeholder perspectives
- Specific areas of evaluation
- Overview of the evaluation design and participants
- The KidsMatter schools
- Participants in the evaluation
- Ethics approvals
- Design of the Whole Cohort Longitudinal Questionnaire study
- Design of the Stakeholder and Student Voice studies
- Interviews and focus groups
- Legitimacy and trustworthiness of interview and focus group data
- KidsMatter Project Officer Reports: An external school perspective
- School Leadership Executive Summaries: A perspective from leadership
- Summary of all data collected
- Analysing and reporting the results
- Round 1 and Round 2 schools
- Confirming the scales
- Measuring change over the time of KidsMatter
- Statistical significance and effect size
- Chapter summary
A summary of the characteristics of the participants in the KM assessment is provided in Table 1. This ensured that all aspects of the assessment were represented in the teacher and parent questionnaires.
Engagement with KidsMatter
Whole Cohort Study and Project Officers
- Teacher perspectives on engagement with the four KidsMatter components
- Teacher perspectives on use of the 7-Step implementation process
- Parent perspectives of KidsMatter implementation
- Evidence about implementation of KidsMatter from the Project Officer Reports
- Professional development as an element in KidsMatter implementation
- General engagement with students’ mental health and wellbeing
- Chapter summary
More than half (52%) of teachers in 1st round schools and 60% of teachers in 2nd round schools rated professional development as. At the same time, only 6% (Round 1) and 5% (Round 2) of teachers nominated results below the neutral level (grade 4) for the quality of KM professional development.
An Implementation Index
- An implementation framework
- School profiles on the Implementation Index
- Implementation quality and meeting the needs of students
- Chapter summary
The school teaches students SEL skills (T) The problems defined (T) Goals set (T) Difficulties identified (T) Strategies developed (T). The Implementation Index indicates that there were significant differences in the extent to which schools made progress in the 7-Step implementation process, involving the involvement and support of parents, staff and school leaders. The findings provide further evidence that there was a positive relationship between the Implementation Index score and the assessment of the school's ability to meet students' social, emotional or behavioral needs.
Engagement with KidsMatter and the Implementation Process: Stakeholder
Key themes from the Stakeholder and Student Voice studies
- Facilitators and barriers
- Strategies and complexities associated with implementing the four components
Change of leadership "While I was gone, the person who took my role was obviously trying to handle all sorts of things, so KidsMatter was not one of the balls he was able to keep in the air." Head School 5. Make sure the teachers are engaged and maintain it from a collaborative point of view, because if it's not done collaboratively, it won't work." Head of School 6. I think I would do it differently… Really hone it and just be more realistic.” School 9.
Possibilities for sustainability
- Impact of the Project Officers
- The Student Voice study: The views of students on KidsMatter implementation and engagement
Comments were also made on the role of project officers in the management summaries obtained at the end of the KM. Included in these reports were two questions asking project officers to rate the extent to which (a) the school's leadership team was satisfied with having them in the school and (b) teachers were satisfied with having them in school. It's about five keys: organization, self-confidence, resilience, perseverance... five keys to... happiness." Student School 5.
Chapter summary
All students who participated in the Student Voice study appeared to have some knowledge of KM. We gave them a crown task, where you put on a crown and you are a queen or a king, what would you change for the children of the world.''
Positive School Community
Implementation, Engagement and Impact on Schools
- Engagement with and implementation of Positive school community
- Impact on Positive school community: Whole cohort study
- Impact on Positive school community: Stakeholder interview and focus groups
- Impact on Positive school community: Parent questionnaire responses
- Chapter summary
In fact, by the end of KM, 64% of teachers across all schools strongly agreed that “the professional development related to KidsMatter has better equipped the school to develop a positive school community.” The question of the impact of KM on the positive school community was examined in the multilevel modeling analysis using HLM. This was intended to get a measure of the impact of KM on parents' positive school community.
Social and Emotional Learning
Engagement with and implementation of Social and Emotional Learning: Whole cohort study
At Time 4, 71% of teachers in Round 1 schools and 66% of teachers in Round 2 schools 'strongly agreed' (score 6 or 7) that “Our school has implemented plans to support social and emotional learning for students to develop”. However, the equivalent figure for Round 2 schools of over 40% more teachers than at Time 1 suggests that, despite starting further behind, Round 2 schools quickly caught up. By the end of KM, 70% of teachers in Round 1 schools and 60% of teachers in Round 2 schools strongly agreed that “the professional development related to KidsMatter has better equipped the school to manage socially and emotionally to develop learning for students. ”.
- Engagement with and implementation of Social and Emotional Learning
An even greater difference was evident in the second item, which examined the extent to which schools had implemented plans for component 2: social and emotional learning for students. Teachers were asked to rate the extent to which professional development had better equipped the school to deliver social and emotional learning. Teacher responses are presented in Figure 15 and show the effect of delayed start for Round 2 schools.
Stakeholder and Student Voice studies
Impact on schools’ provision of Social and Emotional Learning: Whole cohort study
- Student participation in SEL programs
- The Programs Guide
- Student knowledge of SEL from the Student Voice study
The school teaches social and emotional skills to all students regularly (at least once a week) 62% 56%. The school's resources for teaching social and emotional skills meet the needs of our students 54% 43%. The majority (15) programs selected were focused on Component 2: Social and Emotional Learning for students.
Chapter summary
Tell the teacher that he had problems ... talk calmly with someone he knows and with someone he feels safe with." Student School 4. Discussing Coping Strategies "He needs to talk to someone because you can't let it build up inside." Student School 5. Take a deep breath and just try to calm down for a few minutes...go talk to someone about why you're sad and they can probably help you." Student School 7.
Parenting Support and Education
- Engagement with and implementation of Parenting support and education
- Impact on Parenting support and education: Whole cohort study
- Project Officer reports on information to parents
- Parent involvement with schools during KidsMatter
- Impact on Parenting support and education: Stakeholder responses
- School as a point of information access
- Linking mental health with learning outcomes
- Effective parent-teacher relationships
- Chapter summary
However, 22% more teachers (equivalent to a medium to large effect) strongly agreed that the school provided parental support and education. Overall, these findings indicate a positive impact of the overall trial on the level of parental support and education provided by the school. Questionnaire scores indicated some improvement in the provision of parental support and education during the KM period.
Early Intervention for Students
Experiencing Mental Health Difficulties
Implementation, Engagement and Impact on Schools
- Engagement with and implementation of Early intervention for students experiencing
- Impact on Early intervention: Whole cohort study
- Project Officer reports: Links with external agencies
- Impact on Early intervention: Stakeholder responses
- External agency support
- Changes to school culture around mental health
- Chapter summary
The items mainly emphasized the target areas of identifying and supporting students with mental health problems. Improved links with external agencies supporting children with mental health problems: Responses from project staff. Teacher ratings show that KM generally has a positive impact on the provision of early intervention for students with mental health problems.
KidsMatter Impact on Teachers
- Teacher competencies
- Staff approach to Social and Emotional Learning
- Staff attitudes to Social and Emotional Learning
- Teacher knowledge about Social and Emotional Learning
- Teacher SEL programs and resources
- Teacher self-efficacy
- Impact of the professional development
- Impact on teachers from the Stakeholder interviews
- Chapter summary
The questions in this section are about teaching, including teaching social and emotional skills. Table 22 shows that at Time 1 over 80% of teachers gave 'strongly agree' ratings for the items on staff attitudes to Learning Social and Emotional Skills. The evaluation results showed that many teachers at the start of KM already demonstrated a medium to high level of knowledge, competence and confidence in social and emotional learning and in supporting students with mental health problems.
KidsMatter Impact on Family Context
- Parenting knowledge
- Parenting style
- Overall effectiveness as a parent
- Impact on parents’ awareness of children’s social and emotional needs
- Parent interview responses about the impact of KidsMatter
- Chapter summary
Overall, therefore, parents expressed confidence about their parenting knowledge and this did not change over the period of KM. Parents gave high ratings to their overall effectiveness as parents at Time 1 and this did not change over the period of KM. Parents gave high ratings of their parenting effectiveness and knowledge at the start of KM and this changed little over the period of the trial.
KidsMatter Impact on Student Competencies
Impact on student social and emotional competencies
At school "I would probably say it's made kids think about if they do this ... how is it going to affect the other kid. You're not going to get along with everybody ... and you just have to deal with it." Coed 1. We used to have people in our school who used to bully everyone and thought they were in charge of the school...and our biggest bully in school...now she has been out of school and trying to make friends with people." Pupil school 7.
Impact on student schoolwork
- Stakeholder responses about student competencies and learning
- Stakeholder responses about mental health and student learning
- Improvements in ratings of the child’s schoolwork
Chapter summary
KidsMatter Impact on Student Mental Health
- The Strengths and Difficulties Questionnaire
- Total SDQ outcomes for all students
- Changes in the Total SDQ Difficulties normal, borderline and abnormal ranges
- Impact on SDQ subscale scores for all students
- Changes in SDQ Emotional symptoms
- Changes in SDQ Conduct problems
- Changes in SDQ Hyperactivity
- Changes in SDQ Peer problems
- Changes in SDQ Prosocial behaviour
- Student mental health difficulties: An alternative measure
- Student mental health strengths
- Two dimensions of mental health: Difficulties and strengths
- Change in the proportion of children in each range according to the SDQ mental health difficulties
- Change in the proportion of children in each range according to mental health strengths and difficulties
- Chapter summary
Change over time in SDQ Total Difficulties for students in the normal, borderline, and abnormal ranges. For students in the abnormal range, mental health difficulties were reduced to an extent equal to a medium effect size according to teachers and a small effect size according to parents. Averaged across all students in the sample, changes in mental health showed small effect sizes.
KidsMatter in Perspective
- School based interventions: International perspectives
- Promotion, prevention and early intervention
- Working with four components
- The change process
- The KidsMatter conceptual model
- Dimensions of student mental health
- Sustaining KidsMatter
- Portrait of a successful KidsMatter implementation
- Considerations for further initiatives and research
Margaret called a meeting of the Action Team, which now also included the school's well-being coordinator and the school's adviser. At the beginning of the second year, the school put several of their plans into action. They set up a case management team consisting of the principal, key staff and the school counsellor.
Conclusions and Recommendations
Conclusions
It was also evident that implementing components 3 and 4 presented a challenge for many schools. Although there was some variation in the pattern of findings for schools involved in KM for one year and for schools involved for more than two years, the nature of the intervention makes it difficult to interpret or explain the variation. However, one clearly evident factor was the development of the expertise of the KM team in general and of the KM project officers in particular during the first year.
Recommendations
This meant that the introduction of KM in Round 2 schools in 2008 benefited greatly in terms of being able to access the growing base of available knowledge and resources generated by KM activities in Round 1 schools in 2007.
Glossary of Key Terms
The four KidsMatter school-based components
Descriptions of the scales used in the Whole Cohort Longitudinal study
Nature of the intervention
Sampling
Instruments
Duration of the study
Participation and non-participation
Non-participating parents versus participating parents: For the group of parents who chose to participate, there were fewer students nominated 'at risk' and fewer Aboriginal or Torres Strait Islander students. Non-participating teachers versus participating teachers: For the group of teachers who chose to participate, there were fewer older students and fewer students from culturally and linguistically diverse backgrounds. Accordingly, this places a general caveat on the findings that they are not more broadly representative of male students, of those identified as 'at risk', and of students from Aboriginal or Torres Strait Islander, or from culturally and linguistically diverse backgrounds .
Common method variance
The analysis suggests that respondents are not missing at random and missing values should be replaced to minimize potential bias.
Scope of the analysis of change
Related Publications and Presentations
Children's sociable and aggressive behavior with peers: A comparison of the USA and Australia, and the contributions of temperament and parenting styles. Mental health of 13-17 year olds in Australia: Findings from the national survey of mental health and well-being. Retrieved September 4, 2009, from http://www.who.int/mental_health/evidence/MH_Promotion_Book.pdf.
KidsMatter Partners thank the following school communities
New South Wales
Victoria
Queensland
Western Australia
South Australia
Tasmania
Australian Capital Territory
Northern Territory
