THE EFFECTIVENESS OF CONTINUING PROFESSIONAL DEVELOPMENT OF OFFICERS IN THE DISTRICT EDUCATION
OFFICE TO IMPROVE THE QUALITY OF DIFFERENTIATED GUIDANCE TO SCHOOLS IN FACING COVID-19 CHALLENGES
Dr. Suhaimi bin Ismail 1 Zahara binti Jaafar2
1Batu Pahat District Education Office, Malaysia (E-mail: [email protected])
2Batu Pahat District Education Office, Malaysia (E-mail: [email protected])
Abstract: This article describes the effectiveness of Continuing Professional Development in the Mersing District Education Office, which was implemented on a structured, practical and methodical approach to improve the quality of differentiated guidance provided to schools, particularly in light of the challenges of the Covid-19 pandemic. Under the District Transformation Program initiative, District Education Office must provide guidance to the schools that need it most and focus on professional development management. Despite unprecedented challenges, students' academic performance on public exams can be improved through effective differentiated guidance. The Mersing District Education Office applied Data Analysis Strategy Canvas (Razak and Nor, 2018) to identify what measures are needed to improve student performance in the 2020 SPM and STPM examinations.
Continuing Professional Development has been used to improve the professional skills of officials at Mersing District Education Office. The ADDIE model was used to ensure more effective delivery of differentiated guidance. The Kirkpatrick evaluation model was used to analyse and evaluate the outcomes of the programme. The result was an increase in the competencies of officials in implementing differentiated guidance that ultimately improved students' academic performance and success in the 2020 SPM and STPM examinations despite the challenges of the Covid-19 pandemic.
Keywords: (Differentiated Guidance, Data Analysis Strategy Canvas, Continuous Professional Development, ADDIE model, Kirkpatrick Assessment Model)
___________________________________________________________________________
1. Introduction
Shift 6 of the Malaysia Education Blueprint 2013-2025 states that State Education Department (SED), District Education Office (DEO) and schools are empowered to customize solutions based on need (Ministry of Education, 2013). This is done through systematic, district-led programmes rolled out across all states since 2014. Under the District Transformation Program, DEO are required to give guidance and support to schools based on the needs or issues that exist in the school. Due to the Covid-19 pandemic, DEO are faced with challenges unfamiliar to them, especially in providing guidance to schools. Officers in the DEO were faced with challenges in providing online support and guidance to schools.
Mersing is located in the northeast of Johore, 128 kilometres from Johor Bahru. Under the administration of the Mersing District Education Office there are 10 secondary schools, and 37 primary schools. The number of teachers is 1,457 and the number of students is 14,244.
There are 49 staff, consisting of 35 officers and 24 members of the support group. For expert support to the school, there are 2 School Improvement Partners (SIP+) and 4 School
Improvement Specialist Coach (SISC+). In the Sijil Pelajaran Malaysia (SPM) 2019 examination, there was a decrease in the District Average Grade from 4.97 to 4.98 (-0.01) and in Sijil Tinggi Pelajaran Malaysia (STPM) 2019 there was a decrease in the Cumulative Grade Point Average from 2.46 to 2.37 (-0.09). In both examinations, Mersing district ranked last out of 11 districts in the state of Johore. For the Penarafan Kecemerlangan Pejabat Pendidikan Daerah (PKPPD) or District Education Office Excellence Rating in 2019, the Mersing DEO got a score of 95.08% and was given a 4 Star rating.
2. Literature Review
There is a large body of international research on raising the quality of education, with particular emphasis on Continuing Professional Development (CPD) to support professional and pedagogical growth (Darling-Hammond, 2006). According to Giri et al. (2012), CPD is a systematic and ongoing process of education to ensure continuing competence extended knowledge and skills to new responsibilities. Ingwu et al. (2019) emphasised that CPD continues throughout the professional's life to keep up-to-date in recent developments and practices in the profession. The training model is a popular model of CPD (Little,1994) that provides teachers with the opportunity to update their skills with training from an expert. It is essential for employees to feel supported by their managers and company, in order to create a team that seamlessly executes their objectives. Employees undergoing training provides confidence and security in their roles, allowing them to work more effectively. When employees are trained, their efficiency and confidence in their work can improve performance, which in turn can aid in getting recognition for outstanding and fruitful work in their role and potentially more opportunities to rise up the ladder within the company will become available to them.
The Kirkpatrick Model of Evaluation, first developed by Donald Kirkpatrick in 1959, is a globally recognized method of evaluating the results of training and learning programs (Kirkpatrick, 2016). It assesses both formal and informal training methods and rates them against four levels of criteria: Reaction, Learning, Behaviour, and Results. Kirkpatrick's four levels are designed as a sequence of ways to evaluate training programs (Phillips, 1996). Many practitioners believe that as you proceed through each of the levels, the evaluation becomes more difficult and requires more time. The first level of criteria is Reaction, which measures whether learners find the training engaging, favourable, and relevant to their jobs. Level 2, Learning, gauges the learning of each participant based on whether learners acquire the intended knowledge, skills, attitude, confidence and commitment to the training. One of the most crucial steps in the Kirkpatrick Model, Level 3, Behaviour, measures whether participants were truly impacted by the learning and if they’re applying what they learn. The final level, Level 4, Results, is dedicated to measuring direct results (Kauffman, 1996). Level Four measures the learning against an organization's business outcomes— the Key Performance Indicators that were established before learning was initiated.
3. Canvas Strategy Data Analysis
Through the District Transformation Program, the role of the DEO is to provide differentiated
Sekolah Harian, 2017). The Kanvas Strategi Analisis Data (Razak and Nor, 2018) or Data Strategy Analysis Canvas has been adopted as a framework to support the consolidation of this new role. As shown in Figure 1, the canvas consists of eight steps, namely Vision and Direction, KOLB Reflection, PKPPD and SKPMg2, Data Analysis and Gap Analysis, Plan Intervention using SPeCT, Plan Action, Supervise OPPM, Assess Outcome and, Follow-Up, Improvement and Sharing. Standard Kualiti Pendidikan Malaysia Gelombang 2 (SKPMg2) is an assessment instrument used to continuously assess the quality of education towards high levels of achievement (Jemaah Nazir dan Jaminan Kualiti, 2017). It is a quality management tool to be used in schools. Penarafan Kecemerlangan Pejabat Pendidikan Daerah (PKPPD) is one of the mechanisms for measuring the performance of a culture of excellence in the District Education Office (Bahagian Pengurusan Sekolah Harian, 2017)
Figure 1: Data Analysis Strategy Canvas (Razak and Nor, 2018)
In the KOLB Reflection, PKPPD and SKPMg2 step, the experiential learning cycle, which was adopted from Kolb (2015) was applied. In the first cycle, existing experience is reinterpreted, follows by the second cycle, which is to reflect on the experience on a personal basis. The third cycle is where new ideas are formed and finally in the final fourth cycle new ideas are applied. The Mersing District Education Office experiential learning cycle is shown in Figure 2.
Vision and Direction
Figure 2: Mersing District Education Office Experiential Learning Cycle
From the experiential learning cycle, Continuing Professional Development (CPD) and guidance was identified as necessary to assure a high level of expertise and enables DEO officers, school leaderships and teachers to keep their professional skills and knowledge up-to- date. According to FutureLearn (2021), CPD can help avoid the skills gap which is as important for employers as it is for the organization. By engaging in professional development, the required skill sets can be quickly and easily built as required by the organisation. In the Data Analysis and Gap Analysis step of the canvas, the gap in knowledge and skills for effective differentiated guidance of officers in the Mersing DEO were identified, and the effectiveness of differentiated guidance given by officers in the Mersing DEO were analysed. According to the job scope for District Education Officer under the District Transformation Program initiative, 50% of his time is spent on giving differentiated guidance to schools in need and the management of professional development. A Mersing DEO Guidance Competency Profile was developed to determine gaps in the knowledge and skills of officers in the Mersing DEO in performing differentiated guidance to schools. Each officer was required to answer a simple five-item online survey with scores ranging from 1, Strongly Disagree to 5, Strongly Agree.
From the data collected, a Mersing DEO Guidance Competency Profile was developed as shown in Figure 3.
Figure 3: Mersing DEO Guidance Competency Profile
Overall the scores for Understand PKPPD, Perform Task Using PKPPD, Understand SKPMg2, Understand Guiding Using SKPMg2, and Perform Guidance Using SKPMg2 range from 3.7 to 4.1. There were gaps in knowledge, skills, and abilities that hindered efficient differentiated guidance by the Mersing DEO officers. Through a structured CPD, the gaps in knowledge, skills and abilities among officers in the Mersing DEO can be reduced or closed.
The guidance data for 2019 were analysed to determine the quality and quantity of guidance to school, done by officers in the Mersing DEO. As shown in Table 1, 653 guidance was given in 2019 covering 10 different areas. On average, each officer in the Mersing DEO has performed only 18.6 guidance for the year 2019. The amount of guidance given was not sufficient and the areas covered were not focussed on the needs of schools.
In the Plan Intervention using SPeCT and Plan Action steps, issues found from Data Analysis and Gap Analysis step were analysed to determine the appropriate interventions to be taken. These interventions must also take into consideration challenges faced due to the Covid- 19 pandemic, which continues to impact all areas of society, including education. Two interventions were selected to tackle issues faced by the Mersing DEO, namely Enhancement Workshop (PKPPD and Guidance Based on SKPMg2) and School Differentiated Guidance Using SKPMg2. Under the Enhancement Workshop (PKPPD and Guidance Based on SKPMg2), four lecture sessions and two workshops were conducted within two months.
Officers were equipped with the knowledge and skills to implement differentiated guidance based on SKPMg2 and were monitored while performing their tasks. The ADDIE model was used to ensure more effective delivery of differentiated guidance. It is an instructional systems design framework consisting of analysis, design, development, implementation, and evaluation
1:Strongly Disagree, 2:Disagree, 3:Neutral, 4:Agree, 5:Strongly Agree
to develop a structured differentiated guidance (Culatta, 2018). They were required to report their differentiated guidance using iBimbing, an online guidance reporting system. Data from the system were analysed to determine the effectiveness of differentiated guidance given and to determine which officers needed support in performing their tasks. Sharing sessions were conducted to share differentiated guidance good practices. A Buddy System was introduced where inexperienced officers were paired with experienced officers to improve their differentiated guiding skills. One Page Project Manager (OPPM) was used to monitor the implementation of these two interventions by a set schedule. Meetings were held every two weeks to supervise the OPPM and officers in charge were required to report the status of each intervention and further actions needed.
Table 1: 2019 Mersing District Education Office Guidance Data
No Guidance Areas Quantity
1 Curriculum Management 389
2 School Management 146
3 Administration 40
4 Islamic Studies Management 23 5 Human Development Management 19
6 Information Management 13
7 Assessment Management 9
8 Psychological Service 8
9 Supply And Development 4
10 Financial Management 2
Total 653
In the Assess Outcome step, the Kirkpatrick Evaluation Model was used to assess the effectiveness of the CPD given. All lectures and workshops were evaluated after each session using an online program evaluation survey. Four months after the two interventions started, officers in the Mersing DEO were required to answer the Mersing DEO Guidance Competency Profile consisting of a simple online five-item survey with scores ranging from 1, Strongly Disagree to 5, Strongly Agree. The effectiveness of differentiated guidance given by the Mersing DEO officers was analysed using data from iBimbing and data collected from an online survey sent to schools. SPM and STPM 2020 results were analysed and compared to 2019 results to determine the Result Level of the Kirkpatrick Evaluation Model.
4. Results
Figure 4 shows the Reaction Level in the Kirkpatrick Evaluation Model, which measures whether learners find the training engaging, favourable, and relevant to their jobs. The overall score for the Enhancement Workshop (PKPPD and Guidance Based on SKPMg2) is 4.58 (Agree) with scores for each item regarding Content Appropriateness, Clarity of Delivery, Timeline Suitability, and Speaker Suitability at 4 and above (Agree to Strongly Agree).
Officers from the Mersing DEO agreed that the four lecture sessions and two workshops conducted were beneficial to them and helped carry out the task of guiding schools effectively.
Figure 4: Kirkpatrick Evaluation Model Level 1 - Program Effectiveness Analysis
Figure 5 shows the Learning Level in the Kirkpatrick Evaluation Model, which measures whether officers in the Mersing DEO had acquired knowledge and skills from the CPD given. All dimensions related to coaching competencies showed an improvement from before to after the CPD with the highest improvement in the aspects of Understanding Guiding Using SKPMg2 and Perform Guidance Using SKPMg2. From the profile, officers in the Mersing DEO had benefitted from the CPD given and this is shown by the increase in knowledge and skills of the officers in performing their task of guiding schools.
Figure 5: Kirpatrick Evaluation Model Level 2 – Mersing DEO Guidance Competency Profile
Table 2 shows the Behaviour Level in the Kirkpatrick Evaluation Model, which measures whether officers in the Mersing DEO were truly impacted by the CPD and had they applied what they had learned from the training. The data shows an increase in officer guidance to schools for 2020, which is 1429 compared to 653 in 2019. For 2020, officer guidance is more focused on SKPMg2 which is a quality school management tool compared to 2019, which is more diverse. The increase in knowledge and skills among the officers has enabled them to perform differentiated guidance with improvement in terms of quantity and quality.
Table 2: 2020 Mersing District Education Office Guidance Data
No Guidance Areas Quantity
1 Standard 1 SKPMg2 463
2 Standard 2 SKPMg2 394
3 Standard 3 SKPMg2 519
4 Standard 4 SKPMg2 53
TOTAL 1429
Figure 6 shows the effectiveness of guidance given by the Mersing DEO officers to schools. Findings show that 78.72% of schools Strongly Agree that Mersing DEO officers Have Perform Guidance Using SKPMg2 and 76.60% Strongly Agree that Guidance Using SKPMg2 helpful. There are still 6.38% who are Uncertain that Mersing DEO Officers Have the Knowledge and Skills to Guide. However, 57.45% of schools are uncertain regarding the Adequate Guidance Frequency.
1:Strongly Disagree, 2:Disagree, 3:Neutral, 4:Agree, 5:Strongly Agree
Figure 6: Kirkpatrick Evaluation Model Level 3 – Mersing DEO Guidance Effectiveness
At the fourth level or Result Level of the Kirpatrick Evaluation Model, the outcome of the 2020 SPM and STPM results were analysed. There is a commendable increase in SPM 2020 results for Mersing district. The SPM pass percentage increased by 1.93%
from 89.95% to 91.88%. Mersing district had also maintained its position as the best district in the state of Johore regarding SPM pass percentage. There was an increase in the District Grade Average of 0.29 from 4.98 to 4.69 (Figure 7). This increase is the highest in the state of Johore. The Subject Grade Point Average for 25 subjects also showed an improvement in the SPM 2020 examination. In the STPM examination, the Cumulative Grade Point Average increased 0.2 from 2.37 to 2.57 (Figure 8). This increase is also the highest in the state of Johore.
Figure 7: Mersing District SPM Result (2015-2020)
Figure 8: Mersing District STPM Result (2015-2020) 5. Discussion and Conclusion
Mersing DEO has implemented Data Strategy Analysis Data as a systematic and planned effort to improve organizational achievement. The two interventions selected to tackle issues faced by the Mersing DEO, Enhancement Workshop (PKPPD and Guidance Based on SKPMg2) and School Differentiated Guidance Using SKPMg2 have yielded the expected results. These two interventions have been effective in improving the knowledge and skills of officers in the
on Kirkpatrick Evaluation Model showed the effectiveness of structured, practical and methodical CPD to improve the quality of differentiated guidance provided to schools, particularly in light of the challenges of the Covid-19 pandemic. Officers in the Mersing DEO reacted positively to the four lecture sessions and two workshops conducted under the Enhancement Workshop (PKPPD and Guidance Based on SKPMg2). The Overall Rating scores for all six sessions ranged from 4.40 to 4.76, which means they enjoyed their experience and found the material in the program useful for their work. From the Mersing DEO Guidance Competency Profile, we can conclude that learning has taken place among the officers. Evaluation done prior to the sessions and following the sessions has shown an increase in Understand PKPPD, Perform Task Using PKPPD, Understand SKPMg2, Understand Guiding Using SKPMg2, and Perform Guidance Using SKPMg2.
The gain in knowledge and skills from the training together with the School Differentiated Guidance Using SKPMg2 intervention had a postitive impact on how the officers performed their differentiated guidance to schools. Data from iBimbing has shown an increase in guidance given to schools and the officers were more focused on the areas of guidance, where they had focused on SKPMg2, which is a quality tool in schools. The guidance given has shown effectiveness as reported by the school. 100% of schools agreed that the officers had Perform Guidance Using SKPMg2, and Guidance Using SKPMg2 Helpful. This shows that Mersing DEO officers have been able to perform their duties effectively in providing differentiated guidance to schools.
The increase in Mersing DEO officers professional skills had ultimately improved students' academic performance and success in the 2020 SPM and STPM examinations despite the challenges of the Covid-19 pandemic. Mersing district has managed to show the best improvement in the state of Johore for both SPM and STPM examinations. The effectiveness of CPD in Mersing District has shown a very positive impact, especially in the achievement of students in SPM and STPM. These interventions will be continued in the coming year in order to enable all officers to provide effective differentiated guidance and trigger greater excellence.
References
Bahagian Pengurusan Sekolah Harian (2017). Panduan Pengurusan Program Transformasi Daerah. Available online: https://goo.gl/AcGUAT
Culatta, R. (2018, April 2). ADDIE Model. Retrieved September 6, 2021, from http://www.instructionaldesign.org/models/addie/
FutureLearn (2021), https://www.futurelearn.com/info/blog/importance-of-cpd-continuing- professional-development, Assessesd: 6/9/2021
J.A. Ingwu, J. Efekalam, A. Nwaneri, B. Ohaeri, C. Israel, P. Chikeme, ..., N. Omotola.
(2019). Perception towards mandatory continuing professional development programme among nurses working at University of Nigeria Teaching Hospital, Enugu-Nigeria.
International Journal of Africa Nursing Sciences, 11 (2019), Article 100169.
Jemaah Nazir dan Jaminan Kualiti (2017. Standard Kualiti Pendidikan Malaysia Gelombang
2. Available online:
https://www.moe.gov.my/en/pemberitahuan/announcement/skpmg2-2018
Kaufman, R. (1996). Strategic Thinking: A Guide to Identifying and Solving Problems.
Arlington, VA. & Washington, D.C. Jointly published by the American Society for Training & Development and the International Society for Performance Improvement K. Giri, N. Frankel, K. Tulenko, A. Puckett, R. Bailey, H. Ross. (2012). Keeping up to date:
Continuing professional development for health workers in developing countries.
Retrieved from, Intra Health International (2012)
https://www.dphu.org/uploads/attachements/books/books_997_0.pdf. (Accessed 6/9/2021)
Kirkpatrick, J. D. (2016), Kirkpatrick’s Four Level of Trainung Evaluation published by American Society for Training & Development. Available online : https://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/
(Accessed 6/9/2021)
Ministry of Education. (2013). Malaysia Education Blueprint 2013-2025. Available online:
https://www.moe.gov.my/en/dasarmenu/pelan-pembangunan-pendidikan-2013-2025.
(Assessed 6/9/2021)
Phillips, J. (1996). How much is the training worth? Training and Development, 50(4), 20-24 Razak and Nor. (2018). Kanvas Strategi Analisis Data. Institut Aminuddin Baki, Enstek, NS.
