KPIs addressed
Standard 11: Standard 11: Institutional Relationships with the
3. Name of the external benchmark provider
EE Department PSAU, through DQAA
5
Standard 2 - Program Administration
KPI: Stakeholder evaluation of the Policy Handbook, including administrative flow chart and job responsibilities (Average rating on the adequacy of the Policy Handbook on a five- point scale in an annual survey of teaching staff and final year students)
NCAAA KPI Reference Number: S2.1 Institutional KPI Reference Number: S2.1
Actual Benchmark Target
Benchmark Internal
Benchmark* External Benchmark**
[PSAU]
New Target Benchmark
4.65 4 3.95 3.46 4.7
Analysis:
The actual benchmark value for 2018-2019 is 4.65 and is based on the survey “EE Teaching Staff Survey on Program Mission, Vision and Objectives” conducted among EE staff for the year 2018-2019. In particular, rating (on a five-point Likert scale) of questions relevant to this KPI in the aspect: “Regulations and Laws (policies and procedures manual) at the university” were averaged. We note that the actual benchmark is considerably higher than the actual value for the previous year (2017-2018) which also serves as the internal benchmark (3.95). This indicates the institute’s success in improving the available library and media resources. The actual benchmark also comfortably exceeds the target benchmark (4) and the external benchmark provided by Prince Sattam bin Abdulaziz University [PSAU] (3.46). The considerable improvement compared to the previous year is partly due to steps taken to ensure that faculty members are familiar with, and have ready access to, the various policies and organizational units at the department. Keeping in view the actual and internal benchmarks, and the ambitions of the program, the quality committee has decided to set the new target benchmark to 4.7.
Strengths
6 - Relevant surveys are conducted regularly among faculty which allows for consistent evaluation
of policy and administration related aspects.
- The department makes efforts to familiarize stakeholders with the various policies and organizational units at the department, college, and institute levels.
Areas Requiring Improvement
There were no surveys readily available for the final-year students’ evaluation of the policy and administration related aspects. General surveys among all the students at the department are available that cover this aspect, but it is not currently possible or convenient to extract responses by final-year students only.
Priorities for action
- External benchmark is recommended to compare actual benchmark to similar programs given internationally.
- Ensuring that a separate survey covering this aspect is conducted among the final-year students in the future.
* Explain:
1. Why this internal benchmark provider was chosen?
Using EE program’s own actual benchmark from the previous cycle (2017-2018) provides a good internal measure for program improvement from one cycle to the next. The Quality Unit of CoE in collaboration with DQAA conducts and collects regular evaluation surveys including among EE students. The results available in their database were considered an authentic and suitable measure for an internal benchmark.
2. How was the benchmark calculated?
The internal benchmark is based on survey conducted among EE staff in the year 2017-2018, and is an average of the ratings of the questions relevant to this KPI.
3. Name of the internal benchmark provider.
Department of Electrical Engineering, JU, through College Quality Unit.
** Explain:
1. Why this external benchmark provider was chosen?
The Electrical Engineering program at PSAU is similar to the program offered by Jouf University and serves a similar demographic. The two universities also have a mutual KPI sharing agreement.
2. How was the benchmark calculated?
The benchmark is calculated by PSAU and is based on surveys relating to stakeholders’ awareness about PSAU’s EE program mission and objectives.
3. Name of the external benchmark provider.
EE Department PSAU, through DQAA
7
Standard 3 - Management of Program Quality Assurance
KPI: Students' overall evaluation on the quality of their learning experiences.
(Average rating of the overall quality on a five-point scale in an annual survey of final year students.)
NCAAA KPI Reference Number: S3.1 Institutional KPI Reference Number: S3.1
Actual Benchmark Target
Benchmark Internal
Benchmark* External Benchmark**
[PSAU]
New Target Benchmark
4.3 3.8 3.15 4.5 4.4
Analysis:
This KPI by NCAAA has been developed to capture the opinion of the final year students about the quality of learning experience provided by the program. The actual benchmark (4.3) is based on the survey “EE Students Survey on Evaluating the Electrical Engineering Program” conducted among final year students for the year 2018-2019. The aspect included in the calculation of the actual benchmark is “Overall Evaluation”. We note that the actual benchmark value for year 2018-2019 is considerably higher than the actual value for the previous year (2017-2018) which also serves as the internal benchmark (3.15).
The actual benchmark exceeds the target (3.8) comfortably and is reasonably close to the external benchmark provided by Prince Sattam bin Abdulaziz University [PSAU] (4.5). The considerable improvement compared to the previous year shows the EE department’s commitment to improving the quality of learning experience. Keeping in view the actual, internal, and external benchmarks, and the ambitions of the program, the quality committee has decided to set the new target benchmark to 4.4.
8 Strengths
- Relevant surveys are conducted regularly which allows for consistent evaluation of the overall learning experience from final year students’ perspective
- The department attempts to improve all aspects of the students’ experience
Areas Requiring Improvement
An appropriate strategy needs to be developed through focused group discussions with all the relevant stakeholders to address the shortcoming and improve the quality of all the services.
Priorities for action
- An international external benchmark is recommended to compare actual benchmarks to similar programs at international level.
* Explain:
1. Why this internal benchmark provider was chosen?
Using EE program’s own actual benchmark from the previous cycle (2017-2018) provides a good internal measure for program improvement from one cycle to the next. The Quality Unit of CoE in collaboration with DQAA conducts and collects regular evaluation surveys including among EE students. The results available in their database were considered an authentic and suitable measure for an internal benchmark.
2. How was the benchmark calculated?
The internal benchmark is based on the survey conducted among final year EE students for the year 2017-2018.
3. Name of the internal benchmark provider.
Department of Electrical Engineering, JU, through College Quality Unit.
** Explain:
1. Why this external benchmark provider was chosen?
The Electrical Engineering programs at PSAU is similar to the program offered by Jouf University and serve a similar demographic. PSAU and Jouf University also have a mutual KPI sharing agreement.
2. How was the benchmark calculated?
The benchmark is calculated by PSAU and is based on student surveys regarding quality of learning experience at EE Department
3. Name of the external benchmark provider.
EE Department PSAU, through DQAA
9 KPI: Proportion of courses in which
student evaluations were conducted during the year.
NCAAA KPI Reference Number: S3.2 Institutional KPI Reference Number: S3.2 Actual Benchmark Target
Benchmark Internal
Benchmark* External Benchmark**
[PSAU]
New Target Benchmark
100% 100% 100% 94% 100%
Analysis:
Clear policy exists at the institution level to guide the process of student evaluation surveys. Surveys are performed every semester involving all the courses included in that semester. 100% of courses were covered in the course evaluation surveys both during 2018-2019. This can be explained, in part, by the practice of the college/departments to actively encourage students to participant in these surveys. In addition, the web-based survey application developed by DQAA also makes the process smooth and the surveys more accessible. The actual value matches internal benchmark and is higher than the external benchmark (94%) Prince Sattam bin Abdulaziz University [PSAU]. Considering the importance of course evaluations in quality management and assurance, and in line with the program’s commitment to incorporating feedback in its operations, the new target benchmark is also set to the maximum (100%).
Thus, the aim is to continue conducting student evaluations for all courses taught at the EE Department.
Strengths
- Surveys are performed every semester involving all the courses included in that semester. The college/departments actively encourage students to participant in these surveys.
- To facilitate smooth conduct of these surveys, DQAA has developed a web-based survey application to reduce data entry errors from paper- based surveys.
- The strengths and weaknesses of the courses are communicated with department chairman and course directors by the Dean. Action plans are included in course reports to address weaknesses and enhance strengths
10 Areas Requiring Improvement
Priorities for action
Making sure that the online surveys are secure and obtained data safely stored.
* Explain:
1. Why this internal benchmark provider was chosen?
Using EE program’s own actual benchmark from the previous cycle (2017-2018) provides a good internal measure for program improvement from one cycle to the next. The Quality Unit of CoE in collaboration with DQAA conducts and collects regular evaluation surveys including among EE students. The results available in their database were considered an authentic and suitable measure for an internal benchmark.
2. How was the benchmark calculated?
The internal benchmark is based on the proportion of course given in 2017-2018 at EE in which an evaluation survey was conducted.
3. Name of the internal benchmark provider.
Department of Electrical Engineering, JU, through College Quality Unit.
** Explain:
1. Why this external benchmark provider was chosen?
The Electrical Engineering program at PSAU is similar to the program offered by Jouf University and serve a similar demographic. PSAU and Jouf University also have a mutual KPI sharing agreement.
2. How was the benchmark calculated?
The benchmark is calculated by PSAU and is based on the proportion of course in the EE program in which an evaluation survey was conducted.
3. Name of the external benchmark provider.
EE Department PSAU, through DQAA
11 KPI: Proportion of programs in which
there was an independent verification, within the institution, of standards of student achievement during the year.
NCAAA KPI Reference Number: S3.3 Institutional KPI Reference Number: S3.3
Actual Benchmark Target
Benchmark Internal
Benchmark* External Benchmark**
[PSAU]
New Target Benchmark
100% 100% 100% 0% 100%
Analysis:
Clear policy exists for independent verification (within the institution) of the standards of student achievements during the year. As such, performance in all the courses is evaluated by JU faculty external to the EE department, leading to an actual benchmark value of 100%. The actual value matches the target (100%) and internal benchmarks (100%), and exceeds the external benchmark provided by Prince Sattam bin Abdulaziz University [PSAU] (0%). Considering the importance of such independent verifications in quality management and assurance, and in line with the program’s commitment to incorporating feedback in its operations, the new target benchmark is also set to the maximum (100%).
Thus, the aim is to continue obtaining independent verification of standards of student achievement at the department.
Strengths
- Performance in all the courses is evaluated by JU faculty external to the EE department
- The strengths and weaknesses of the program are communicated with department chairman and course directors by the Dean.
-
Areas Requiring Improvement Priorities for action
12 - International external benchmark is recommended to compare actual benchmark to similar
programs given internationally.
* Explain:
1. Why this internal benchmark provider was chosen?
Using EE program’s own actual benchmark from the previous cycle (2017-2018) provides a good internal measure for program improvement from one cycle to the next.
2. How was the benchmark calculated?
The internal benchmark is based on the data for 2017-2018.
3. Name of the internal benchmark provider.
Department of Electrical Engineering, JU, through College Quality Unit.
** Explain:
1. Why this external benchmark provider was chosen?
The Electrical Engineering program at PSAU is similar to the program offered by Jouf University and serve a similar demographic. PSAU and Jouf University also have a mutual KPI sharing agreement.
2. How was the benchmark calculated?
The benchmark is calculated by PSAU and is based on the proportion of course in the EE program in which there was an independent verification of student achievement standards.
3. Name of the external benchmark provider.
EE Department PSAU, through DQAA
13
Standard 4 - Learning and Teaching
KPI: Ratio of students to teaching staff.
(Based on full time equivalents) NCAAA KPI Reference Number: S4.1 Institutional KPI Reference Number: S4.1 Actual Benchmark Target
Benchmark Internal
Benchmark* External Benchmark**
[PSAU]
New Target Benchmark
9:1 10:1 18:1 8:1 8:1
Analysis:
Actual value (9:1) of this KPI is calculated by dividing the number of EE students by the number of full- time teaching staff at the department during the year 2018-2019. As a clear sign of improvement in this aspect, the actual KPI is significantly better than the actual value for the previous year (2017-2018) which also serves as the internal benchmark (18:1). This is primarily due to hiring of several new full-time faculty members (mostly at Assistant Professor level) during 2018. The actual benchmark exceeds the target (10:1) and is reasonably close to the external benchmark provided by Prince Sattam bin Abdulaziz University [PSAU] (8:1). Keeping in view the actual, internal, and external benchmarks, and the ambitions of the program, the quality committee has decided to set the new target benchmark to 8:1.
Strengths
- The EE program provides sufficient number of full-time teaching staff to the students.
Areas Requiring Improvement
- Most of the PhD teaching staff are Assistant Professors. There is a need to increase the proportion of Associate and Full Professors in the EE department.
Priorities for action
14 - An international external benchmark is recommended to compare actual benchmarks to similar
programs at international level.
* Explain:
1. Why this internal benchmark provider was chosen?
Using EE program’s own actual benchmark from the previous cycle (2017-2018) provides a good internal measure for program improvement from one cycle to the next.
2. How was the benchmark calculated?
The internal benchmark is calculated by dividing the number of EE students by the number of full-time teaching staff at the EE department during the year 2017-2018.
3. Name of the internal benchmark provider.
Department of Electrical Engineering, JU, through College Quality Unit.
** Explain:
1. Why this external benchmark provider was chosen?
The Electrical Engineering program at PSAU is similar to the program offered by Jouf University and serves a similar demographic. PSAU and Jouf University also have a mutual KPI sharing agreement.
2. How was the benchmark calculated?
The benchmark is calculated by PSAU and gives the ratio of students at the EE Department to full-time teaching staff at the department.
3. Name of the external benchmark provider.
EE Department PSAU, through DQAA
15 KPI: Students overall rating on the
quality of their courses. (Average rating of students on a five-point scale on overall evaluation of courses.)
NCAAA KPI Reference Number: S4.2 Institutional KPI Reference Number: S4.2
Actual Benchmark Target
Benchmark Internal
Benchmark* External Benchmark**
[PSAU]
New Target Benchmark
4.03 4.4 4.31 3.81 4.4
Analysis:
This KPI by NCAAA has been developed to capture the opinion of the students about the quality of the courses given by the program. The actual benchmark (4.03) is based on several course evaluation surveys conducted among the EE students for course given in the year 2018-2019. We note that although the actual benchmark value is considerably higher than the external benchmark provided by Prince Sattam bin Abdulaziz University [PSAU] (3.81) it fails to meet the target (4.4) and is also lower than the actual benchmark for the previous cycle (4.31) which also serves as the internal benchmark. Keeping in view the actual, internal, and external benchmarks, and the ambitions of the program, the quality committee has decided to retain the target benchmark at 4.4.
Strengths
- Course evaluations are conducted regularly which allows for consistent evaluation of the course quality from EE students’ perspective
16 Areas Requiring Improvement
The aspect relating to the overall utility of the course got relatively lower scores in the surveys, indicating a need for emphasizing the practical applications and need of the topics taught.
Priorities for action
- An international external benchmark is recommended to compare actual benchmarks to similar programs at international level.
* Explain:
1. Why this internal benchmark provider was chosen?
Using EE program’s own actual benchmark from the previous cycle (2017-2018) provides a good internal measure for program improvement from one cycle to the next. The Quality Unit of CoE in collaboration with DQAA conducts and collects regular evaluation surveys including among EE students. The results available in their database were considered an authentic and suitable measure for an internal benchmark.
2. How was the benchmark calculated?
The internal benchmark is based on the survey conducted among EE students for courses given in the year 2017-2018.
3. Name of the internal benchmark provider.
Department of Electrical Engineering, JU, through College Quality Unit.
** Explain:
1. Why this external benchmark provider was chosen?
The Electrical Engineering programs at PSAU is similar to the program offered by Jouf University and serve a similar demographic. PSAU and Jouf University also have a mutual KPI sharing agreement.
2. How was the benchmark calculated?
The benchmark is calculated by PSAU and is based on student surveys regarding quality of courses given at EE Department
3. Name of the external benchmark provider.
EE Department PSAU, through DQAA
17 KPI: Proportion of teaching staff with
verified doctoral qualifications. NCAAA KPI Reference Number: S4.3 Institutional KPI Reference Number: S4.3 Actual Benchmark Target
Benchmark Internal
Benchmark* External Benchmark**
[PSAU]
New Target Benchmark
65% 70% 50% 100% 70%
Analysis:
Actual value (65%) of this KPI is ratio of EE teaching staff with verified doctoral degrees to the total teaching staff with doctoral degrees at the EE Department during 2018-2019. As a clear sign of improvement in this aspect, the actual KPI is higher than the actual value for the previous year (2017- 2018) which also serves as the internal benchmark (50%). This is primarily due to hiring of several new full-time faculty members (mostly at Assistant Professor level and all with verified doctoral qualifications) during 2018-2019. The actual benchmark is still lower the target (70%), and the external benchmark provided by Prince Sattam bin Abdulaziz University [PSAU] (100%). Keeping in view the actual, internal, and external benchmarks, and the ambitions of the program, the quality committee has decided to retain the target benchmark at 70%.
Strengths
- Procedures are in place to ensure that new hired PhD faculty has verified degrees. This should result in a gradual improvement of the KPI.
Areas Requiring Improvement
- Procedures for verification of PhD degrees should be followed strictly.
Priorities for action
18 - An international external benchmark is recommended to compare actual benchmarks to similar
programs at international level.
* Explain:
1. Why this internal benchmark provider was chosen?
Using EE program’s own actual benchmark from the previous cycle (2017-2018) provides a good internal measure for program improvement from one cycle to the next.
2. How was the benchmark calculated?
The internal benchmark is the ratio of EE teaching staff with verified doctoral degrees to the total teaching staff with doctoral degrees at the EE Department during the year 2017-2017.
3. Name of the internal benchmark provider.
Department of Electrical Engineering, JU, through College Quality Unit.
** Explain:
1. Why this external benchmark provider was chosen?
The Electrical Engineering program at PSAU is similar to the program offered by Jouf University and serve a similar demographic. PSAU and Jouf University also have a mutual KPI sharing agreement.
2. How was the benchmark calculated?
The benchmark is calculated by PSAU and gives the proportion of teaching staff at the EE department with verified doctoral degrees.
3. Name of the external benchmark provider.
EE Department PSAU, through DQAA