Ⅰ. Background and Purpose
◦ Technological changes resulting from the Fourth Industrial Revolution have changed jobs, and employers seek employees with creative and comprehensive problem-solving skills, or competencies, which allow flexible response to these changes. With the advent of automation and robotics, the production sites are shifting manual labor, production, and identical repetitive verification process to machines. In contrast, human labor is placed for tasks that require an understanding of comprehensive operational processes and problem-solving skills.
◦ Therefore, this research aims to examine whether the national technical qualification testing system is applicably operated from the perspective of competency-based evaluation and derive various policy implications for improving identified shortfalls.
New Strategies for Qualification Test in Response to the Accelerating Technological Change
Dong-Im Lee, Senior Research Fellow Tel: 044-415-5349 / e-mail: [email protected]
Summary
From the perspective of competency-based evaluation in response to accelerating technological change, this research explores the future trends in qualification testing based on an analysis of industry-side demand and provides a new strategy on how to accurately verify individual’s sufficient competencies required on the job through qualification testing.
▸Key Words: Technological Change, Competency Evaluation, Qualification Testing, Qualification System
Ⅱ. Main Contents and Results
□ Changes of Qualification Testing (External) Environment from Technological Changes
◦ The research reviewed the changes of jobs and demanded competencies resulting from technological change, then analyzed recent trends in competency-based skills evaluations (in areas such as school evaluation and employment evaluation). Attention must be given to the fact that examinations based on multiple-choice questions, short answer questions, and strictly defined answers cannot evaluate the aptitude needed for the future workforce. Namely, digital transformation brought the need to measure competency in comprehending the overall work processes, such as network thinking thought processing, adaptability to new environments, and problem-solving capacity.
◦ Therefore, this research analyzed the competency-based evaluation trends in both Korean and German professional national qualifications focusing on health and medicine field, and aviation professional field qualification. First, ‘competency’ is defined as the comprehensive skill to address and manage various situations that may ordinarily occur in a job. Second, competency-based evaluation must be administered in a setting most similar to the work environment to maximize realistic evaluation and computer and IT-based testing could be highly effective in enhancing the evaluation efficiency.
□ Current Status Analysis of Korea’s Qualification Testing from the Perspective of Competency-Based Evaluation
◦ The research analyzed the importance of competency-based evaluation with industry survey.
◦ The result of the survey on Korea’s accredited private qualifications and national professional qualifications from a competency evaluation perspective showed that there is a high concentration of questions testing knowledge-based simple memorization capacity, rather than testing competencies.
◦ To analyze the state of the national technical qualification testing from competency evaluation perspective, a case study was conducted on motor vehicle maintenance qualifications. The case study found that technological changes in motor vehicles are significantly altering the motor vehicle maintenance working environment; however, qualification testing (contents, facilities and equipment, etc.) has not reflected these
development.
□ Diagnosis of Problems in National Technical Qualification System from the Point of Competency-Based Evaluation
◦ Lastly, the research diagnosed the suitability of the national technical qualification system (qualification standards, passing standards, questionnaire standards, evaluation infrastructure, etc.). The shortcomings and improvement recommendations from the diagnostic results are as follows.
<Shortcomings and Improvement Recommendations
from the Diagnostic Results of the National Technical Qualification System>
Diagnosis
Criteria Shortcomings Identified from Diagnostic Results Improvement Recommendations
Question Type
- Predominantly, the written examinations consist of memorization questions, practical examinations consist of one-dimensional questions assessing repair, diagnosis, and craftsmanship skills.
- Lack of examination questions that evaluate comprehensive problem-solving skills based on cases
- The need to change from the written, performance, and essay test format to case types, problem-solving, and scenarios format questions
- This research developed and presents sample questions in suggested format
Testing Methodo logy
- Lack of diversity in testing methodology from the perspective of competency-based evaluation and lack of technology-based evaluation methodology for effective competency evaluation
- Lack of implementing various improvements to testing methodology planned throughout four-times
‘Basic Plans on Improvement of National Technical Qualification System’.
- Written examinations need to adopt CBT evaluation and utilize various medium, such as audio, graphics, and video, to allow evaluation to take place in a setting similar to the work environment.
- Essay format examinations need to prioritize adopting scenario-based PC simulation testing.
Testing Standards
- Complex competency-based evaluation,
incorporating consolidation, evaluation, and analysis, is possible for professional engineer testing. There are, however, limitations on competency-based evaluation for lower-levels.
- The scope of qualification is too small to evaluat comprehensive problem-solving skills.
- To include the context and level of competency, testing standards need to be reformed based on Korea Qualification Framework(KQF).
- The need to align the qualification scope with the occupational unit for employment according to a skill map, in order to evaluate comprehensive problem-solving skills
Question Contents
- Current question standards centered on subject areas makes it difficult to develop field-based case type questions.
- Reform to consolidate subject areas and question standards based on the occupational task is needed for competency evaluation.
- Competency target guideline needs to be developed to create case centered
Ⅲ. Conclusion and Policy Suggestion
□ Tasks for Innovating Qualification Testing (Competency Evaluation Infrastructure Establishment)
◦ (Task 1) Strengthen research on competency-based evaluation
- Need to identify relevant vocational competency and define what needs to be measured through research
◦ (Task 2) Establish a competency identification system (Qualification Demand Change Management System)
- Need to establish a system that can regularly identify the competencies demanded by industries
◦ (Task 3) Design qualification categories where competency-based evaluation is feasible: prepare skills map
- Because the scope of full qualification is important to evaluate competency, skills map need to be prepared in advance
◦ (Task 4) Prepare competency target guideline for competency-based evaluation - For example, in the case of motor vehicle maintenance, description of required tasks
for solving problems cause by main car issues
◦ (Task 5) Implement technology-based testing methodology for effective competency-based evaluation
- Prioritize the development of Expert System to introduce PC-based simulation evaluation. Then AR/VR-based qualification evaluation can be made possible by adding
problem-solving questions.
Passing Standards
- The meaning of competency shifts throughout ages.
Because test scores cannot guarantee adequate job performance, it is inappropriate to have a fixed passing score of 60 points from the perspective of competency-based evaluation.
- Need to consider PASS or FAIL standard for competency-based evaluation (PASS if target achievement is reached, else FAIL)
Testing Infrastr
ucture
- Challenges for competency-based evaluation are identified in examiners, practical examination testing infrastructure, and question bank management.
- Need for securing expertise of examiners, standardizing practical examination sites, and introducing systematic question bank management system
one additional device (3D). Moreover, there is the need to track the qualification obtainers, analyze their labor market performance, create a database, and attempt AI evaluation through deep learning
◦ (Task 6) Prompt active industry participation in qualification testing and strengthen the competency of testing institutions (personnel)
- To test comprehensive problem-solving skills, there is the need for working-level professionals to develop examination questions or collaborate with examiner researchers. Furthermore, collaborations with competency development and measurement experts are needed to develop questions and evaluate logical reasoning and creativity, and secure the expertise and independence of the examiner/evaluator.
- The need to secure plenty of the latest case examination questions, or problem-solving type questions in the question bank
- Practical challenges, such as facilities, infrastructure, and particularly, funds for competency-based evaluation need to be reviewed.
◦ (Task 7) Introduce a pilot program for competency-based and new technology-based evaluations
- There is a need for this program to focus on a competency model, CBT-based competency measuring methodology, and development of digital measurement instruments.
∙ Related Ministry: Ministry of Employment and Labor
