KRIVET Issue Brief

(1)

KRIVET Issue Brief People are Our Hope

Training Performance Comparison by Work–Learning Dual-Program On-the- Job Training (OJT) Method

- The apprentices who received practice-based OJT found that their job skills improved through the OJT and that they were able to use much of what they had learned through the work–learning dual program.

- Compared to those who received the three other OJT methods, the apprentices who received autonomous OJT perceived the following negatively: adequacy of operation methods, job performance improvement through OJT, level of usage of what they had learned through the work–learning dual program, effects of the work–learning dual program on their organizational and career immersion, and level of satisfaction with the in-company trainers.

- Most apprentices are in the early stages of their careers. Therefore, the apprenticeship is likely to become more effective if the in-company trainers clearly present the goals and contents of the work–learning dual program to the apprentices through lectures, practice, or homework and increase their interaction with the apprentices than for them to leave the apprentices to their own devices.

01 Need for Analysis and Analysis Data

| Need to seek ways to enhance the effectiveness of the work–learning dual program through training performance analysis and comparison of the work–learning dual-program OJT operation methods

The 118th KRIVET Issue Brief verified that learning transfer, job performance improvement, job satisfaction, organizational immersion, and career immersion differ by work–learning dual-program OJT method. Moreover, it stated the need to activate practice-based OJT rather than lecture-based OJT.

The data obtained from the work–learning dual-program status surveys from 2016 to 2019 (based on the answers of the apprentices), however, showed no significant changes in the use of practice-based OJT, with 51.3% (2016)

→

51.5% (2017)

→

50.2% (2018)

→

53.4% (2019).

This brief utilizes the data from the 2019 work–learning dual-program status survey questionnaires filled out by the apprentices to compare their training performances and to explore the useful implications of such data for improving the effectiveness of the work–learning dual program.

| Data for analysis: Raw data from the Human Resources Development Service of Korea and KRIVET’s 2019 Survey of Apprentices on the Actual Condition of Each Work–Learning Dual- Program Participant

Subject of analysis: Of the 2,021 apprentices surveyed (899 in training, 1,122 graduates)

¹

, the responses of 16 apprentices who selected “other” for OJT operation method were excluded. A total of 2,005 survey questionnaires were analyzed

→

(1) 630 for lecture-based OJT (in-company trainers transferring knowledge and skills to apprentices through lectures); (2) 1,080 for practice-based OJT (in-company trainers and apprentices working together or performing similar tasks to transfer knowledge and skills); (3) 205 for homework-based OJT (in-company trainers assign homework to apprentices and check this periodically); and (4) 90 for autonomous OJT (apprentices learn on their own and in-company trainers check their learning results or achievements).

Method of analysis: Depending on the OJT method, the following were analyzed: (1) adequacy of the OJT method; (2) degree of the OJT method’s contribution to job performance improvement; (3) effects of the work–learning dual-program training process (contribution to job performance capacity and organizational and career immersion)

²

; (4) level of satisfaction with the in-company trainer; and (5) comparisons of OJT methods’ external evaluation results (cross-analysis, chi square [χ2] test, and one-way ANOVA [analysis of variance]).

I Footnote I

1) Due to lack of space, please refer to “Kim Sangho et al.

(2019), Survey on the actual condition of each participating subject of the work–learning dual program, Human Resources Development Service of Korea & KRIVET, consigned project 2019-15-5” for the method of collecting more detailed data that was used in the study.

2) The 2019 Work–Learning Dual-Program Status Survey used a 5-point Likert scale to measure ①–④, with “3” being “average,” which is likely to be interpreted as the respondents’ neutral or reserved answer. Therefore, this analysis considered “3” a missing answer and analyzed only survey questionnaires with “1,”“2,”“4,” and “5” scores (which were recoded as “1,”“2,”“3,” and “4,” respectively). Also, the level of satisfaction with the in- company trainer for ④ was measured through the sum and average of the scores for six items, including OJT preparation, job expertise, OJT attitude, and compliance with time.

Publisher: Young Sun Ra | Date of issue: April 13, 2020 | Issued by: Korea Research Institute for Vocational Education and Training (KRIVET)

2020

No.182

(2)

KRIVET Issue Brief

-

.임 임

02 Adequacy of OJT Method and Improvement of Job Performance through OJT

.

| Apprentices who mostly received OJT that required them to self-learn, with the in-company trainers checking their results on a regular basis (autonomous method) were more skeptic about the adequacy of the training method and its effect on subsequent improvement of their job performance than those who received a different type of OJT.

[Figure 1] Adequacy of OJT Method [Figure 2] Improvement of Job Performance Capacity through OJT

There were differences in response among the apprentices regarding OJT method adequacy.

- The percentages of positive responses (adequate, very adequate) were as follows: lecture-based method, 92.0%; practice-based method, 91.3%; homework-based method, 87.7%; and autonomous method, 65.5%.

There were differences in response among the apprentices regarding the improvement of their job performance capacity through OJT according to the OJT method that they received.

- The percentages of positive responses (helpful, very helpful) by OJT method were as follows: practice-based method, 93.2%;

lecture-based method, 88.3%; homework-based method, 94.9%; and autonomous method, 67.8%.

- This means that the apprentices who received practice-based OJT perceived their job performance capacity improvement most positively among all the groups.

\ 03 Utilization of What Was Learned through OJT and Enhancement of Job Performance Capacity through the Work–Learning Dual Program

| Apprentices who received autonomous OJT were more skeptical than those who received a different type of OJT about the usefulness of what they learned during OJT and the helpfulness of the work–learning dual-program in improving their job performance capacity.

[Figure 3] Utilization of OJT results [Figure 4] Improvement of Job Performance Capacity through the Work–Learning Dual Program

Significant differences in response were found among the apprentices regarding the extent to which they utilize what they had learned through the work–learning dual program according to the OJT they received.

⁵

- The percentages of positive responses (utilize, utilize a lot) by OJT method were as follows: practice-based method, 75.8%;

homework-based method, 71.3%; lecture-based method, 64.2%; and autonomous method, 42.3%.

Very inadequate Inadequate Adequate Very adequate

Not helpful at all Not very helpful Helpful Very helpful

Don’t utilize at all Don’t utilize Utilize at times Utilize a lot

Not helpful at all Not very helpful Helpful Very helpful

(lecture- based method)

(N=412)

(N=427) (practice-based

method) (N=794)

(practice-based method)

(N=834) (homework-

based method) (N=138)

(homework- based method)

(N=139) (autonomous

method) (N=58)

(autonomous method)

(N=59) 63.3%

59.2% 58.7%

41.4%

61.1%

57.9%

53.2%

49.2%

28.6% 32.1%

29.0%

24.1% 27.2%

35.3%

31.7%

18.6%

(N=338)

(lecture-based method)

method) (N=637)

(practice-based method])

(N=718) (homework-

(N=115) (autonomous

method) (N=52)

(autonomous method)

(N=56) 42.9%

49.8%

47.0%

28.8%

61.8%

66.2%

58.3%

53.6%

21.3%

26.1%

24.3%

13.5%

22.8% 23.4% 25.2%

16.1%

I Footnote I

3) ‘No difference in response tendency was shown between the lecture- and practice- based methods and between the lecture-, practice-, and homework-based methods (χ 2 = 2.463, df = 3, p = 0.482 and χ² = 6.606, df = 6, p = 0.359, respectively), but a significant difference was found between all the four methods (χ² = 48.986, df = 9, p _< 0.001).

4) Significant differences in response tendency were found between the four OJT methods (X² = 68.424, df = 9, p _< 0.001); between the lecture-, practice-, and homework-based methods;

and between the lecture- and practice-based methods (X²

= 20.396, df = 6, p = 0.002 and X² = 14.091, df = 3, p = 0.003, respectively).

I Footnote I

5) There were significant differences in response tendency between the four OJT methods (X² = 40.250, df = 9, p

< 0.001); between the lecture-, practice-, and homework- based methods; and between the lecture- and practice- based methods (X²= 16.337, df

= 6, p = 0.012 and X² = 14.825, df = 3, p = 0.002, respectively).

(3)

April 13, 2020

- The above results mean that the apprentices who received practice-based OJT perceived their level of utilization of what they had learned through the work–learning dual program most positively.

Significant differences in response tendency were found among the apprentices regarding their level of job performance capacity improvement through the work–learning dual program according to the type of OJT they received.

- The percentages of positive responses (helpful, very helpful) by OJT method were as follows: practice-based method, 89.6%;

lecture-based method, 84.7%; homework-based method, 83.5%;

04 Improved Organizational and Career Immersion through the Work–Learning Dual Program

| Apprentices who mostly received autonomous OJT were more skeptical of their training than those who received a different type of OJT on the extent to which their organizational and career immersion had enhanced through the work–learning dual program.

[Figure 5] Improved Organizational Immersion through the

Work–Learning Dual Program [Figure 6] Improved Career Immersion through the Work–Learning Dual Program

There were significant differences in response among the apprentices regarding their organizational-immersion improvement through the work–learning dual program according to the type of OJT they received.

⁷

- The percentages of positive responses (improved, improved a lot) by OJT method were as follows: practice-based method, 86.4%; lecture-based method, 82.4%; homework-based method, 79.0%; and autonomous method, 64.9%.

There were also significant differences in response among the apprentices regarding the improvement of their career immersion through the work–learning dual program according to the type of OJT they received.

⁸

- The percentages of positive responses (improved, improved a lot) by OJT method were as follows: practice-based method, 86.0%; lecture-based method, 81.1%; homework-based method, 77.3%; and autonomous method, 63.5%.

\ 05 Satisfaction with the In-Company Trainers and Results of External Evaluation

| Apprentices who mostly received autonomous OJT have a lower level of satisfaction with the in-company trainers than those who received a different type of OJT. On the other hand, there were no significant differences in the responses about the external evaluation results of the OJT methods.

[Figure 7] Satisfaction with the in-company trainers [Figure 8] Results of external evaluation (lecture-based

method) (N=341)

(lecture-based method)

method) (N=696)

(practice-based method) (N=677) (homework-

(N=119) (autonomous

method) (N=57)

(autonomous method)

(N=52) 56.6%

62.5%

52.9%

49.1%

55.6% 59.8%

53.8%

40.4%

25.8% 23.9% 26.1%

15.8%

25.5% 26.1%

23.5% 23.1%

Has not improved at all Has not improved much Has improved Has improved a lot

I Footnote I

6) There were significant differences in response tendency between the four OJT methods (X² = 32.228, df = 9, p _< 0.001) and no significant differences between the lecture-, practice-, and homework-based methods and between the lecture- and practice-based methods (X²

= 10.055, df = 6, p = 0.122 and X² = 5.944, df = 3, p = 0.114, respectively).

I Footnote I

7) There were significant differences in response tendency between the four OJT methods (X² = 34.766, df = 9, p _< 0.001) and no significant differences between the lecture-, practice-, and homework-based methods and between the lecture- and practice-based methods (x²

= 8.140, df = 6, p = 0.228 and x² = 4.420, df = 3, p = 0.220, respectively).

8) There were significant differences in response tendency between the four OJT methods (X² = 38.059, df = 9, p _< 0.001) and no significant differences between the lecture-, practice-, and homework-based methods and between the lecture- and practice-based methods (x²

= 9.770, df = 6, p = 0.135 and x² = 5.192, df = 3, p = 0.096, respectively).

(lecture-based method) (N=525)

(lecture-based method) (N=370) (practice-based

method) (N=929)

(practice-based method) (N=570) (homework-

(N=117) (autonomous

method) (N=69)

(autonomous method)

(N=51)

3.243 3.240

3.203

2.825

44.3%

51.2%

47.0%

23.1%

29.9% 29.4%

19.6%

51.0%

23.5%

25.3%

29.7%

25.9%

Pass Fail Awaiting results

(4)

KRIVET Issue Brief

｜ KRIVET Social Policy Building, Sejong National Research Complex, 370, Sicheong-daero, Sejong-si, Republic of Korea ｜ Tel: 044-415-5000/5100 ｜ www.krivet.re.kr ｜

It was found that the level of satisfaction with the in-company trainers was lowest for the apprentices who mostly received

autonomous OJT.

⁹

- The scores given by the apprentices for their level of satisfaction with the in-company trainers by OJT method were as follows:

lecture-based method, 3.243; practice-based method, 3.240; homework-based method, 3.203; and autonomous method, 2.825.

It cannot be said that there were significant differences among the apprentices regarding the external evaluation results of the OJT methods.

^{10, 11}

- Passing rates were achieved by all the OJT methods, as follows: practice-based method, 51.2%; autonomous method, 51.0%;

homework-based method, 47.0%; and lecture-based method, 44.3%.

- Meanwhile, there were failing rates within each OJT method, as follows: lecture-based method, 29.7%; practice-based method, 23.5%; homework-based method, 23.1%; and autonomous method, 19.6%.

- - \

06 Implications

The OJT method wherein the apprentices learn on their own and wherein an in-company trainer checks their learning results or achievements (autonomous method) is still being utilized, albeit relatively less frequently.

As opposed to the analysis results obtained 3 years ago (2016, KRIVET Issue Brief No. 118

¹²

), it is difficult to say if there is a large difference in effectiveness between practice- and lecture-based OJT.

- The analysis of this report’s data showed that practice- and lecture-based OJT significantly differ in the degree of their contribution to job performance capacity improvement and in the extent to which what is learned through the work–learning dual program is utilized. No significant differences in OJT method adequacy, overall job performance capacity improvement through the work–learning dual program, organizational- and career-immersion improvement level, level of satisfaction with the in-company trainer, and external evaluation results were found.

- The foregoing can be attributed to the overall improvement of in-company trainers’ OJT execution capacities with the continuation of the work–learning dual program, which should of course be carefully verified later.

The aforementioned results show that autonomous OJT is inefficient compared to the three other OJT methods. However, there were no significant differences within its external evaluation results.

- Although apprentices are adults, except for those enrolled in industry–university-integrated apprenticeship schools, most are in the early stages of their careers. Therefore, rather than leaving the apprentices on their own to learn, it will be more effective if in-company trainers clearly present to the apprentices the OJT goals and contents and increase their interaction with the apprentices through lectures, practice, or homework.

- The fact that there were no significant differences in external evaluation results among all the OJT methods shows that in- company trainers do not pay much attention to apprentices’ external evaluation of the OJT methods.

Kim, Daeyoung (Ph.D., Research Fellow, KRIVET)

I Footnote I

9) The average of the autonomous method was significantly lower than those of the three other methods (F = 9.537, p _< 0.001).

10) No significant differences were found between the four OJT methods; between the lecture-, practice-, and homework-based methods;

and between the lecture- and practice-based methods (χ2 = 7.804, df = 6, p = 0.253;

χ2 = 6.792, df = 4, p = 0.147;

and χ² = 5.590, df = 2, p = 0.061, respectively).

11) In the analysis of the data excluding “awaiting results,” all the OJT methods got passing rates, as follows:

autonomous method, 72.2%;

practice-based method, 68.5%; homework-based method, 67.1%; and lecture- based method, 54.9%. No significant differences were found between the four OJT methods and between the lecture-, practice-, and homework-based methods, but a significant difference was found between the lecture- and practice-based methods (χ² = 5.546, df = 1, p

= 0.019).

12) This is a result of the analysis not of the Work–Learning Dual-Program Status Survey (2016) results but of the data collected for the analysis of work–learning dual-program apprentices’ adjustment to the labor market and settlement o u t c o m e s . I t w a s a n analysis of the data obtained from 381 apprentices (196 receiving practice- based OJT, 185 receiving lecture-based OJT) in the machinery, information and communication, and electrical and electronics fields.