CHAPTER 2: LITERATURE REVIEW
2.10 Kirkpatrick’s Four Level Evaluation Model
Kirkpatrick ‘s model (1959) has been used to measure training effectiveness for more than fifty years. As depicted in Figure 2.1, Kirkpatrick ‘s model is considered to be the critical evaluation criteria to measure the effectiveness and/or efficiency of training programmes. The model identifies weaknesses and further improves future training programmes (Saks & Burke-Smalley, 2012). According to Chen et al. (2007), Kirkpatrick’s model provides a straightforward and simplified system of evaluating the training outcomes. However, Bates and Coyne (2005) identified several shortfalls or limitations of the models such as incompleteness, and assumption of causality. Supporting these limitations were Ford and Kraiger (1995) who indicate that the over simplicity of the model in evaluating the effectiveness of training omitted essential factors such as trainee characteristics, work environment, training design, continuous support for the acquisition of skills and change in trainees’ behaviour. In 1959, the original idea of Kirkpatrick ‘s model and its related methodology was developed, and since then, it has become well established within the training and development profession (Homklin et al., 2013; Saks & Burke, 2012). Kirkpatrick (1959, 1967; 1994, 1996, 2006, 2008) proposed the four levels for this model: reaction, learning, behaviour, and results.
36 | P a g e Figure 2. 1 Kirkpatrick four-levels model -adapted from Homklin et al. (2014, p. 15)
The model has helped organisations to focus on training evaluation practice on measuring outcomes and emphasised the importance of examining multiple measures of training effectiveness (Homklin et al., 2014). The first level of the framework assesses the reaction of the trainees on the training they attended. This level assesses reactions to training factors such as the trainer, training content, duration of the training and training setting. Kirkpatrick (1959), initially discussed this level of how well the trainees enjoyed a training program. In practice, the measures at this level have advanced and directed at assessing the affective responses of trainees on the quality or relevance of the training intervention (Homklin et al., 2014). Positive reactions to a training program may motivate and encourage trainees to attend future programs. However, if the trainees did not like the program or do not see its relevance, there is a possibility of reluctance in transferring the skills and knowledge in the workplace.
Furthermore, other trainees may be discouraged from attending such a training programme in the future. In summary, this level measures the trainees’ attitudes and perceptions of the training program. Level two determines the extent to which learning has occurred. Learning is when there is a change of attitudes, improved skills, and knowledge as a result of training (Kirkpatrick, 1996).
37 | P a g e He further emphasised that no change in behaviour can be expected from the trainees unless there has been the attainment of one or more of these learning elements. Rouse (2011) accentuates that though trainees might possess skills, knowledge, and attitudes acquired from the training, application of such cannot be guaranteed. Training effectiveness is when learned skills and knowledge are retained and applied on the job (Rouse, 2011). In understanding the emphasis by Rouse (2011), it is clear that training organisations should find innovative ways to ensure transferability of learning to the job after the training. Level three refers to change in trainees’
behaviour as a result of training intervention (Kirkpatrick & Kirkpatrick, 2008). The change in behaviour on this level refers to the transfer of learning on the job, meaning the application of new knowledge, skills or/and attitude. The efforts of the training are ineffective if learned skills, knowledge or attitudes are not transferred back on the job (Homklin et al., 2014). Levels one and levels two evaluations should be concluded before proceeding to the third and fourth levels of evaluations (Kirkpatrick & Kirkpatrick, 2008). The final level of the model measures the impact of the training on the organisation. The measures can include the attainment of organisational goals and objectives, such as a reduction in absenteeism and personnel turnover, productivity gains, and cost reductions. It is worth noting that Kirkpatrick ‘s four-level model is significant to achieve more exceptional results and evaluating the effectiveness of training programmes (Kirkpatrick &
Kirkpatrick, 2006).
2.10.1 Critiques of Kirkpatrick’s Four Level Evaluation Model
Notwithstanding the valuable contribution by Kirkpatrick’s model to the thinking and practices of evaluating training programmes, the model has been critiqued by some practitioners and researchers critiqued the model. There are at least three limitations of Kirkpatrick’s model in evaluating training programs. These include the incompleteness of the model, the assumption of causality, and the assumption of the increasing importance of information as the levels of outcomes are ascended (Bates, 2004). The model does not consider the evaluation of contextual factors that significantly influences the effectiveness of training (Baldwin & Ford, 1988).
The incompleteness of the Model
The model presents an oversimplified view of evaluating the effectiveness of training by not considering individual or contextual factors that influencing training effectiveness (Bates, 2004).
38 | P a g e Over the past two decades, different research studies have documented the presence of a wide range of organisational, individual and the design and delivery of training as factors that can influence the effectiveness of training before, during or after the training (Bates, 2004). Predictors such as post-training motivation and self –efficacy to transfer learning, peer support, collaboration, team learning and strategic link of performance transfer to the career advancement of the trainees have shown to have a positive influence on the transfer of learning on the job (Tews & Tracey, 2008). Kirkpatrick’s evaluation model implicitly assumes that an examination of these factors is not essential for evaluating the effectiveness of training (Bates, 2004).
Causal Linkages Assumptions
Kirkpatrick’s model assumes that the levels represent a causal linkage. The causal link such as positive reactions leading to more significant learning, and positive learning producing higher job transferability, and subsequently leading to positive organisational impact. Kirkpatrick states that
“if training is going to be effective, it is important that trainees react favourably”
(Kirkpatrick,1996:27), and that “without learning, no change in behaviour will occur”
(Kirkpatrick, 1996:51). Research, however, has failed to confirm such causal linkages by Kirkpatrick. Kirkpatrick (1996) found a causal relationship between the four levels of his model.
Furthermore, positive reactions of trainees were found to increase learning and learning lead to change in behaviour, and change in behaviour lead to desired organisational impact. Thus, the lower (from reaction to results) level of Kirkpatrick ‘s model must be evaluated first to gain useful results from the evaluation (Alliger & Janak, 1989).
The reaction level of trainees should be evaluated first as positive reactions to a training programme may encourage trainees to attend future training programmes. In contrast, adverse reactions may discourage them from attending future training (Reio Jr. et al., 2017). In other words, it might be difficult or even impossible to have a proper and good evaluation at the upper levels of the model unless the lower/preceding levels are measured first. According to Stewart and Brown (2011) , the outcomes of all the four levels provide different types of information about the training programme which might be less or more beneficial depending on what the purpose of the evaluation was.
39 | P a g e Incremental Importance of information
Kirkpatrick’s model assumes that data from each of the preceding levels of the evaluation process is more informative and essential than the last (Bates, 2004). Consequently, this assumption has led to perceptions that establishing level four results will provide the most useful information about training program effectiveness. However, the weak conceptual linkages that occur within the model and resulting data generated do not provide a sufficient basis for this assumption (Homklin et al., 2014). As a result of the critiques and pressures in organisations to measure the return on investments of the pieces of training, the model has been continuously improved to overcome difficulties in training evaluations (Al-Mughairi, 2015). The critiques of Kirkpatrick’s model helped in the expansion of the model as well as the development of new training evaluation frameworks. The following section describes the developed frameworks in improving the shortfall of Kirkpatrick’s model.