Donald Kirkpatrick's staff training assessment model. How to evaluate the effectiveness of Kirkpatrick training. D. Kirkpatrick's learning efficiency assessment model
This article is devoted to a comparative analysis of various models of the effectiveness of personnel training, including the most common models: Kirkpatrick, Berne, Stafflebeam and Phillips. The models under consideration were studied on the basis of secondary information, including a study carried out by the British company Embrion, as well as a study by the American company ASTD from a qualitative point of view: the composition, features, limitations and advantages with disadvantages of each of them were analyzed. The comparison results were verified using an expert survey carried out by the authors regarding the accuracy and prevalence of each approach to assessing the effectiveness of training. Based on the results obtained, conclusions were drawn regarding the applicability of each of the models in the practice of Russian companies. In addition, the authors offer some recommendations regarding the use of these approaches to obtain greater economic benefits.
Phillips model
Berne's model
Stafflebeam model
Kirkpatrick's model
staff development
assessment of the effectiveness of training
1. Kirkpatrick D.L., Kirkpatrick D.D. "Four Steps to Successful Training: A Practical Guide to Evaluating the Effectiveness of Training" Moscow 2008. S. 50-51.
2. Mironov V. Evaluation of training according to Kirkpatrick: a test of time [Electronic resource]. - Access mode: http://www.top-personal.ru/issue.html?2185 (date of access: -01.12.14).
3. Oparina N.N. Evaluation of the effectiveness of training and development of top managers [Electronic resource]. - Access mode: http://istina.msu.ru/media/publications/articles/e4f/7fc/422942/Otsenka_effektivnosti.pdf (date of access: 10.11.14).
4. Skiba E. Topic March 06: assessment of the effectiveness of training. Digest on nine basic assessment models [Electronic resource]. - Access mode: http://www.trainings.ru/library/articles/?id=6328 (date of access: 5.12.14).
5. Casebourne I., Downes A. Evaluation of learning –a family affair? [Electronic resource]. - Access mode: http://epiclearninggroup.com/uk/files/2013/10/WP_Evaluation_of_-learning.pdf (date accessed: 22.11.14).
6. Passmore J., Velez M.J. SOAP-M: A training evaluation model for HR // Industrial & Commercial Training, 2012. Vol.6, No. 44. P. 315-326.
7. Topno N. Evaluation of Training and Development: An Analysis of Various Models // Journal of Business and Management, 2012. Vol. 5, No. 2. P. 16-22.
In modern conditions of globalization, fierce competition in national and international markets, one of the factors to increase competitiveness is to improve the quality of the organization's personnel. However, the total amount of investments in personnel retraining, or various trainings aimed at developing the qualities of an individual employee necessary for the company, cannot serve as an accurate assessment of the effect obtained. Thus, an essential part of the problem is the question of how to assess the effectiveness of investments in personnel development. To this end, separate learning assessment models were developed, which mainly used the Kirkpatrick approach.
Kirkpatrick's model
Without assessing the effectiveness of training, it is impossible to build a training and development system that provides the necessary business results. In 1954, Donald Kirkpatrick defended his Ph.D. thesis at the University of Wisconsin (USA) on the topic "Evaluating the effectiveness of program management." He proposed a concise formula for describing the learning cycle: response - learning - behavior - results (see Figure 1). This scheme is necessary for effective training of company personnel, as well as to obtain the required business result.
Fig. 1 Kirkpatrick's model
The first level of learning is reaction. The reaction allows you to know what the clients think about the program, based on the answers, the trainer needs to make some changes to the program to improve it. Assessment of the first level, "Reaction" is very important, the trainer needs to know the opinion of the participants about their program, and the listeners must be sure that their opinion is not indifferent to the trainer.
Second level: knowledge gained. It is good if the participants are happy with the training, but this does not mean that they have learned something. To assess the knowledge gained, tests on the knowledge of the studied material are used, and also skill check sheets are used. Testing is as follows: some time after completing the course, participants are invited to a conversation with an experienced colleague or supervisor to find out what knowledge they have learned from the training.
Third level: behavior. Kirkpatrick defines this level as the most important and difficult. It is at this level that the assessment of how the behavior of the participants has changed as a result of training, how the acquired knowledge and skills are applied in the workplace.
Fourth level: results.
The main thing in assessing the fourth level is to imagine the final result. The task of the trainer is to fulfill the expectations of the stakeholders, i.e. payback.
Donald Kirkpatrick's method shows us how to make an effective business tool out of the learning process, and to integrate a separate training into an organization, to make training a corporate one. It explains in detail how to plan, how
set goals, how to justify the need for training to managers, and, which is very important, how to exclude unnecessary training if it does not solve the stated problems with its help.
In practice, however, not all four levels are used, but more often only two, in decreasing order of importance (based on research conducted by the American Society for Training and Development ASTD) (see Fig. 2).
Figure 2 Organizations' use of Kirkpatrick's model levels
Thus, it is necessary to approach the assessment of the effectiveness of the learning process from the standpoint of financial management and quantify the results of such an investment. The solution to this problem was proposed in 1997 by Jack Phillips.
Jack Phillips model
In the early 90s, renowned HR expert Jack Phillips developed a methodology for evaluating the effectiveness of investments in employee training. In his work, published in 1975, Kirkpatrick expressed an unexpected thought: “Do not, under any circumstances, try to convert the fourth level (business outcome) into money. It is not necessary, you cannot. " And in 1991, another person, Jack Phillips, not only stated that it was possible, but also proposed a calculation algorithm. One more level has been added to the already named four levels - return on investment, or the calculation of the ratio of the profit from the project and the cost of it.
This indicator allows you to assess the effectiveness of training. Phillips' techniques work well in regularly-managed companies.
ROI helps to obtain a number of benefits, in particular, for company leaders: to assess the financial performance of investments in employees; get a clear and reliable tool for determining the effectiveness of activities for working with personnel; to make measurable the "human factor" and its impact on the business result of the company. First of all, the ROI model is needed to calculate the return on investment in human resources and in HR programs, when management invests in people, they need to understand what the return on these investments is.
Stufflebeam Model (CIPP)
The Stafflebeam abbreviation (see Fig. 3) stands for the following:
Rice. 3 Stufflebeam Model (CIPP)
The model allows you to evaluate both the results and the learning and development process itself. Therefore, for long-term modular training programs for top managers, this model can be successfully applied.
Bern Model (CIRO)
The CIPO scheme (see Fig. 4) developed by Berne is similar to the Stafflebeam model. The model includes the following stages of assessment:
Rice. 4 Bern Model (CIRO)
This model can also be used to assess the effectiveness of training top managers, especially in the context of launching modular long-term programs.
Advantages and Disadvantages of Effective Learning Models
|
Advantages |
Flaws |
||
|
Kirkpatrick |
The presence of four levels of assessment: according to the reaction of the participants, according to the results of the final control, according to changes in production behavior and according to changes in the company's business indicators. Choice for assessing the effectiveness of training a top manager. |
The difficulty of measuring the third level, the unwillingness of top managers to undergo the assessment procedure or work with coaches. Refusal to evaluate according to financial indicators. |
|
|
Development of the Kirkpatrick model. Introduction of the fifth level for assessing the effectiveness of training and development. The ability to use a financial indicator of return on investment in training and development. |
The calculation of the return on investment is possible only if a full-fledged management financial accounting is maintained in the organization. It is mainly used to calculate the effectiveness of those programs that are long, expensive and complex. |
||
|
Stafflebeam |
The ability to evaluate both the process and the results of training and development in close connection with the goals of the company. |
Lack of clarity of methods and procedures for assessing the learning outcomes and development of top managers. |
|
|
Development of the Stafflebeam model and partly of Kirkpatrick. Setting goals, identifying opportunities, identifying participants' views on learning and development, evaluating results. |
The difficulty of maintaining a constant procedure for assessing the effectiveness of training and development of top managers, application for long-term programs. |
||
The table shows a comparison of the most popular models and methods for assessing the effectiveness of development and training of personnel in terms of advantages and disadvantages, including financial, computational complexity, and so on.
It is worth noting that today there is no ideal model of effective learning that would include all the necessary assessment parameters. Each of the presented models has its own pros and cons, which are clearly presented above.
The authors conducted a survey of 7 HR experts on the most accurate models for assessing the effectiveness of training (each expert had to choose the most accurate models, in his opinion, from the proposed list). The survey data are presented below (see Fig. 5):
Fig 5. Results of the survey on the accuracy of training models
This survey showed that the Kirkpatrick and Phillips models are the most accurate learning models, and the Stafflebeam and Berne models are the least accurate.
Conclusion
In conclusion, we can say about the need to take into account the difference between the Western culture of education and ours. In the West, corporate training has long become an indispensable condition and part of business life; there is no need to explain many things to employees. In Russia now, the attitude towards business education has begun to change in the direction of interest, and the technologies themselves are just beginning to be introduced. Thus, the interviewed respondents showed that in general, effective training models are most often used by Western companies. The most accurate models can be called the Kirkpatrick and Phillips models. The least accurate models include the Stafflebeam and Berne models; according to the respondents, these models are least often used both in the West and in Russia.
Kirkpatrick's model shows how to turn learning into an effective business tool and how to integrate individual training into an organization. The Phillips model helps an organization measure the financial results of learning and development. The Stafflebeam model allows you to evaluate both the results and the learning process itself. Berne's model helps us to define goals, opportunities, opinions of the participants of the training. Thus, we can say that each of the presented models has its own advantages and disadvantages. Currently, there is no ideal model that would include all the parameters for evaluating effective learning at once.
Reviewers:
Putilov A.V., Doctor of Technical Sciences, Professor, Dean of the Faculty of Management and Economics of High Technologies, Ministry of Education and Science of the Russian Federation, Federal State Autonomous Educational Institution of Higher Professional Education "National Research Nuclear University" MEPhI ", Moscow ;
Tupchienko V.A., Doctor of Economics, Professor, Professor of the Department of Business Project Management, Ministry of Education and Science of the Russian Federation, Federal State Autonomous Educational Institution of Higher Professional Education National Research Nuclear University MEPhI, Moscow Moscow.
Bibliographic reference
Udovidchenko R.S., Kireev V.S. COMPARATIVE ANALYSIS OF MODELS FOR ASSESSING THE EFFICIENCY OF STAFF TRAINING // Modern problems of science and education. - 2014. - No. 6 .;URL: http://science-education.ru/ru/article/view?id=16909 (date of access: 23.12.2019). We bring to your attention the journals published by the "Academy of Natural Sciences"
D. Kirkpatrick
50 years ago, training professionals first became aware of the Donald L Kirkpatricks training evaluation model - four levels of learning evaluation. Today, this system has gained incredible popularity, having established itself as the most effective and professional approach to training. According to this model, the effectiveness of learning must be assessed at four levels: response, learning, behavior and results.
There are other approaches to assessing the effectiveness of training: Tylers Objectives Approach; SCREENS 'model, (Scrivens Focus On Outcomes); Stufflebeam CIPP model; CIRO scheme; Gubas NaturalisticApproach, Bruce Aarons Model, Jack PHILIPS ROI (Return on Investment) model.
However, all of them to one degree or another are derived from the Kirkpatrick model or use its elements. That is why D. Kirkpatrick's system has been actively used all over the world for exactly half a century. The D. Kirkptrick system will be presented in Moscow in November by Jim Kirkpatrick, son and follower of Donald Kirkpatrick.
Our model provides a real opportunity to assess the real effect of training. This is the first thing to talk about with company leaders when discussing what they would like to see in the training program presented to them, in other words, at the stage of identifying their expectations. At this stage, it is very important to work with the management of the company. This is necessary in order to come to a consensus on the following issues: what kind of employee behavior will meet their expectations; what types of training are needed to achieve the set goals; what atmosphere must be created for people so that they not only undergo training, but also receive real practical benefits from it. Working through all four levels in detail is the best way to improve your training programs to truly meet your business needs.
We also insist that business coaches actively work with company leaders at the third level - the level of applying the skills developed in solving work problems and situations. It is at this stage that we get a specific positive or negative result. After all, once in a familiar work environment, people behave differently than during training!
We are also convinced that the client and employees sent to training must clearly understand what training is needed for. Too often people are sent to training without understanding why the company needs it and what it will bring in the end to the students themselves. And management's attention to assessment and prioritization of the value of learning will help to give some kind of integrity to the learning process.
The key step in our four-tier model in determining learning effectiveness is, without a doubt, level 3 - behavior. Learning will be of no value if the knowledge learned does not translate into the behavior of the learner. We even wrote a book about this, calling it "Transforming Learned into Behavior."
We know that if there is no such development of knowledge gained in the learning process, the desired result will never be achieved. Let's give an example of effective level work. In an interview with Olympic figure skating champions from Russia (unfortunately, I do not remember their names), when asked by a reporter about how they intend to win, they replied that they would just go out on the ice and perform all the elements that they perform every day. That is, the learning process is reflected in daily behavior. The same can be said for business. When what the employees have been taught turns into a skill, into a norm of work, it will be possible to say that the training was completed successfully. Speaking about the results, we would like to note that our model differs from others in that we do not try to consider the material effect of learning in isolation. Rather, we collect all the data and all the necessary information (facts, evidence) to demonstrate the value of education, training and coaching in terms of practical results. We ask the company's management: what are their expectations from the program? Then we ask the question: "What is the company's success?" Only when they formulate their expectations by saying at least something like “sales revenue” or “saving money” or “keeping employees in the company” can this be reflected in the fourth level, that is, the necessary result.
Donald Kirkpatrick is Professor Emeritus at the University of Wisconsin, author of a four-tiered approach to evaluating the effectiveness of training programs, business coach, and Chairman Emeritus of Kirkpatrick & Partners.
He is the founder of the Kirkpatrick model, which is recognized as the best model for professional training in the world.
Kirkpatrick's contribution to the development of training programs
When D. Kirkpatrick worked as a teacher at the university, the idea came to him that it would be nice not only to conduct classes, but also to evaluate them.
Initially, he wanted to gauge the reaction of the audience to his lectures and seminars. But this was not enough. Then he wanted to know if the students learned anything thanks to his studies, how they apply this knowledge in practice, whether their behavior has changed and whether they have received positive results?
This is what served as the basis for the well-known model for assessing the effectiveness of training. This model consists of four levels:
1) Reaction (assessment at this level determines how the participants in the program react to it, i.e. it is a series of thoughts, feelings and emotions of students).
2) Learning (at this level, students' acquired knowledge and skills are assessed).
3) Behavior (this stage is characterized by the assessment of changes in the behavior of students under the influence of the acquired knowledge).
4) Results (this includes all changes that have occurred to the participants in the training program).
Subsequently, Donald Kirkpatrick wrote a thesis on “Evaluating Learning in Human Relations. Executive Program and received his Ph.D. from the University of Wisconsin.
What did Kirkpatrick write?
Five years later, he wrote four articles for the American Society for Learning and Development. These articles dealt directly with each of the levels of the learning assessment model. The society then recognized Kirkpatrick as a “guru” in learning assessment.
He is serious enough about the application of his learning assessment model and argues that dwelling on the first and second is simply unacceptable. It is necessary to obtain information on each stage.
Major works of Donald Kirkpatrick:
- Evaluating Training Programs: Four Levels (1975);
- “Four Steps to Effective Training”;
- “How to Plan and Conduct Production Meetings”;
- "Effective change management".
All of his books have become bestsellers and have sold millions of copies.
Peru D. Kirkpatrick owns a huge number of articles that were published from 1959 to 2011.
Kirkpatrick's achievements
The controversy surrounding the model proposed by Kirkpatrick does not subside, but, nevertheless, it was and remains one of the main in the field of training and education.
Donald Kirkpatrick's career has been very successful: he has written several books that are currently the most cited; was president of the American Society for Learning and Development.
For a long period of time, he continued to engage in social activities, write articles and conduct seminars at the largest forums in the world.
The famous coach devotes all his leisure time to fishing, golfing and family.
Today, Donald Kirkpatrick has retired from publicity and began to use his literary and musical talent in songwriting.
Links
This is a preparation of an encyclopedic article on this topic. You can contribute to the development of the project by improving and supplementing the text of the publication in accordance with the rules of the project. You can find the user manual
Kirkpatrick's four-level scoring model
Perhaps the most famous model for assessing the educational process isDonald Kirkpartick model
The four-tiered valuation model, which was first presented in a series of articles in 1959 in theJournal of American Society of Training Directors
(now known as T + D Magazine). This series was later compiled and published as an articleTechniques for Evaluating Training Programs in Kirkpatrick's book Evaluating Training Programs in 1975.
However, the four levels of Kirkpatrick's model only became popular after the publication of his book Evaluating Training Programs. in 1994. These four levels have become the cornerstone of the learning industry today.
While most experts consider these four criteria for assessing the learning process as "levels", Kirkpatrick himself never used this term, he usually called them steps (Craig, 1996). I will add that he did not call it a model, but used words such as “techniques for conducting the evaluation” to refer to it (Craig, 1996, p294).
These four steps consist of:
Kirkpatrick's concept very important because it is a great tool for planning, assessing, diagnosing problems, especially if we make the minor improvements shown below.
Not just for training
Some experts mistakenly assume that Kirkpatrick's model can be used for training (here we mean formal learning processes: trainings, seminars - E.B.'s note), but the model could be used in work with other educational processes. For example, a profession Human Resource Development (HRD) is concerned not only with the development of formal learning processes such as coaching, but also other forms such as informal learning, development, mentoring, and nurturing (Nadler, 1984). The manual, published by Human Resource Development (HRD) co-founder Leonard Nadler (1984), uses the Kirkpatrick's four-level model as the primary model for assessing the learning process.
Kirkpatrick himself wrote: “These goals (referring to his own article) will relate to in-house classroom programs. Many procedures and principles apply to all types of learning activities, such as performance analysis, participation in external programs, programmed learning, reading specially selected literature ”(Craig, 1996, p294).
Development of the four-tier model
Due to the "age" of the model, as well as with the advent of new, modern technologies, Kirkpatrick's model often criticized for being too outdated and simple. Nevertheless, even five decades after its presentation, there was no adequate model that could replace the Kirkpatrick model. And I'm pretty sure the reason why the model swaps don't happen is because Kirkpatrick was mostly right, but he made a few small mistakes:
Motivation, Not Reaction
When a learner moves in the stream of an educational process, such as an e-learning course, an episode of informal learning, or receiving assistance from a mentor in carrying out his work activities, he must decide what to focus on first. If the goal or tasks are assessed by him as important or feasible, then the student is normally motivated to achieve them. (Markus, Ruvolo, 1990). But if the task seems to him to be weakly related to his activities, or there is only a small probability of successful completion of this task, respectively, the motivation for performing is low. In addition, research evaluating Feedback usually shows that it is not a valid tool for evaluating success (see the last section of the Critique article)
This is different from wordsKirkpatrick(1996), who wrote that Feedback is the extent to which learners evaluate a particular learning process. However, the less the teaching material is to the student, the more effort must be made to develop and present the material. Therefore, if the training material is not related to the needs of the student himself, then the training material should "catch" the student with sophisticated design, humor, games, etc ... This does not mean that design, humor or games are unimportant; however, their use in the learning process is much more than just making the process fun, they help and advance the learning process. And if the teaching material is based on clear goals and design, then it helps the student to bridge the gap between existing and required activities. Therefore, they (design, humor, games) should motivate to learn - if this does not happen, then something went awry in the planning and creation of the educational process! If you suddenly catch yourself thinking that you are trying to hook the trainees with the help of all sorts of enticements (design, humor, games), you should probably reconsider the goals of your educational process.
Activity, not behavior (Performance, Not Behavior)
Performance is better than behavior because activity has two aspects: behavior is the means and the effect of behavior is results. And in fact these results interest us most of all Gilbert noted (1998).
Turn it into a Better Model
The model, where the two most important points - results and behavior - are at the end, is turned upside down, as the importance of the points is imprinted in people's heads in the order they are placed. Thus, we reverse the model and add the above and get:
This model becomes a tool for both planning and assessment = problem diagnosis (Chyung, 2008):
Objectives (planning) | Assessment level |
| What are the goals of our company for business development? | results Have you felt the expected effect? |
| What do our learners need to be able to do to achieve these goals? | Activity Have the trainees transferred the acquired skills to work? |
| What new skills, knowledge and resources are required for learners to be able to perform the activity? | Education Have the trainees mastered the necessary skills and / or resources? |
| What do learners need in order to learn and perform consciously? | Motivation Are they motivated to learn and get the job done? |
The revised model can now be used for planning (left column) and estimation (right column). In addition, it can be used to diagnose problem areas in the learning process. For example, you know that employees received the necessary skills in the learning process, but do not apply them in the process of work, then the following problem areas become obvious (in this example, this is the Activity cell or the cell to the left of it):
The diagram below shows how these revised model processes fit together.
Organization
1. Results
Cumulative score
Activity
2. Working environment
Formative assessment
Learning environment
3. Training
Formative assessment
People
4. Motivation
Formative assessment
As the diagram above shows, Outcome Assessment is most interesting to business management, while the other three levels of assessment (performance, learning and motivation) are the main ones for the training designer in the company to plan and evaluate educational processes; and of course Performance Evaluation is also important to him as it is a supplier of business development goals.
Level One - Results
Results or impact
Although it is usually more difficult and time consuming than the other three levels, it provides information that is of immense value: it proves the value of learning processes and performing activities. However, using the Objective / Plan / Assessment model should make the process simpler and easier, and you will have a clear picture of what you are trying to achieve. That is. When you start planning something, you are more able to figure out how to evaluate it.
Motivation, Learning and Activities are more a matter of “soft” units of measurement (so-called non-financial metrics); however, decision-makers who validate learning processes prefer outcomes (return on investment or impact). Jack Phillips (1996), who probably knows the best four-levelKirkpatrick's model , writes that the value of information becomes much higher if we go from motivation to results.
The above does not mean that the other three levels are useless, in fact, the benefit of using them lies in the localization of problems in the educational process:
Note the difference between "information" and "refunds." Measures of Motivation, Learning and Performance provides you with information to improve and evaluate the learning process, which is largely owned by the designers of learning in the company; while measuring Outcomes gives you a measure of return on investment in an educational process that has more to do with business leaders.
Performance measurement can be done in a balanced scorecard approach (Kaplan, Norton, 2001), which shows the impact or return on investment from four perspectives:
Level two - Activities
This assessment includes testing the trainee's ability to incorporate the skills acquired in the learning process into work. This assessment can be done formally (testing) or informally (observation). It is determined by how the question is answered: Do workers use newly acquired skills in the process of work?
It is very important to measure performance, since the basic goal of corporate training is to improve results by transferring new knowledge and skills to employees so that they can then apply them in their work. Measurement of performance should be as long as the workers do their job; the measurement should usually be taken by the specialist who is closely associated with the trainee: a supervisor, a trained observer or an interviewer.
Level three - Education
It is an indicator of how deeply learners assimilate knowledge, develop skills and change attitudes and attitudes as a result of participation in the educational process. Assessment of learning usually requires post-testing in order to establish what skills were learned by the trainees during the process, what skills the trainees already possess.
Measuring the outputs of the learning process is important from the point of view of validating the learning goal. Measuring learning usually focuses on the following questions:
A trainee assessment is created to ensure that the trainee is able to perform an activity. There are two aspects of this process: the actual collection of information and data (testing the trainees) and the assessment of this information (what does this data communicate about?). This assessment should not be confused with a learning assessment (evaluation). The Assessment shows the progress and individual achievements of the trainees, while the Assessment of Learning (evaluation) generally on the evaluation of training programs (Tovey, 1997, p88).
Level four - Motivation
Assessment at this level measures how learners perceive and respond to learning processes and activities. This level is measured using the attitude questionnaires (motivational questionnaires) offered after many training sessions. The learners are most often well aware of what they need to complete the task. If the educational process did not satisfy their needs, then you need to decide: either it is a mistake of the pedagogical designer (a specialist who designs the educational process in the company), or the trainees did not appreciate the advantages of the process.
When a learner begins to study a study material, no matter if it is eLearning, mLearning, classroom training or learning through social media, he must decide what, first of all, he pays attention to in this material.If the goal or tasks are assessed by him as important or feasible, then the student is normally motivated to achieve them. (Markus, Ruvolo, 1990). But if the task seems to him to be weakly related to his activities, or there is only a small probability of successful completion of this task, respectively, the motivation for performing is low.
Criticism
There are three problematic assumptions of the Kirkpatrick model: 1) the levels are not arranged in ascending order (there is no hierarchy of levels); 2) the levels are not causally related to each other; 3) levels are positively correlated with each other (Alliger and Janak, 1989).
The only part of Kirkpatrick's four-tier model that hasn't stood the test of time is Feedback (first tier). For example, the School Trainer of the XXI century has some of the lowest indicators in the first level - the feedback level, but is responsible for the high performance indicators (fourth level), which measure the productivity of its graduates. And this is not an isolated case: from study to study, results show a very low correlation between Feedback and how well workers perform after training (Boehle, 2006).
Much more important than measuring feedback, as we have established, is preparing trainees for the learning process through conversations with the supervisor about the need to participate in the training, followed by their accompaniment in order to make sure that the skills transferred in the learning process are mastered (Wick, et al. 2006), and this is another reason why Feedback should be changed to Motivation.
The Kirkpatrick four-level model is applied only after the end of the educational process. While the purpose of the assessment should be to accompany the entire educational process, starting from the initial stage - the stage that precedes the educational process.
In fact, this criticism is not accurate. For example,The ASTD Training & Development Handbook (1996) published by Robert Craig includes chapterKirkpatrickwith the simple title "Assessment". In this chapter, Kirkpatrick discusses control groups as well as before and after methods (such as pre- and post-testing). He continues to discuss that the fourth level should include post-training assessment three or more months after the end of the training process in order to monitor how the trainees began to apply the skills learned during the training. Kirkpatrick further notes that Assessment should be included in all stages of the learning process, not only during each session or module, but also after each subject or topic.
The four-level model only works for classroom teaching and not for other forms of learning.
As noted in the section “Not just for training” (see above),Kirkpatrickwrote about the possibility of using a four-level model for other types of learning and HR, which helps to provide services for the processes of both formal and non-formal learning, and whereKirkpatrick's model acts as one of the main assessment models. And maybe the real reason that non-formal education advocates do not see the benefit of usingKirkpatrick's models that the four levels "were not invented for them."
Four levels of assessment are of little relevance to other parts of the company and lines of business
One of the best learning and development booksThe Six Disciplines of Breakthrough Learning by Wick, Pollock, Jefferson, Flanagan (2006). They seem to offer the most grounded criticism I've seen: “Unfortunately, this model is not widely used among line managers who are directly related to learning outcomes. Therefore, when training leaders write or speak in terms of grade levelsKirkpatrick's models with colleagues in business, instead of clarifying it, it often confuses the discussion and requires additional effort to build an understanding between the business and the learning function. ”
And it is quite possible that this criticism is essentially not directed against the veryKirkpatrick's models but against those of us who build communications with business. We say to business: the first level shows how happy the trainees are from learning, at the second level they successfully passed the test, etc. throughout the model. Plus, according to research I've seen, learning outcomes are rarely used (which is what businesses value the most). All other levels of assessment can only be useful in the design of the educational process, as they help us understand which type of assessment to apply in which problem situation. However, outside of this framework, these levels are of no interest to anyone. Most of the management is only interested in the impact of training and the answer to the question: the resources that we spent on training employees contribute to the development and prosperity of our business?
Learning assessment is carried out in order to understand how to increase its efficiency, in what ways can it be improved.
In this regard, it is proposed to answer the following eight questions:
· To what extent does the content of the training meet the needs of the participants?
· Is the choice of the teacher optimal?
· Does the trainer use the most effective methods to maintain the interest of the participants, transfer knowledge to them, and build their skills and attitudes?
· Are the training conditions satisfactory?
· Are the participants satisfied with the class schedule?
· Do audiovisual media improve communication and keep participants interested?
· Was the coordination of the program satisfactory?
· What else can be done to improve the program?
Learning assessment has four levels:
1. Reaction
2. Learning
3. Behavior
4. Results
Level 1 Assessment - Reaction determines how participants in the program respond to it. In the case of in-house training, the response of the participants is not always interpreted as customer satisfaction. The fact is that participation in such trainings is mandatory. People just don't have a choice. The company's management determines the need for this training and obliges employees to take part in it. It would seem, in this case, it is necessary to talk about the reaction of the leadership. Kirkpatrick emphasizes that in this case too the reaction of the participants is a very important criterion for the success of the training, for at least two reasons. First, people share their impressions of the training with their management, and this information goes higher. Therefore, it influences the decision to continue the training. Second, if participants do not respond positively, then they will not be motivated to learn. According to Kirkpatrick, a positive reaction does not guarantee successful mastering of new knowledge, skills and abilities. A negative reaction to training almost certainly means a decrease in the likelihood of learning.
Teaching determined how to change attitudes, improve knowledge and improve the skills of participants as a result of their training program. Kirkpatrick argues that a change in the behavior of participants as a result of training is possible only when learning occurs (attitudes change, knowledge improves or skills improve). Behavior - n At this level, an assessment is made of the extent to which the behavior of the participants has changed as a result of the training. Kirkpartik points out that the absence of changes in the behavior of the participants does not mean that the training was ineffective. Situations are possible when the reaction to the training was positive, the learning took place, but the behavior of the participants did not change in the future, since the necessary conditions for this were not met. Therefore, the absence of a change in the behavior of the participants after the training cannot be a reason for making a decision to terminate the program. Kirkpartik recommends in these cases, in addition to assessing the reaction and learning, to check the presence of the following conditions:
· Participants' willingness to change behavior.
· Participants have knowledge of what to do and how to do it.
· The presence of an appropriate socio-psychological climate.
· Rewarding participants for behavior change.
TO results refers to the changes that have occurred due to the fact that the participants received training. As examples of results, Kirkpatrick cites increased productivity, improved quality, fewer accidents, increased sales, and decreased turnover. Kirkpatrick insists that results should not be measured in money. He believes that the above changes may, in turn, lead to increased profits. Assessment at this level is the most difficult and costly. Here are some practical guidelines that can help you measure your results:
If possible, use a control group (not trained),
To carry out the assessment after some time so that the results become noticeable,
If possible, conduct an assessment before and after the program,
