Designing & Measuring Training

Hi all,

Thanks for today's posts on the basics of training measurement and design, and your personal stories of training successes and mistakes.

Muller's (2013) three steps for developing a training program are a good place to start a training design effort:

1. Set program goals.
2. Determine instructional design/medium(media)
3. Establish success measures.

Meister and Willyerd's (2010) social learning ecosystem can be helpful in step 2, as can the 70/20/10 model. Step three is where Kirkpatrick (1998) comes in -- see Eric's post for a great discussion linking Kirkpatrick to Muller.

Your posts included a variety of interpretations on Kirkpatrick's levels, so let's try to focus on a common understanding here.

Level 1 - Reaction

This level measures how participants feel about the program. Did they enjoy it? Do they feel they learned anything? How did they like the training room or the instructor? All personal reactions to the program. Check out Paul and Derek's posts for a couple of guys who were having a less-than-positive reaction to their training; Derek with a computer-based module, and Paul with reading. How would their companies have benefited from a collecting data about their reactions? How could that data then be used to improve the experience of future new hires?

Level 2 - Learning

This level measures the learning -- in this case knowledge -- gained. Brian, Janine, and others talked about testing related to the training they went through. This testing checks on learner retention of facts, polices, etc., from the training session. This is an example of Kirkpatrick's second level of measurement.

Level 3 - Transfer of Behavior

Level three is all about how the new behaviors learned in training are transferred to the job. Again, Janine's post provides a helpful illustration here. She went through the training (in this case, sexual harassment prevention), took the exams (level 2), and then went back to the job. There she experienced no support or acknowledgement of the training she had received, nor was there any measurement of her behavior in regards to the training (at least that was visible to her), leaving her wondering about the importance of the training to the organization. This is a classic training problem. People go to training programs, but the learning in those programs is not acknowledged or supported by their managers when they return to the job. This limits the likelihood that the new learned behaviors will be retained and transferred to the job. And not measuring employees' new behaviors limits the organization's ability to measure the effectiveness of the training program.

Level 4 - Results

This level measures impact of the training on the business. This could be in terms of revenue generation, costs savings or avoidance, or more intangible aspects of organizational life (e.g., inclusive organizational culture, etc.). The more hard numbers that be applied at this level, the better. Rob provides us with an interesting example in his post. His training consisted of a lot of safety training (a good thing), but then the measures applied after training were all related to productivity. Rob's effectiveness in terms of workplace safety was not referred to again. This is problematic in two respects. First, the company doesn't know if their safety training programs have been effective at the worker-level (though they may have some store-level measures in use). Second -- and this is particularly important related to safety training, as well see in a another case in a few weeks -- the message to Rob and his co-workers is that production is of primary importance to the company. Safety is practiced in the training room, not on the retail floor.

Conclusion

The main thing here is to (1) measure training effectiveness, and (2) to link training measures to the impact it has on the business -- Kirkpatrick's level 4. Not measuring the effectiveness of our training programs means that we may be wasting people's time, the company's money, and hampering the organization's ability to meet it's objectives. We're also not taking our responsibility as "stewards of human creativity" (Chewning, et al., 1990) seriously.