ProgressIt ® --Supporting Your Inner Coach
and the Kirkpatrick Evaluation Model

Is your investment in development and learning worth it? Generations of trainers, researchers, academics and even accountants have wrestled with this challenge, across a wide range of industries. Consequently, we are rich in models and methodologies claiming to produce the answer, some more convincing than others. Even so, over a quarter of (27%) of all UK companies attempt no real evaluation of their development efforts.

Despite the variety of approach now on offer, the original Donald Kirkpatrick four level model of evaluation presented in 1975 remains a classic and still widely used:

	Level One: Reaction
	Level Two: Learning
	Level Three: Behaviour
	Level Four: Results

We can apply these levels to technology-based training and more traditional forms of delivery. However, while Kirkpatrick's goal was to create a hierarchical model spanning many uses, he also argued that no single formula could produce an effectiveness rating. Especially at the higher levels of evaluation, it is probably not possible to isolate causes of effectiveness. What we now know is that to obtain each level of assessment incurs increasing trade offs in terms of expense and complexity.

In most organisations, it is relatively easy to obtain information about the first two levels. Though whether it is worth actually testing for new learning depends on the gains expected from the development. However, the more difficult levels 3 and 4 seldom receive much attention, and it is here that MLA tends to focus its energies.

Behaviour change

Maynard Leigh Associates helps organisations achieve changes in behaviour, results that require Level 3 evaluation. Based on our experience of the last 15 years, few clients have the necessary funds or time to obtain scientific proof of behavioural change. Instead, they more sensibly opt for plausibility. That is, they rely on multiple sources of information that combined tend to confirm or deny the existence of the desired behavioural changes.

These multiple sources include what learners themselves say, line manager responses, critical incidents, assessment centres, external feedback such as comments from clients, sales data, regular assessments, and so on. Since we can expect to actually observe new behaviour this level of evaluation can be relatively easy to obtain.

Level 4 usually implies that the new behaviour produced by the development investment actually adds value to the organisation in respect of business results. A chief executive for example, probably does not care if a sales person can now present more effectively, but does care if increased effectiveness actually leads to winning more business. Demonstrating this remains difficult, but possible at a price. As ever, there is a trade-off between obtaining the necessary information, and costs, and the benefits from knowing the answer.

Where ProgressIt ® fits

ProgressIt ® is our new on-line support service that provides learners with up to nine weeks follow-through support and reminders. It also tracks progress in reaching agreed behavioural goals and seeks information on lessons learned when the person reaches a goal.

We have introduced ProgressIt ® partly in response to the problems our clients meet in taking evaluation beyond levels one and two. It provides previously unobtainable information about what people actually do because of their learning experience. Each person taking part in the ProgressIt ® service makes a commitment at the end of the learning event to achieving up to three behavioural goals. We encourage them to choose SMART goals, that are Stretching, Achievable, Measurable, Recorded and Time limited. Once they state their goals, ProgressIt ® asks them for regular updates on what they have done towards achieving these aims.

Here is an extract from a participant's report, using ProgressIt ®. This person recently learned how to coach and works in the top team of a major UK drinks company. We have removed information that would breach confidentiality:

Goal 2: To understand the career aspirations of one of my Senior Managers - what they would like to do/when. Also, build a plan to help them achieve their next move. PROGRESS SO FAR: Major ACTIONS TAKEN: At 1-2-1 meeting used coaching techniques to lead the Manager into opening up about their thoughts on opportunities for them within the company and where they would like to be in 12 months time. We agreed to have a follow-up session over the next month. Submitted on 07 03 2003 Had less formal discussions as a follow up to the formal 1-2-1. The Manager is beginning to consider her options. Submitted on 04 04 2003 What's your best chance to make some progress next week: None. The Manager needs a few weeks to formulate views on her future. From now on, I make myself available when the manager wishes to discuss the issue.

This kind of information accumulates over nine weeks to produce a substantial Business Benefits Report. It generates new evidence about whether and how learners apply newly acquired skills.

Amongst other evidence presented in the report are facts on what each person has learned from achieving their goals.

Conclusion

Line managers traditionally complain that when their direct reports attend development events they seldom know how their people subsequently apply their new learning. ProgressIt ® offers them direct access to such information.

While ProgressIt ® may never entirely provide entirely comprehensive information about Level 4 evaluation, the business results, it does make an important contribution to this issue.

HR departments for example, enriched by a substantial amount of credible behavioural data from the Business Benefits Report, should now be in a much better position to interpret the results and consequently to produce more credible evidence on whether the development investment been worth it.

For more information on ProgressIt ®, go to www.maynardleighonline.co.uk/progressit_v2