Continuing our exploration of processes and tools that help instructional designers (ID) make better data-driven decisions, in this blog, we focus on program evaluation, the practice of answering questions about programs, policies, or projects through systematic, evidence-based processes.
Program Evaluation
Although instructional design practitioners usually deal with programs related to training and learning, program evaluation can also encompass large instructional and non-instructional programs, sometimes significantly more extensive than straightforward training programs. Some examples include academic curricula, grant-funded projects, large organizational incentives, business ventures, or entire departments or organizations. Program evaluations can get big rather quickly. Many consider the art and science of program evaluation a profession unto itself, as the expertise required to conduct large-scale or very tedious evaluation can take years of experience and specialized training to acquire.
Kirkpatrick’s Four Levels of Training Evaluation
Therefore, most program evaluations you will perform in your instructional design career are smaller in scope and do not require complex evaluation frameworks. One framework that caters to the ID profession and can be easily implemented for program evaluation is Kirkpatrick’s Four Levels of Training Evaluation. Kirkpatrick is the de facto industry standard for training evaluation, and you will come across it in most workplaces staffed with training and learning professionals.
Evaluation processes exist so that we may show thoughtful engagement around questions concerning the following issues:
- Making improvements to existing programs or initiatives
- Maximizing the transfer of learning toward behavior and therefore subsequent organizational results
- Demonstrating the value of learning initiatives within organizations
An ID’s mantra should always be “program improvement.” Gone are the days when practitioners developed training programs and let them sit unchanged for years. Constant, incremental changes and updates are the reality and a significant consideration of instructional development.
To maximize the transfer of learning to behavior and subsequent organizational results is equally important. We should measure the impact our learning programs have on the organization and the degree to which the instructional needs addressed in our training transfer to the organization’s workflow or culture.
Additionally, as in any other department, instructional designers need to be able to demonstrate their value to organizations. They are not exempt from proving their worth through thoughtful, earnest analysis and communication of results.
Four Levels of Evaluation
Kirkpatrick’s model offers four levels of evaluation: results, behavior, learning, and reaction. These four levels provide multi-dimensional feedback to help you make viable decisions and changes to your learning programs.
One sometimes neglected property of Kirkpatrick’s model is that instructional designers should use aspects of the model while developing a training program, not merely after they have delivered it. This means that IDs and trainers should integrate it along the lifespan of the development and deployment process to ensure the proper flow and consistent collection of data. Conducting a program evaluation without the benefit of building a plan and process for it after the fact is possible, but it is much more manageable when it is a natural part of the design process. Moreover, Kirkpatrick’s module is best used within this “build as you go” context.
In summary, program evaluation is an essential instructional design skill where IDs and organizations seek input on instructional development and demonstrate they are making a needed difference.
Interested in a career that improves learning outcomes across a variety of institutions and industries?
Eastern Kentucky University’s online Master of Science in Instructional Design and Learning Technology prepares competent, capable creators of learning solutions across the professional landscape. The degree provides relevant, real-world course content, experienced faculty, practical experiences, and portfolio preparation to showcase your skills and abilities to hiring decision-makers in the 21st-century workplace. Come learn with us!
Kirkpatrick, W. K., Kirkpatrick, J. D. (2016). Kirkpatrick’s Four Levels of Training Evaluation. United States: ATD Press.