Abstract
Assessment of educational effectiveness provides vitally important feedback to Institutions of Higher Education. It also provides important information to external stakeholders, such as prospective students, parents, governmental and local regulatory entities, professional and regional accrediting organizations, and representatives of the workforce. However, selecting appropriate indicators of educational effectiveness of programs and institutions is a difficult task, especially when criteria of effectiveness are not well defined. This article proposes a comprehensive and systematic approach to aligning criteria for educational effectiveness with specific indicators of achievement of these criteria by adapting a popular organizational training evaluation framework, the Kirkpatrick’s four level model of training criteria (Kirkpatrick 1959; 1976; 1996), to assessment in Higher Education. The four level model consists of reaction, learning, behavior and results criteria. Adaptation of this model to Higher Education helps to clarify the criteria and create plans for assessment of educational outcomes in which specific instruments and indicators are linked to corresponding criteria. This provides a rich context for understanding the role of various indicators in the overall mosaic of assessment. It also provides Institutions of Higher Education rich and multilevel feedback regarding the effectiveness of their effort to serve their multiple stakeholders. The importance of such feedback is contextualized both in the reality of stakeholder pressures and in theoretical understanding of colleges and universities as open systems according to the systems theory (Katz and Kahn 1966). Although the focus of this article is on Higher Education, core principles and ideas will be applicable to different types and levels of educational programs.
This is a preview of subscription content, log in via an institution to check access.
References
Allen, M. J. (2006). Assessing general education programs (2006). Bolton: Anker.
Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of relations among training criteria. Personnel Psychology, 50, 341–358.
Arthur, W., Jr., Bennett, W. J., Edens, P. S., & Bell, S. T. (2003a). Effectiveness of training in organizations: a meta-analysis of design and evaluation features. The Journal of Applied Psychology, 88, 234–245.
Arthur, W., Jr., Tubre, T. C., Paul, D. S., & Edens, P. S. (2003b). Teaching effectiveness: the relationship between reaction and learning criteria. Educational Psychology, 23, 275–285.
Bers, T. H. (2008). The role of institutional assessment in assessing student learning outcomes. New Directions for Higher Education, 141, 31–39. doi:10.1002/he.291.
Biesta, G. (2009). Good education in an age of measurement: on the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability, 21, 33–46. doi:10.1007/s11092-008-9064-9.
Boyer, E. L., & Hechinger, F. M. (1981). Higher learning in the nation’s service. New York: Carnegie Foundation for the Advancement of Teaching.
Brittingham, B., O’Brien, P. M., & Alig, J. (2008). Accreditation and institutional research: the traditional role and new dimensions. New Directions for Higher Education, 141, 69–76. doi:10.1002/he.294.
Colby, A., Ehrlich, T., Beaumont, E., & Stephens, J. (2003). Educating citizens: Preparing America’s undergraduates for lives of moral and civic responsibility. San Francisco: Jossey-Bass.
Dalton, J. C., Russell, T. R., & Kline, S. (2004). Editors’ notes. New Directions for Institutional Research, 122, p1–p2.
De Roulet, D., Praslova, L., Carmody, T., Roberson, M., & Wilson, M. (2009). Assessing core curriculum. Working document, Vanguard University of Southern California.
Dysvik, A., & Martinsen, Ø. L. (2008). The relationship between trainees’ evaluation of teaching and trainee performance among Norwegian executive students. Educational Psychology, 28, 747–756. doi:10.1080/01443410802259253.
Ewell, P. T. (2001). Accreditation and student learning outcomes: A proposed point of departure. Council for Higher Education Accreditation, Washington, DC.
Ewell, P. T. (2006). Making the grade: How boards can ensure academic quality. Washington: Association of Governing Boards of Colleges and Universities.
Halpern, D. F., & Hakel, M. D. (2003). Applying the science of learning to the university and beyond: teaching for long-term retention and transfer. Change, July/August, 2–13.
Hansen, J. B. (1994). Applying systems theory to systemic change: a generic model for educational reform. Paper presented the Annual Meeting of the American Educational Research Association, New Orleans, LA, April 1994.
Karpenko, O., Bershadskaia, M., & Voznesenskaia, I. (2009). The PISA international study and problems of the development of Higher Education. Russian Education & Society; 51(4), 68–91. doi:10.2753/RES1060-9393510404.
Katz, D., & Kahn, R. L. (1966). The social psychology of organizations. New York: Wiley.
Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors, 13, 3–9.
Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training and development handbook: A guide to human resource development (2nd ed., pp. 301–319). New York: McGraw-Hill.
Kirkpatrick, D. L. (1996). Invited reaction: reaction to Holton article. Human Resource Development Quarterly, 7, 23–25.
Landy, F. J., & Conte, J. M. (2007). Work in the 21 century. An introduction to industrial and organizational psychology (2nd ed.). Malden: Blackwell.
Russell, T. R. (2004). Assessing outcomes of character-building programs: problems and prospects. New Directions for Institutional Research, 122, p105–p110. doi:10.1002/ir.113.
Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: a decade of progress. Annual Review of Psychology, 52, 471–499.
Sax, L. J. (2004). Citizenship development and the American college student. New Directions for Institutional Research, 122, 65–80. doi:10.1002/ir.110.
Toutkoushian, R. K. (2005). What can institutional research do to help colleges meet the workforce needs of states and nations? Research in Higher Education, 46, 955–984. doi:10.1007/s11162-005-6935-5.
Van Buren, M. E., & Erskine, W. (2002). The 2002 ASTD state of the industry report. Alexandria: American Society of Training and Development.
Voorhees, R. A., & Harvey, L. (2005). Higher education and workforce development: a strategic role for institutional research. New Directions for Institutional Research, 128, 5–12.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Praslova, L. Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education. Educ Asse Eval Acc 22, 215–225 (2010). https://doi.org/10.1007/s11092-010-9098-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11092-010-9098-7