Skip to main content

Advertisement

SpringerLink
Log in

Trends and Developments in Program Evaluation in General and Criminal Justice Programs in Particular

  • Published:
European Journal on Criminal Policy and Research Aims and scope Submit manuscript

Abstract

The paper first describes general trends in evaluation in (mainly) western societies. Why is evaluation growing, what are the characteristics of this ‘growth industry’ and what developments are occurring outside western societies? Trends in the evaluation of criminal justice programs in the USA, the UK and the Netherlands are then discussed. Two important developments are therafter highlighted: experimental evaluations and theory-driven evaluations. Both approaches are discussed, and some pros and cons are listed. Finally, the paper outlines some challenges for future work in the evaluation of criminal justice programs, stressing the importance of combining good designs with both program and social science theories.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bennet, T., The effectiveness of a police-initiated fear-reducing strategy. British Journal of Criminology, 31(1), pp. 1–14, 1991.

    Google Scholar 

  • Boruch, R., P. Davies, H. Soydan, I. Chalmers, H. Cooper, A. Petrosino, D. DeMoya, D. Myers, A. Oxman, G. Macdonald, L. Hedges, R. Harden, R. Lilley et al., Progress on Developing the Campbell Collaboration: July 1999 and February 2000. Philadelphia: Center for Research & Evaluation in Social Policy, University of Pennsylvania, 2000.

  • Campbell, D.T. and J.C. Stanley, Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally, 1963.

    Google Scholar 

  • Chelimsky, E., The politics of program evaluation. Social Science and Modern Society, 25(24–32), 1987.

    Google Scholar 

  • Cook, T.D., Why have educational evaluators chosen not to do randomized experiments? The Annals of the American Academy of Political and Social Science, 589, pp. 114–148, 2003.

  • Cook, T.D. and C.S. Reichardt (Eds.), Qualitative and Quantitative Methods in Evaluation Research. Beverly Hills, CA: Sage, 1979.

    Google Scholar 

  • Cooksy, L. and F.L. Leeuw, Evaluating the performance of development agencies: The role of meta-evaluations. In: G.K. Pitman, O.N. Feinstein and G.K. Ingram (Eds.), Evaluating development effectiveness, World Bank Series on Evaluation and Development, pp. 95–108. New Brunswick: Transaction Publishers, 2004.

  • Derlien, H.U., Genesis and structure of evaluation efforts in comparative perspective. In: R.C. Rist (Ed.), Program evaluation and the management of government, Patterns and prospects across eight nations, pp. 147–177. New Brunswick, NJ: Transaction Publishers, 1989.

  • Dickens, B., Contribution to the Symposium of Learning What Works: Evaluating Complex Social Interventions, Washington, DC: The Brookings Institution, 1997.

  • Duflo, E. and M. Kramer, Use of randomization in the evaluation of development effectiveness. In: G.K. Pitman, O.N. Feinstein and G.K. Ingram (Eds.), Evaluating development effectiveness, World Bank Series on Evaluation and Development, pp. 205–232. New Brunswick, NJ: Transaction Publishers, 2004.

  • Eckblom, P. and K. Pease, Evaluation crime prevention. In: M.H. Tonry and M.H. Farrington (Eds.), Building a Safer Society: Strategic Approaches to Crime Prevention. Chicago: University of Chicago Press, 1995.

    Google Scholar 

  • Farrington, D.P., Methodological quality standards for evaluation. The Annals of the American Academy of Political and Social Science Studies, 587, pp. 49–68, 2003.

  • Farrington, D.P. and B.C. Welsh, Home Office Research Study 251: Effects of improved street lighting on crime: A systematic review. London: Home Office, Development and Statistics Directorate, 2002.

    Google Scholar 

  • Freeman, H.E., The present status of evaluation research. In: M.A. Guttenberg and S. Saar (Eds.), Evaluation Studies Review Annual, 2, pp. 17–51, 1977.

  • Furubo, J.E., R.C. Rist and R. Sandahl (Eds.), The International Atlas of Evaluation. New Brunswick, NJ: Transaction Publishers, 2002a.

    Google Scholar 

  • Furubo, J.E. and R. Sandahl, Introduction: a diffusion perspective on global developments in evaluation. In: J.E. Furubo, R.C. Rist and R. Sandahl (Eds.), The International Atlas of Evaluation, pp. 1–27. New Brunswick, NJ: Transaction Publishers, 2002b.

  • Gill, M. and A. Spriggs, Assessing the impact of CCTV. London: Home Office, Development and Statistics Directorate, 2005.

    Google Scholar 

  • Goldblatt, P. and C. Lewis, Home Office Research Study 187: Reducing offending: An assessment of research evidence on ways of dealing with offending behaviour. London: Home Office, Development and Statistics Directorate, 1998.

    Google Scholar 

  • Gray, A., B. Jenkins and F.L. Leeuw (Eds.), Collaboration in public services: the challenge for evaluation. New Brunswick, NJ: Transaction Publishers, 2003.

    Google Scholar 

  • Hedstrom, P. and R. Swedberg (Eds.), Social Mechanisms. An Analytical Approach to Social Theory. Cambridge, UK: Cambridge University Press, 1998.

    Google Scholar 

  • Hovland, C.I., et al., Experiments in Mass communication. Vol. III: The American Soldier. Princeton, NJ: Princeton University Press, 1949.

    Google Scholar 

  • Johnson, S.D., K.J. Bowers, P. Jordan, N. Davidson and A. Hirschfield, Estimating crime reduction outcomes: how many crimes were prevented? Evaluation 10(3), pp. 327–348, 2004.

    Article  Google Scholar 

  • Klein Haarhuis, C., M. van Ooyen-Houben, E.R. Kleemans and F.L. Leeuw, Kennis voor beleid; een synthese van 58 (evaluatie)onderzoeken op het gebied van rechtshandhaving, The Hague: WODC, 2005.

  • Latour, B. and S. Woolgar, Laboratory Life. Beverly Hills, CA: Sage, 1979.

    Google Scholar 

  • Leeuw, F.L., Policy theories, knowledge utilization, and evaluation. Knowledge and Policy, 4, pp. 73–92, 1991.

  • Leeuw, F.L., Unintended side effects of auditing: The relationship between performance auditing and performance improvement and the role of trust. In: J. Weesie and W. Raub (Eds.), The management of durable relations, pp. 95–97. Amsterdam: Thela, 2000.

  • Leeuw, F.L., Reconstructing program theories: Methods available and problems to be solved. American Journal of Evaluation, 24(1), pp. 5–20, 2003.

    Google Scholar 

  • Leeuw, F.L., Managing evaluations in the Netherlands and types of knowledge. In: R. Rist and N. Stame (Eds.), From Studies to Streams. New Brunswick, NJ: Transaction Publishers, in press.

  • Leeuw, F.L., R.C. Rist and R.C. Sonnichsen (Eds.), Can governments learn: comparative perspectives on evaluation and organizational learning. New Brunswick, NJ: Transaction Publishers, 1994.

    Google Scholar 

  • Leeuw, F.L. and J. Toulemonde, Evaluation activities in Europe: A quick scan of the market in 1998. Evaluation, 5, pp. 487–496, 1999.

    Article  Google Scholar 

  • Leeuw, F.L. and S. Van Thiel, The performance paradox in the public sector. Public Productivity and Management Review, 25, pp. 123–143, 2002.

    Google Scholar 

  • Mayne, J. and R. Schwartz (Eds.), Quality Matters. New Brunswick, NJ: Transaction Publishers, 2004.

    Google Scholar 

  • Muir, E., They blinded me with political science: On the use of on-peer reviewed research in education policy. Political Science and Politics, 32(4), pp. 762–764, 1999.

    Article  Google Scholar 

  • Nuttal, C., The home office and random allocation experiments. Evaluation Review, 27(3), pp. 267–290, 2003.

    Article  PubMed  Google Scholar 

  • Oakley, A., Experiments in knowing. London: Polity Press, 2000.

    Google Scholar 

  • Orr, L.L., Social Experiments: Evaluating Public Programs with Experimental Methods. Thousand Oaks, CA: Sage, 1999.

    Google Scholar 

  • Pawson, R., Does Megan's Law Work? A theory-driven systematic review, London: Queen Mary University of London, Working Paper 8, ESRC UK Centre for Evidence Based Policy and Practice, 2002a.

  • Pawson, R., Evidence-based policy: The promise of ‘realist synthesis'. Evaluation, 8(3), pp. 340–358, 2002b.

    Article  Google Scholar 

  • Pawson, R., Simple principles for the evaluation of complex programmes. Working Paper, Leeds: Leeds University, 2004.

  • Pawson, R. and N. Tilley, What works in evaluation research. British Journal of Criminology, 34(3), pp. 291–307, 1994.

    Google Scholar 

  • Pawson, R. and N. Tilley, Realistic evaluation. London: Sage, 1997.

    Google Scholar 

  • Pease, K., A review of street lighting evaluations: Crime reductions effects. Crime Prevention Studies, 10, pp. 47–76, 1999.

    Google Scholar 

  • Petrosino, A., et al., A Proposal for a Campbell Criminal Justice Group Paper developed for the February 2000 meeting of the Campbell Collaboration Draft 3, February 8th 2000. Campbell Collaboration, 2000.

  • Picciotto, R., Evaluation in the World Bank: antecedents, methods, and instruments. In: J. E. Furubo, R. C. Rist and R. Sandahl (Eds.), The International Atlas of Evaluation, pp. 425–439. New Brunswick, NJ: Transaction Publishers, 2002.

  • Power, M., The Audit Society. London: Penguin, 1997.

    Google Scholar 

  • Power, M., The Risk Management of Everything. Rethinking the Politics of Uncertainty. London: Demos, 2004.

    Google Scholar 

  • Powers, E. and H. Witmer, An experiment in the prevention of juvenile delinquency: The Cambridge-Somerville Youth Study. New York: Columbia University Press, 1951.

    Google Scholar 

  • Rist, R.C. (Ed.), Program evaluation and the management of government. Patterns and prospects across eight nations. New Brunswick, NJ: Transaction Publishers, 1989.

    Google Scholar 

  • Rist, R. and K.L. Paliokas, The rise and fall (and rise again?) of the evaluation function in the U.S. Government. In: J.E. Furubo, R.C. Rist and R. Sandahl (Eds.), International Atlas of Evaluation. New Brunswick, NJ: Transaction Publishers, 2002.

    Google Scholar 

  • Rist, R. and N. Stame (Eds.), From Studies to Streams. New Brunswick, NJ: Transaction Publishers, in press.

  • Rossi, P.H., H.E. Freeman and M.W. Lipsey, Evaluation. A Systematic Approach. Thousand Oaks: Sage Publication, 1999.

    Google Scholar 

  • Scriven, M., Goal-free evaluation. In: E.R. House (Ed.), School Evaluation: The Politics and Processes (pp. 319–328). Berkeley, CA: McCutchan, 1973.

    Google Scholar 

  • Scriven, M., The Logic of Evaluatin. Inverness, CA: Edgepress, 1980.

    Google Scholar 

  • Shadish, W.R., Program Evaluation: A Pluralistic Enterprise. San Francisco: Jossey Bass, 1993.

    Google Scholar 

  • Shadish, W.R., T.D. Cook and L.C. Leviton, Foundations of program evaluation, theories of practice. London: Sage, 1995.

    Google Scholar 

  • Sherman, L.W., D.P. Farrington, B.C. Welsh and D. Layton Mackenzie (Eds.), Evidence-based crime prevention. Routledge: London, 2002.

    Google Scholar 

  • Sherman, L.W., D. Gottfredson, D. Mackenzie, J. Eck, P. Reuter and S. Bushway. Preventing Crime: What Works, What Doesn't, What's Promising, a Report to the United States Congress. Prepared for the National Institute of Justice, Department of Criminology and Criminal Justice, University of Maryland, 1997.

  • Stame, N., Theory-based evaluation and types of complexity. Evaluation, 10(1), pp. 58–76, 2004.

    Article  Google Scholar 

  • Stame, N., Streams of evaluative knowledge: Introduction. In: R. Rist and N. Stame (Eds.), From studies to streams, pp. 1–23. New Brunswick, NJ: Transaction Publishers, in press.

  • Summa, H. and J. Toulemonde, Evaluation in the EU: Addressing complexity and ambiguity. In: J.E. Furubo, R.C. Rist and R. Sandahl (Eds.), The International Atlas of Evaluation, pp. 407–425. New Brunswick, NJ: Transaction Publishers, 2002.

    Google Scholar 

  • The Economist, Try it and see, 28 February 2002.

  • Tilley, N., Applying theory-driven evaluation to the British Crime Reduction Programme. Criminal Justice, 4(3), pp. 255–276, 2004.

    Article  Google Scholar 

  • US GAO, Prospective Evaluation Methods: The prospective evaluation synthesis. Washington: US General Accounting Office, 1990.

    Google Scholar 

  • US GAO, Teenage pregnancy: 500,000 births a year but few tested programs. Washington: US General Accounting Office, 1996.

    Google Scholar 

  • Vischer, C.A. and D. Weisburd, Identifying what works: Recent trends in crime prevention strategies. Crime, Law and Social Change, 28, pp. 223–242, 1998.

    Google Scholar 

  • Wartna, B., N.J. Baas and E. Beenakkers, Beter, anders en goedkoper: een literatuurverkenning ten behoeve van het traject Modernisering Sanctietoepassing. Memorandum 2004-1. The Hague: WODC, 2004.

  • Weeks, H.A., Youthful offenders at Highfields. Ann Arbor, MI.: University of Michigan Press, 1958.

    Google Scholar 

  • Weiss, C.H., Where politics and evaluation research meet. Evaluation, 1, pp. 37–45, 1973.

    Google Scholar 

  • Weisburd, D. and J.E. Eck, What can police do to reduce crime, disorder, and fear? The Annals of the American Academy of Political and Social Science, 593, pp. 42–65, 2004.

    Article  Google Scholar 

  • Wholey, J.S., Evaluability assessment: developing program theory. New Directions for Program Evaluation, 33, pp. 77–92, 1987.

    Article  Google Scholar 

  • Wildavsky, A., The self-evaluating organization. Public Administration Review, 32, pp. 509–520, 1972.

    Article  Google Scholar 

  • Wittebrood, K. and M. Van Beem, Sociale veiligheid vergroten door gelegenheidsbeperking: wat werkt en wat niet. Raad voor de Maatschappelijke Ontwikkeling, bijlage 6, pp. 271–368, 2004.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. WODC, The Hague & University of Utrecht The Netherlands, Heemsteedse Dreef 134, 2102 KR, Heemstede, The Netherlands

    Frans L. Leeuw

Authors
  1. Frans L. Leeuw
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frans L. Leeuw.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Leeuw, F.L. Trends and Developments in Program Evaluation in General and Criminal Justice Programs in Particular. Eur J Crim Policy Res 11, 233–258 (2005). https://doi.org/10.1007/s10610-005-0835-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10610-005-0835-2

Keywords

Search

Navigation