Skip to main content
SpringerLink
Log in

Measuring Fidelity

  • Chapter
  • First Online:
Defining Prevention Science

Part of the book series: Advances in Prevention Science ((Adv. Prevention Science))

Abstract

There is evidence that fidelity moderates program outcomes. With the widespread dissemination of evidence-based programs, a major challenge is to assess the fidelity with which interventions are delivered. Fidelity is a multidimensional construct. Knowing which features of fidelity moderate intervention effectiveness is important to understand how programs achieve or fail to achieve their goals. A key issue, therefore, is the availability of tools for assessing fidelity that can be readily and reliably applied. It will be helpful to advance our understanding of program effectiveness to have methods that can be applied in similar ways across multiple interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Abbott, R. D., O’Donnell, J., Hawkins, J. D., Hill, K. G., Kosterman, R., & Catalano, R. F. (1998). Changing teaching practices to promote achievement and bonding to school. American Journal of Orthopsychiatry, 68, 542–552.

    Article  CAS  PubMed  Google Scholar 

  • Adams, A. S., Soumerai, S. B., Lomas, J., & Ross-Degnan, D. (1999). Evidence of self-report bias in assessing adherence to guidelines. International Journal of Quality Health Care, 11, 187–192.

    Article  CAS  Google Scholar 

  • Baranowski, T., Anderson, C., & Carmack, C. (1998). Mediating variable framework in physical activity interventions: How are we doing? How might we do better? American Journal of Preventive Medicine, 15, 266–297.

    Article  CAS  PubMed  Google Scholar 

  • Baranowski, T., Lin, L. S., Wetter, D. W., Resnicow, K., & Hearn, M. D. (1997). Theory as mediating variables: Why aren’t community interventions working as desired? Annals of Epidemiology, 7, S89–S95.

    Article  Google Scholar 

  • Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.

    Article  CAS  PubMed  Google Scholar 

  • Basen-Enquist, K., O'Hara-Tompkins, N., Lovato, C. Y., Lewis, M. J., Parcel, G. S., & Gingiss, P. (1994). The effect of two types of teacher training on implementation of Smart Choices: A tobacco prevention curriculum. Journal of School Health, 64, 334–339.

    Article  Google Scholar 

  • Becker, D. R., Bond, G. R., McCarthy, D., Thompson, D., Xie, H., McHugo, G. J., et al. (2001). Converting day treatment centers to supported employment programs in Rhode Island. Psychiatric Services, 52(3), 351–357.

    Article  CAS  PubMed  Google Scholar 

  • Bellg, A., Borrelli, B., Resnick, B., Ogedegbe, G., Hecht, J., Ernst, D., et al. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the Behavioral Change Consortium. Health Psychology, 23(5), 443–451.

    Article  PubMed  Google Scholar 

  • Biggs, B., Vernberg, E., Twemlow, S., Fonagy, P., & Dill, E. (2008). Teacher adherence and its relation to teacher attitudes and student outcomes in an elementary school-based violence prevention program. School Psychology Review, 37(4), 533–549.

    Google Scholar 

  • Bishop, D. C., Pankratz, M. M., Hansen, W. B., Albritton, J., Albritton, L., & Strack, J. (2013). Measuring fidelity and adaptation: Reliability of a instrument for school-based prevention programs. Evaluation and the Health Professions, doi: 10.1177/0163278713476882

  • Blakely, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W., Roitman, D. B., et al. (1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253–268.

    Article  Google Scholar 

  • Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H. K. (2000). Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research, 2(2), 75–87.

    Article  CAS  PubMed  Google Scholar 

  • Boruch, R. R., & Gomez, H. (1977). Sensitivity, bias, and theory in impact evaluation. Professional Psychology, 8, 411–433.

    Article  Google Scholar 

  • Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273, 1106–1112.

    Article  CAS  PubMed  Google Scholar 

  • Botvin, G. J., Baker, E., Dusenbury, L., Tortu, S., & Botvin, E. M. (1990). Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: Results of a 3-year study. Journal of Consulting and Clinical Psychology, 58, 437–446.

    Article  CAS  PubMed  Google Scholar 

  • Botvin, G. J., Baker, E., Filazzola, A. D., & Botvin, E. M. (1990). A cognitive-behavioral approach to substance abuse prevention: One-year follow-up. Addictive Behaviors, 15, 47–63.

    Article  CAS  PubMed  Google Scholar 

  • Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., & Kerner, J. (1989). A skills training approach to smoking prevention among Hispanic youth. Journal of Behavioral Medicine, 12, 279–296.

    Article  CAS  PubMed  Google Scholar 

  • Breitenstein, S. M., Fogg, L., Garvey, C., Hill, C., Resnick, B., & Gross, D. (2010). Measuring implementation fidelity in a community-based parenting intervention. Nursing Research, 59(3), 158–165.

    Article  PubMed Central  PubMed  Google Scholar 

  • Dane, A., & Schneider, B. H. (1998). Integrity in primary prevention programs: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.

    Article  CAS  PubMed  Google Scholar 

  • Dent, C. W., Sussman, W., Hennesy, M., Galaif, E. R., Stacy, A. W., Moss, M., et al. (1998). Implementation and process evaluation of a school-based drug abuse prevention program: Project Towards no Drug Abuse. Journal of Drug Education, 28(4), 361–375.

    Article  CAS  PubMed  Google Scholar 

  • Donaldson, S. I. (2001). Mediator and moderator analysis in program development. In S. Sussman (Ed.), Handbook of program development for health behavior research and practice (pp. 470–496). Thousand Oaks, CA: Sage.

    Chapter  Google Scholar 

  • Donaldson, S. I., Graham, J. W., & Hansen, W. B. (1994). Testing the generalizability of intervening mechanism theories: Understanding the effects of adolescent drug use prevention interventions. Journal of Behavioral Medicine, 17(2), 195–216.

    Article  CAS  PubMed  Google Scholar 

  • Donaldson, S. I., & Grant-Vallone, E. J. (2002). Understanding self-report bias in organizational behavior research. Journal of Business and Psychology, 17(2), 245–262.

    Article  Google Scholar 

  • Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting the implementation. American Journal of Community Psychology, 41, 327–350.

    Article  PubMed  Google Scholar 

  • Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.

    Article  PubMed  Google Scholar 

  • Dusenbury, L., Brannigan, R., Hansen, W. B., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20, 308–313.

    Article  PubMed  Google Scholar 

  • ebp.networkofcare.org. (2012). Introduction to the evidence-based practice fidelity scales. http://ebp.networkofcare.org/uploads/Fidelity_Scales_4673293.pdf. See also http://ebp.networkofcare.org/definitions/index.cfm?pageName=DefsFidelity

  • Ennett, S. T., Haws, S., Ringwalt, C. L., Vincus, A. A., Hanley, S., Bowling, J. M., et al. (2012). Evidence-based practice in school substance use prevention: Fidelity of implementation under real-world conditions. Health Education Research, 26, 361–371.

    Article  Google Scholar 

  • Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Bridging science to practice: Achieving prevention program implementation fidelity in the community youth development study. American Journal of Community Psychology, 41, 235–249.

    Article  PubMed  Google Scholar 

  • Fagan, A., & Mihalic, S. (2003). Strategies for enhancing the adoption of school-based prevention programs: Lessons learned from the Blueprints for Violence Prevention replications of the LifeSkills Training Program. Journal of Community Psychology, 31, 235–253.

    Article  Google Scholar 

  • Giles, S. M., Harrington, N. G., & Fearnow-Kenney, N. (2001). Evaluation of the All Stars Program: Student and teacher factors that influence mediators of substance use. Journal of Drug Education, 31(4), 385–397.

    Article  CAS  PubMed  Google Scholar 

  • Giles, S. M., Jackson-Newsom, J., Pankratz, M. M., Hansen, W. B., Ringwalt, C. L., & Dusenbury, L. (2008). Measuring quality of delivery in a substance use prevention program. Journal of Primary Prevention, 29, 489–501.

    Article  PubMed  Google Scholar 

  • Gottfredson, D. C., Cross, A., Wilson, D., Rorie, M., & Connell, N. (2009). An experimental evaluation of the All Stars prevention curriculum in a community after school setting. Prevention Science, 11, 142–154.

    Article  Google Scholar 

  • Greenberg, M. T., Kusche, C. A., Cook, E. T., & Quamma, J. P. (1995). Promoting emotional competence in school-aged children: The effects of the PATHS curriculum. Development and Psychopathology, 7, 117–136.

    Article  Google Scholar 

  • Hall, G. E., & Loucks, S. F. (1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 14, 263–276.

    Article  Google Scholar 

  • Hanley, S., Ringwalt, C., Vincus, A., Ennett, S., Bowling, J., Haws, S., et al. (2009). Implementing evidence-based substance use prevention curricula with fidelity: The role of teacher training. Journal of Drug Education, 39(1), 39–58.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hansen, W. B. (1996). Pilot test results comparing the All Stars program with seventh grade DARE: Program integrity and mediating variable analysis. Substance Use & Misuse, 31, 1359–1377.

    Article  CAS  Google Scholar 

  • Hansen, W. B., Bishop, D. C., Dusenbury, L., Pankratz, M. M., Albritton, J., Albritton, L. P., et al. (2011). Feeding information learned from adaptations back into program design. Presentation made at the 20th Annual Meeting, Society for Prevention Research, Washington, DC. Retrieved from http://www.tanglewood.net/powerpoint/Hansen_SPR_2011.ppt

  • Hansen, W. B., Graham, J. W., Wolkenstein, B. H., Lundy, B. Z., Pearson, J. L., Flay, B. R., et al. (1988). Differential impact of three alcohol prevention curricula on hypothesized mediating variables. Journal of Drug Education, 18, 143–153.

    Article  CAS  PubMed  Google Scholar 

  • Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Program integrity as a moderator of prevention program effectiveness: Results for fifth grade students in the Adolescent Alcohol Prevention Trial. Journal of Studies on Alcohol, 52, 568–579.

    CAS  PubMed  Google Scholar 

  • Hansen, W. B., & McNeal, R. B. (1996). The law of maximum expected potential effect: Constraints placed on program effectiveness by mediator relationships. Health Education Research, 11, 501–507.

    Article  Google Scholar 

  • Hansen, W. B., & McNeal, R. B. (1997). How DARE works: An examination of program effects on mediating variables. Health Education & Behavior, 24, 165–176.

    Article  CAS  Google Scholar 

  • Hansen, W. B., Pankratz, M. M., & Bishop, D. C. (under review). Differences in observers' and teachers' assessments of fidelity and adaptation. Health Education Research.

    Google Scholar 

  • Hansen, W. B., Pankratz, M. M., Dusenbury, L., Giles, S. M., Bishop, D. C., Albritton, J., Albritton, L. P., & Strack, J (2013) Styles of adaptation: The impact of frequency and valence of adaptation on preventing substance use. Health Education 113(4):345–363.

    Article  Google Scholar 

  • Harachi, T. W., Abbott, R. D., Catalano, R. F., Haggerty, K. P., & Fleming, C. B. (1999). Opening the black box: Using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology, 27, 715–735.

    Article  Google Scholar 

  • Hill, L. G., Maucione, K., & Hood, B. (2007). A focused approach to assessing program fidelity. Prevention Science, 8(1), 25–34.

    Article  PubMed  Google Scholar 

  • Johnson, K., Ogilvie, K., Collins, D., Shamblen, S., Dirks, L., Ringwalt, C., et al. (2010). Studying implementation quality of a school-based prevention curriculum in frontier Alaska: Application of video-recorded observations and expert panel judgment. Prevention Science, 11, 275–286.

    Article  PubMed Central  PubMed  Google Scholar 

  • Judd, C. M., & Kenny, D. A. (1981). Estimating the effects of social interventions. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education and Behavior, 31, 242–257.

    Article  Google Scholar 

  • MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58, 593–614.

    Article  PubMed Central  PubMed  Google Scholar 

  • MacKinnon, D. A., Johnson, C. A., Pentz, M. A., Hansen, W. B., Flay, B. R., & Yang, Y. F. I. (1991). Mediating mechanisms in a school-based drug prevention program: First year effects of the Midwestern Prevention Project. Health Psychology, 10, 164–172.

    Article  CAS  PubMed  Google Scholar 

  • McFarlane, W. R., Dixon, L., Lukens, E., & Lucksted, A. (2003). Family psychoeducation and schizophrenia: A review of the literature. Journal of Marital and Family Therapy, 29, 223–245.

    Article  PubMed  Google Scholar 

  • McGrew, J. H., Bond, G. R., Dietzen, L. L., & Salyers, M. P. (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62, 670–678.

    Article  CAS  PubMed  Google Scholar 

  • McNeal, R. B., & Hansen, W. B. (1999). Developmental patterns associated with the onset of drug use: Changes in postulated mediators during adolescence. Journal of Drug Issues, 29(2), 381–400.

    Google Scholar 

  • McNeal, R. B., Hansen, W. B., Harrington, N. G., & Giles, S. M. (2004). How All Stars works: An examination of program effects on mediating variables. Health Education Quarterly, 31, 165–178.

    Article  Google Scholar 

  • Minkoff, K. (1989). An integrated treatment model for dual diagnosis of psychosis and addiction. Hospital & Community Psychiatry, 40, 1031–1036.

    CAS  Google Scholar 

  • Mowbrey, C. T., Holter, M. C., Teague, T. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.

    Google Scholar 

  • Mueser, K. T., Corrigan, P. W., Hilton, D., Tanzman, B., Schaub, A., Gingerich, S., et al. (2002). Illness management and recovery for severe mental illness: A review of the research. Psychiatric Services, 53, 1272–1284.

    Article  PubMed  Google Scholar 

  • O’Donnell, C. L. (2007). Fidelity of implementation to instructional strategies as a moderator of curriculum unit effectiveness in a large-scale middle school science quasi-experiment. Washington, DC: George Washington University.

    Google Scholar 

  • Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. P., Dwyer, J. H., Johnson, C. A., et al. (1990). Effects of program implementation on adolescent drug use behavior: The Midwestern Prevention Project. Evaluation Review, 14, 264–289.

    Article  Google Scholar 

  • Ransford, C., Greenberg, M. T., Domitrovich, C. E., Small, M., & Jacobson, L. (2009). The role of teachers’ psychological experiences and perceptions of curriculum supports on implementation of a social emotional curriculum. School Psychology Review, 38, 510–532.

    Google Scholar 

  • Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., et al. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Education and Behavior, 30, 375–391.

    Article  PubMed  Google Scholar 

  • Ringwalt, C. L., Pankratz, M. M., Gottfredson, N., Jackson-Newsom, J., Dusenbury, L., Giles, S. M., et al. (2009). The effects of students’ curriculum engagement, attitudes towards their teachers, and perception of their teachers’ skills on school-based prevention curriculum outcomes. Journal of Drug Education, 39, 223–237.

    Article  PubMed Central  PubMed  Google Scholar 

  • Rohrbach, L. A., Dent, C. W., Skara, S., Sun, P., & Sussman, S. (2007). Fidelity of implementation in Project Towards No Drug Abuse (TND): A comparison of classroom teachers and program specialists. Prevention Science, 8, 125–132.

    Article  PubMed Central  PubMed  Google Scholar 

  • Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260.

    Article  CAS  PubMed  Google Scholar 

  • Rohrbach, L. A., Gunning, M., Sun, P., & Sussman, S. (2010). The Project Towards No Drug Abuse (TND) dissemination trial: Implementation fidelity and immediate outcomes. Prevention Science, 11, 77–88.

    Article  PubMed Central  PubMed  Google Scholar 

  • Ruiz-Primo, M. A. (2006). A multi-method and multi-source approach for studying fidelity of implementation. CSE Report 677. Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing, UCLA.

    Google Scholar 

  • Sloboda, Z., Stephens, P., Pyakuryal, A., Teasdale, B., Stephens, R. C., Hawthorne, R. D., et al. (2009). Implementation fidelity: The experience of the Adolescent Substance Abuse Prevention Study. Health Education Research, 24(3), 394–406.

    Article  PubMed  Google Scholar 

  • Sobol, D. F., Rohrbach, L. A., Dent, C. W., Gleason, L., Brannon, B. R., Johnson, C. A., et al. (1989). The integrity of smoking prevention curriculum delivery. Health Education Research, 4, 59–67.

    Article  Google Scholar 

  • Spoth, R., Guyll, M., Lillehoj, C. J., Redmond, C., & Greenberg, M. (2007). PROSPER study of evidence-based intervention implementation quality by community-university partnerships. Journal of Community Psychology, 35, 981–999.

    Article  PubMed Central  PubMed  Google Scholar 

  • Spoth, R., Guyll, M., Redmond, C., Greenberg, M., & Feinberg, M. (2011). Six-year sustainability of evidence-based intervention implementation quality by community-university partnerships: The PROSPER study. American Journal of Community Psychology, 48(3–4), 412–425.

    Article  PubMed Central  PubMed  Google Scholar 

  • Spoth, R., Redmond, C., & Shin, C. (1998). Direct and indirect latent-variable parenting outcomes of two universal family-focused preventive interventions: Extending a public health-oriented research base. Journal of Consulting and Clinical Psychology, 66(2), 385–399.

    Article  CAS  PubMed  Google Scholar 

  • Swanson, E., Wanzek, J., Haring, C., Ciullo, S., & McCulley, L. (2011). Intervention fidelity in special and general education research journals. Journal of Special Education, 20, 1–11.

    Google Scholar 

  • Tappe, M. K., Galer-Unti, R. A., & Bailey, K. C. (1995). Long-term implementation of the Teenage Health Teaching Modules by trained teachers: A case study. Journal of School Health, 65.

    Google Scholar 

  • Teague, G. B., & Bond, G. R. (2003a). Protocol for Assertive Community Treatment Fidelity Scale. http://ebp.networkofcare.org/uploads/ACT_Protocol_9942960.pdf

  • Teague, G. B., & Bond, G. R. (2003b). Dartmouth Assertive Community Treatment Scale. http://ebp.networkofcare.org/uploads/ACT_Scale_3113827.pdf

  • Teague, G. B., Bond, G. R., & Drake, R. E. (1998). Program fidelity in assertive community treatment: Development and use of a measure. American Journal of Orthopsychiatry, 68, 216–232.

    Article  CAS  PubMed  Google Scholar 

  • Teasdale, B., Stephens, P. C., Sloboda, Z., Grey, S. F., & Stephens, R. C. (2009). The influence of program mediators on eleventh grade outcomes for seventh grade substance users and nonuser. Drug and Alcohol Dependence, 102, 11–18.

    Article  PubMed  Google Scholar 

  • Tobler, N. S., & Stratton, H. (1997). Effectiveness of school-based drug prevention programs: A meta-analysis of the research. Journal of Primary Prevention, 18, 71–128.

    Article  Google Scholar 

  • Twemlow, S. W., Fonagy, P., Sacco, F. C., Gies, M. L., Evans, R., & Ewbank, R. (2001). Creating a peaceful school learning environment: A controlled study of an elementary school intervention to reduce violence. American Journal of Psychiatry, 158, 808–810.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tanglewood Research, Inc., Greensboro, NC, USA

    William B. Hansen

Authors
  1. William B. Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William B. Hansen .

Editor information

Editors and Affiliations

  1. Ontario, Ohio, USA

    Zili Sloboda

  2. JBS International, Inc., Rockville, USA

    Hanno Petras

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Hansen, W.B. (2014). Measuring Fidelity. In: Sloboda, Z., Petras, H. (eds) Defining Prevention Science. Advances in Prevention Science. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7424-2_15

Download citation

  • DOI: https://doi.org/10.1007/978-1-4899-7424-2_15

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4899-7423-5

  • Online ISBN: 978-1-4899-7424-2

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation