Skip to main content
SpringerLink
Log in

Methods Used to Document Procedural Fidelity in School-Based Intervention Research

  • Original Paper
  • Published:
Journal of Behavioral Education Aims and scope Submit manuscript

Abstract

There has been a paucity of guidance on the methodological details needed for measuring and sampling the independent variable or actual intervention occurrences in research and practice. Furthermore, the planning and support necessary to document the independent variable in both circumstances may be considerable. The current study extends prior research by reviewing the methods used to document procedural fidelity in school-based intervention research with student participants published between 2005 and 2012 in journals known for publishing school-related intervention studies. Two hundred sixty-six articles met the inclusion criteria and the majority (70 %) used fidelity data in the analysis of intervention effectiveness. By systematically examining contributions of school-based intervention researchers across targeted variables as well as qualities of interventions and settings, suggestions can be made for (a) planning measurement of fidelity across phases of the intervention, (b) sampling fidelity occurrences, (c) preventing intervention failure by providing needed supports, and (d) responding to patterns of fidelity required for intervention and outcome measurement in both research and practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Athens, E. S., Vollmer, T. R., & St. Peter Pipkin, C. C. (2007). Shaping academic task engagement with percentile schedules. Journal of Applied Behavior Analysis, 40, 475–488. doi:10.1901/jaba.2007.40-475.

    Article  PubMed  Google Scholar 

  • Austin, J. L., & Soeda, J. M. (2008). Fixed-time teacher attention to decrease off-task behaviors of typically developing third graders. Journal of Applied Behavior Analysis, 41, 279–283. doi:10.1901/jaba.2008.41-279.

    Article  PubMed  Google Scholar 

  • Barnett, D. W., Hawkins, R. O., & Lentz, F. E., Jr. (2011). Intervention adherence for research and practice: Necessity or triage outcome? Journal of Educational and Psychological Consultation, 21, 175–190. doi:10.1080/10474412.2011.595162.

    Article  Google Scholar 

  • Barnett, D. W., Lentz, F. E., Jr., & Macmann, G. (2000). Psychometric qualities of professional practice. In E. Shapiro & T. R. Kratochwill (Eds.), Behavioral assessment in schools (2nd ed., pp. 355–386). New York: Guilford.

    Google Scholar 

  • Begeny, J. C., & Silber, J. M. (2006). An examination of group-based treatment packages for increasing elementary-aged students’ reading fluency. Psychology in the Schools, 43, 183–195.

    Article  Google Scholar 

  • Boat, M. B., Carr, V., Barnett, D., Nichols, A., Macmann, G., Moomaw, S., et al. (2009). Instructional change in preschool classrooms: A study of empirically-based teacher support. NHSA Dialog: A Research to Practice Journal for the Early Intervention Field, 12, 307–326. doi:10.1080/15240750903218814.

    Article  Google Scholar 

  • Casey, A. M., & McWilliam, R. A. (2007). Graphical feedback to increase teachers’ use of incidental teaching. Journal of Early Education, 30, 251–268. doi:10.1177/1053815108319038.

    Google Scholar 

  • Codding, R., Livanis, A., Pace, G. M., & Vaca, L. (2008). Using performance feedback to improve treatment integrity of classwide behavior plans; an investigation of observer reactivity. Journal of Applied Behavior Analysis, 41, 417–422. doi:10.1901/jaba.2008.41-417.

    Article  PubMed  Google Scholar 

  • Cossairt, A., Hall, R. V., & Hopkins, B. L. (1973). The effects of experimenter’s instructions, feedback, and praise on teacher praise and student attending behavior. Journal of Applied Behavior Analysis, 6, 89–100. doi:10.1901/jaba.1973.6-89.

    Article  PubMed  Google Scholar 

  • Daunic, A., Smith, S., Garvan, C., Barber, B., Becker, M., Peters, C., et al. (2012). Reducing developmental risk for emotional/behavioral problems: A randomized controlled trial examining the tools for getting along curriculum. Journal of School Psychology, 50, 149–166.

    Article  PubMed  Google Scholar 

  • DiGennaro Reed, F. D., Reed, D. D., Baez, C. N., & Maguire, H. (2011). A parametric analysis of errors of commission during discrete-trial training. Journal of Applied Behavior Analysis, 44, 611–615. doi:10.1901/jaba.2011.44-611.

    Article  PubMed  Google Scholar 

  • Ducharme, J. M., & Schecter, C. (2011). Bridging the gap between clinical and classroom intervention: Keystone approaches for challenging behavior. School Psychology Review, 40, 257–274.

    Google Scholar 

  • Duhon, G. J., Mesmer, E. M., Atkins, M. E., Greguson, L. A., & Olinger, E. S. (2009a). Quantifying intervention intensity: A systematic approach to evaluating student response to increasing intervention frequency. Journal of Behavioral Education, 18, 101–118. doi:10.1007/s10864-009-9086-5.

    Article  Google Scholar 

  • Duhon, G. J., Mesmer, E. M., Gregerson, L., & Witt, J. C. (2009b). Effects of public feedback during RTI team meetings on teacher implementation integrity and student academic performance. Journal of School Psychology, 47, 19–37. doi:10.1016/j.jsp.2008.09.002.

    Article  Google Scholar 

  • Fryling, M., Wallace, M. D., & Yassine, J. N. (2012). Impact of treatment integrity on intervention effectiveness. Journal of Applied Behavior Analysis, 45, 449–453. doi:10.1901/jaba.2012.45-443.

    Article  PubMed  Google Scholar 

  • Gansle, K. A., & Noell, G. H. (2007). The fundamental role of intervention implementation in assessing response to intervention. In S. R. Jimerson, M. K. Burns, & A. M. VanderHeyden (Eds.), Handbook of response to intervention (pp. 244–251). New York, NY: Springer. doi:10.1007/978-0-387-49053-3_18.

    Chapter  Google Scholar 

  • Gilbertson, D., Witt, J. C., LaFleur Singletary, L., & VanDerHeyden, A. (2007). Supporting teacher use of interventions: Effects of response dependent performance feedback on teacher implementation of a math intervention. Journal of Behavioral Education, 16, 311–326. doi:10.1007/s10864-007-9043-0.

    Article  Google Scholar 

  • Gresham, F. M. (2005). Response to intervention: An alternative means of identifying students as emotionally disturbed. Education and Treatment of Children, 28, 328–344.

    Google Scholar 

  • Gresham, F. M. (2009). Evolution of the treatment integrity concept: Current status and future directions. School Psychology Review, 38, 533–540.

    Google Scholar 

  • Gresham, F. M., Gansle, K. A., & Noell, G. H. (1993a). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 26, 257–263. doi:10.1901/jaba.1993.26-257.

    Article  PubMed  Google Scholar 

  • Gresham, F. M., Gansle, K., Noell, G. H., Cohen, S., & Rosenblum, S. (1993b). Treatment integrity in school based intervention studies: 1980–1990. School Psychology Review, 22, 254–272.

    Google Scholar 

  • Haring, N. G., & Eaton, M. D. (1978). Systematic procedures: An instructional hierarchy. In N. G. Haring, T. C. Lovitt, M. D. Eaton, & C. L. Hansen (Eds.), The fourth R: Research in the classroom (pp. 23–40). Columbus, OH: Charles E. Merrill Publishing Company.

    Google Scholar 

  • Horner, R. H., Carr, E. G., McGee, G., Odom, S., & Wolery, M. (2005). The use of single subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.

    Google Scholar 

  • Jitendra, A. K., DuPaul, G. J., Volpe, R. J., Tresco, K. E., Vile Junod, R. E., Lutz, J. G., et al. (2007). Consultation-based academic intervention for children with attention deficit hyperactivity disorder: School functioning outcomes. School Psychology Review, 36(2), 217–236.

    Google Scholar 

  • Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and tactics of behavioral research (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Jones, K. M., & Wickstrom, K. F. (2010). Using functional assessment to select behavioral interventions. In G. Gimpel Peacock, R. A. Ervin, E. J. Daly III, & K. W. Merrell (Eds.), Practical handbook of school psychology (pp. 192–210). New York: Guilford.

    Google Scholar 

  • Kara, A., & Wahler, R. (1977). Organizational features of a young child’s behaviors. Journal of Experimental Child Psychology, 24, 24–39. doi:10.1016/0022-0965(77)90017-0.

    Article  Google Scholar 

  • Kennedy, C. (2005). Single case designs for educational research. Boston: Allyn & Bacon.

    Google Scholar 

  • Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from What Works Clearinghouse website: http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.

  • Massetti, G. M. (2009). Enhancing emergent early literacy skills of preschoolers from low-income environments through a classroom-based approach. School Psychology Review, 38, 554–569.

    Google Scholar 

  • McIntyre, L. L., Gresham, F. M., DiGennaro, F. D., & Reed, D. D. (2007). Treatment integrity of school-based interventions with children in Journal of Applied Behavior Analysis studies from 1991–2005. Journal of Applied Behavior Analysis, 40, 659–672. doi:10.1901/jaba.2007.659-672.

    PubMed  Google Scholar 

  • Menesses, K. F., & Gresham, F. M. (2009). Relative efficacy of reciprocal and nonreciprocal peer tutoring for students at-risk for academic failure. School Psychology Quarterly, 24, 266–275.

    Article  Google Scholar 

  • Miles, N. I., & Wilder, D. A. (2009). The effects of behavioral skills training on caregiver implementation of guided compliance. Journal of Applied Behavior Analysis, 42, 405–410. doi:10.1901/jaba.2009.42-405.

    Article  PubMed  Google Scholar 

  • Moncher, F. M., & Printz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266. doi:10.1016/0272-7358(91)90103-2.

    Article  Google Scholar 

  • Munro, D. W., & Stephenson, J. (2009). The effects of response cards on student and teacher behavior during vocabulary instruction. Journal of Applied Behavior Analysis, 42, 795–800. doi:10.1901/jaba.2009.42-795.

    Article  PubMed  Google Scholar 

  • Neef, N. A. (1995). Research on training trainers in program implementation: An introduction and future directions. Journal of Applied Behavior Analysis, 28, 297–299. doi:10.1901/jaba.1995.28-297.

    Article  PubMed  Google Scholar 

  • Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., et al. (2005). Treatment implementation following behavioral consultation: A comparison of three follow-up strategies. School Psychology Review, 34, 87–106.

    Google Scholar 

  • Page, T. J., Iwata, B. A., & Ward, D. H. (1982). Pyramidal training: A large-scale application with institutional staff. Journal of Applied Behavior Analysis, 15, 335–351. doi:10.1901/jaba.1982.15-335.

    Article  PubMed  Google Scholar 

  • Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 447–492. doi:10.1901/jaba.1982.15-477.

    Article  Google Scholar 

  • Reed, D. D., Fienup, D. M., Luiselli, J. K., & Pace, G. M. (2010). Performance improvement in behavioral health care: Collateral effects of planned treatment integrity observations as an applied example of schedule induced responding. Behavior Modification, 34, 367–385. doi:10.1177/0145445510383524.

    Article  PubMed  Google Scholar 

  • Riley-Tillman, T. C., & Burns, M. K. (2009). Evaluating educational interventions: Single-case designs for measuring response to intervention. New York: Guilford.

    Google Scholar 

  • Sainato, D., Strain, P. S., & Lyon, S. R. (1987). Increasing academic responding of handicapped preschool children during group instruction. Journal for the Division of Early Childhood, 12, 23–31. doi:10.1177/105381518701200104.

    Google Scholar 

  • Sanetti, L. M. H., Gritter, K. L., & Dobey, L. M. (2011). Treatment integrity of interventions with children in the school psychology literature from 1995 to 2008. School Psychology Review, 40, 72–84.

    Google Scholar 

  • Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction to the special issue. School Psychology Review, 38, 445–459.

    Google Scholar 

  • Sarokoff, R. A., & Sturmey, P. (2004). The effects of behavioral skills training on staff implementation of discrete-trial teaching. Journal of Applied Behavior Analysis, 37, 535–538. doi:10.1901/jaba.2004.37-531.

    Article  PubMed  Google Scholar 

  • Schepis, M. M., Reid, D. H., Ownby, J., & Parsons, M. B. (2001). Training support staff to embed teaching within natural routines of young children with disabilities in an inclusive school. Journal of Applied Behavior Analysis, 34, 313–317. doi:10.1901/jaba.2001.34-313.

    Article  PubMed  Google Scholar 

  • Sechrest, L., West, S. G., Phillips, M., Redner, R., & Yeaton, W. H. (1979). Some neglected problems in evaluation research: Strength and integrity of treatments. In L. Sechrest, S. G. West, M. Phillips, R. Redner, & W. H. Yeaton (Eds.), Evaluation studies review annual (Vol. 4, pp. 15–35). Beverly Hills, CA: Sage.

    Google Scholar 

  • Solomon, B. G., Klein, S. A., & Politylo, B. C. (2012). The effect of performance feedback on teachers’ treatment integrity: A meta-analysis of the single-case literature. School Psychology Review, 41, 160–175.

    Google Scholar 

  • St. Peter Pipkin, C., Vollmer, T. R., & Sloman, K. N. (2010). Effects of treatment integrity failures during differential reinforcement of alternative behavior: A translational model. Journal of Applied Behavior Analysis, 43, 47–70. doi:10.1901/jaba.2010.43-47.

    Article  PubMed  Google Scholar 

  • VanDerHeyden, A. M., & Witt, J. C. (2008). Best practices in can’t do/won’t do assessment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (5th ed., Vol. 2, pp. 131–140). Bethesda, MD: National Association of School Psychologists.

    Google Scholar 

  • VanDerHeyden, A. M., Witt, J. C., & Gatti, S. (2001). Descriptive assessment method to reduce overall disruptive behavior in a preschool classroom. School Psychology Review, 30, 548–567.

    Google Scholar 

  • Wilder, D. A., Atwell, J., & Wine, B. (2006). The effects of varying levels of treatment integrity on child compliance during treatment with a three-step prompting procedure. Journal of Applied Behavior Analysis, 39, 369–373. doi:10.1901/jaba.2006.144-05.

    Article  PubMed  Google Scholar 

  • Witt, J. C., Daly, E. M., I. I. I., & Noell, G. (2010). Functional assessment: A step-by-step guide to solving academic and behavior problems. Longmont, CO: Sopris West.

    Google Scholar 

  • Witt, J., VanDerHeyden, A., & Gilbertson, D. (2004). Troubleshooting behavioral interventions: A systematic process for finding and eliminating problems. School Psychology Review, 33, 363–383.

    Google Scholar 

  • Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156–168.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cincinnati, OH, USA

    David Barnett, Renee Hawkins, Dacia McCoy, Elaine Wahl, Ashley Shier, Hilary Denune & Lauren Kimener

Authors
  1. David Barnett
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Renee Hawkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dacia McCoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elaine Wahl
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ashley Shier
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hilary Denune
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lauren Kimener
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Barnett.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Barnett, D., Hawkins, R., McCoy, D. et al. Methods Used to Document Procedural Fidelity in School-Based Intervention Research. J Behav Educ 23, 89–107 (2014). https://doi.org/10.1007/s10864-013-9188-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10864-013-9188-y

Keywords

Search

Navigation