Skip to main content
Log in

Practical Implications of Data Reliability and Treatment Integrity Monitoring

  • Published:
Behavior Analysis in Practice Aims and scope Submit manuscript

Abstract

Data reliability and treatment integrity have important implications for clinical practice because they can affect clinicians’ abilities to accurately judge the efficacy of behavioral interventions. Reliability and integrity data also allow clinicians to provide feedback to caregivers and to adjust interventions as needed. We present reasons why reliability and integrity measures are paramount in clinical work, discuss events that may result in decreased reliability or integrity, and provide several efficient means for collecting data and calculating reliability and integrity measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Allen, K. D., & Warzak, W. J. (2000). The problem of parental nonadherence in clinical behavior analysis: Effective treatment is not enough. Journal of Applied Behavior Analysis, 33, 373–391.

    Article  PubMed  PubMed Central  Google Scholar 

  • Brackett, L., Reid, D. H., & Green, C. W. (2007). Effects of reactivity to observations on staff performance. Journal of Applied Behavior Analysis, 40, 191–195.

    Article  PubMed  PubMed Central  Google Scholar 

  • DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback procedures on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40, 447–461.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S. & Rosenblum, S. (1993). Treatment integrity of school-based behavioral intervention studies: 1980–1990. School Psychology Review, 22, 254–272.

    Google Scholar 

  • Hartmann, D. P. (1977). Considerations in the choice of interobserver reliability estimates. Journal of Applied Behavior Analysis, 10, 103–116.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hawkins, R. P. and Dotson, V. A. (1975). Reliability scores that delude: An Alice in Wonderland Trip through the misleading characteristics of interobserver agreement scores in interval recording. In E. Ramp and G. Semb (Eds.), Behavior Analysis: Areas of research and application. Englewood Cliffs, New Jersey: Prentice-Hall.

    Google Scholar 

  • Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABCs of reliability. Journal of Applied Behavior Analysis, 10, 141–150.

    Article  PubMed  PubMed Central  Google Scholar 

  • McIntyre, L. L., Gresham, F. M., DiGennaro, F. D., & Reed, D. D. (2007). Treatment integrity of school-based interventions with children in Journal of Applied Behavior Analysis Studies from 1991 to 2005. Journal of Applied Behavior Analysis, 40, 659–672.

    Article  PubMed  Google Scholar 

  • Noell, G. H., & Witt, J. C. (1998). Toward a behavior analytic approach to consultation. In T. S. Watson and F. M. Gresham (Eds.), Handbook of child behavior therapy. New York, NY: Plenum Press.

    Google Scholar 

  • Noell, G. H., Witt, J. C., LaFleur, L. H., Mortenson, B. P., Ranier, D. D., & LeVelle, J. (2000). Increasing intervention implementation in general education following consultation: A comparison of two follow-up strategies. Journal of Applied Behavior Analysis, 33, 271–284.

    Article  PubMed  PubMed Central  Google Scholar 

  • O’Leary, K. D., Kent, R. N., & Kanowitz, J. (1975). Shaping data collection congruent with experimental hypotheses. Journal of Applied Behavior Analysis, 8, 43–51.

    Article  PubMed  PubMed Central  Google Scholar 

  • Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477–492.

    Article  PubMed  PubMed Central  Google Scholar 

  • St. Peter Pipkin, C. C. (2006). A laboratory investigation of the effects of treatment integrity failures on differential reinforcement procedures. Unpublished doctoral dissertation, University of Florida, Gainesville.

    Google Scholar 

  • Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10, 349–367.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sulzer-Azaroff, B., & Mayer, G. R. (1991). Behavior analysis for lasting change. Fort Worth: Holt, Rinehart, & Winston.

    Google Scholar 

  • Vollmer, T. R., Marcus, B. A., & LeBlanc, L. (1994). Treatment of self-injury and hand mouthing following inconclusive functional analysis. Journal of Applied Behavior Analysis, 27, 331–344.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Timothy R. Vollmer Ph.D., BCBA.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vollmer, T.R., Sloman, K.N. & St. Peter Pipkin, C. Practical Implications of Data Reliability and Treatment Integrity Monitoring. Behav Analysis Practice 1, 4–11 (2008). https://doi.org/10.1007/BF03391722

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03391722

Descriptors

Navigation