Skip to main content

Advertisement

Log in

Applied Empiricism: Ensuring the Validity of Causal Response to Intervention Decisions

  • Published:
Contemporary School Psychology Aims and scope Submit manuscript

Abstract

School personnel make a variety of decisions within multitiered problem-solving frameworks, including the decision to assign a student to group-based support, to design an individualized support plan, or classify a student as eligible for special education. Each decision is founded upon a judgment regarding whether the student has responded to intervention. These and other conclusions are inherently causal, thus requiring that educators carefully consider the internal, construct, and conclusion validity of each decision to ensure its defensibility. Researchers have identified multiple variables that are likely to moderate these validities, including the integrity with which interventions are implemented, the psychometric adequacy of progress-monitoring tools, the extent to which interventions and supports are matched to a student's needs, and the approach to single-case research design. We therefore review each of these variables in the interest of assisting practitioners to design acceptable and valid multi-tiered frameworks of prevention and service delivery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Alberto, P. A., & Troutman, A. C. (2009). Applied behavior analysis for teachers (8th ed.). Upper Saddle River: Pearson.

    Google Scholar 

  • Biggs, B. K., Vernberg, E. M., Twemlow, S. W., Fonagy, P., & Dill, E. J. (2008). Teacher adherence and its relation to teacher attitudes and student outcomes in an elementary school-based violence prevention program. School Psychology Review, 37, 533–549.

    Google Scholar 

  • Brown-Chidsey, R., Bronaugh, L., & McGraw, K. (2009). RTI in the classroom: guidelines and recipes for success. New York: Guilford.

    Google Scholar 

  • Burns, M. K., Ganuza, Z. M., & London, R. M. (2009). Brief experimental analysis of written letter formation: single-case demonstration. Journal of Behavioral Education, 18, 20–34.

    Article  Google Scholar 

  • Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: new directions for psychology and education (pp. 187–212). Hillsdale: Erlbaum.

    Google Scholar 

  • Campbell, J. M. (2004). Statistical comparison of four effect sizes for single-subject designs. Behavior Modification, 28, 234–246.

    Article  PubMed  Google Scholar 

  • Campbell, A., & Anderson, C. M. (2008). Enhancing effects of check-in/check-out with function-based support. Behavioral Disorders, 33, 233–245.

    Google Scholar 

  • Chafouleas, S. M. (2011). Direct behavior rating: a review of the issues and research in its development. Education and Treatment of Children, 34, 575–591.

    Article  Google Scholar 

  • Chafouleas, S. M., Riley-Tillman, T. C., & Sugai, G. (2007). School-based behavioral assessment: informing intervention and instruction. New York: Guilford.

    Google Scholar 

  • Cochrane, W. S., & Laux, J. M. (2008). A survey investigating school psychologists' measurement of treatment integrity in school-based interventions and their beliefs about its importance. Psychology in the Schools, 45, 499–507.

    Article  Google Scholar 

  • Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River: Pearson.

    Google Scholar 

  • Crone, D. A., & Horner, R. H. (2003). Building positive behavior support systems in schools: functional behavioral assessment. New York: Guilford.

    Google Scholar 

  • Daly, E. J., Witt, J. C., Martens, B. K., & Dool, E. J. (1997). A model for conducting a functional analysis of academic performance problems. School Psychology Review, 26, 554–574.

    Google Scholar 

  • Drummond, T. (1994). The student risk screening scale. Grants Pass: Josephine County Mental Health Program.

    Google Scholar 

  • Filter, K. J., & Horner, R. H. (2009). Function-based academic interventions for problem behavior. Education and Treatment of Children, 32, 1–19.

    Article  Google Scholar 

  • Fuchs, D. M., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness-to-intervention: definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18, 157–171.

    Article  Google Scholar 

  • Good, R. H., & Jefferson, G. (1989). Contemporary perspectives on curriculum-based measurement validity. In M. Shinn (Ed.), Advanced applications of curriculum-based measurement (pp. 61–88). New York: Guilford.

    Google Scholar 

  • Gresham, F. M. (2010). Data-based decision making for students' social behavior. Journal of Evidence-Based Practices for Schools, 11, 149–168.

    Google Scholar 

  • Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18, 37–50.

    Google Scholar 

  • Hammond, D., & Gast, D. L. (2010). Descriptive analysis of single subject research designs: 1983–2007. Education and Training in Autism and Developmental Disabilities, 45, 187–202.

    Google Scholar 

  • Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.

    Article  Google Scholar 

  • Kazdin, A. E., Kratochwill, T. R., & Vanden Bos, G. (1986). Beyond clinical trials: generalizing from research to practice. Professional Psychology: Research and Practice, 3, 391–398.

    Article  Google Scholar 

  • Knapp, T. J. (1983). Behavior analysts' visual appraisal of behavior change in graphic display. Behavioral Assessment, 5, 155–164.

    Google Scholar 

  • Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., et al. (2010). Single-case designs technical documentation. Resource document. What Works Clearinghouse.: http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf. Accessed 18 Aug 2012

  • Kratochwill, T. R., Hoagwood, K. E., Kazak, A. E., Weisz, J. R., Hood, K., Vargas, L. A., et al. (2012). Practice-based evidence for children and adolescents: advancing the research agenda in schools. School Psychology Review, 41, 215–235.

    Google Scholar 

  • Lane, K. L., Menzies, H. M., Oakes, W. P., & Kalberg, J. R. (2012). Systematic screenings of behavior to support instruction: from preschool to high school. New York: Guilford.

    Google Scholar 

  • March, R. E., Horner, R. H., Lewis-Palmer, T., Brown, D., Crone, D., Todd, A. W., et al. (2000). Functional Assessment Checklist: Teachers and Staff (FACTS). Eugene, OR: Educational and Community Supports.

    Google Scholar 

  • McIntosh, K., Campbell, A. L., Carter, D. R., & Dickey, C. R. (2009). Differential effects of a tier two behavior intervention based on function of problem behavior. Journal of Positive Behavior Interventions, 11, 82–93.

    Article  Google Scholar 

  • Mercer, S. H., & Sterling, H. E. (2012). The impact of baseline trend control on visual analysis of single-case data. Journal of School Psychology, 50, 403–419.

    Article  PubMed  Google Scholar 

  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.

    Article  Google Scholar 

  • National Association of School Psychologists (2010). Model for comprehensive and integrated school psychology services. Communique, 39(4), 1–6. Resource document.: http://nasponline.org/standards/2010standards/2_PracticeModel.pdf. Accessed 18 Aug 2012

    Google Scholar 

  • Noell, G. H., & Gansle, K. A. (2006). Assuring the form has substance: treatment plan implementation as the foundation of assessing response to intervention. Assessment for Effective Intervention, 32, 32–39.

    Article  Google Scholar 

  • Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., et al. (2005). Treatment implementation following behavioral consultation in schools: a comparison of three follow-up strategies. School Psychology Review, 34, 87–106.

    Google Scholar 

  • Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for single-case research. Exceptional Children, 75, 135–150.

    Google Scholar 

  • Peterson, L., Homer, A., & Wonderlich, S. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477–492.

    Article  PubMed Central  PubMed  Google Scholar 

  • Riley-Tillman, T. C., & Burns, M. K. (2009). Evaluating educational interventions: single-case design for measuring response to intervention. New York: Guilford.

    Google Scholar 

  • Ruble, L., McGrew, J. H., & Toland, M. D. (2012). Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 42, 1974–1983.

    Article  PubMed Central  PubMed  Google Scholar 

  • Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: introduction to the special series. School Psychology Review, 38, 445–459.

    Google Scholar 

  • Sanetti, L. M. H., Fallon, L. M., & Collier-Meek, M. A. (2011). Treatment integrity assessment and intervention by school-based personnel: practical applications based on a preliminary study. School Psychology Forum, 5, 87–102.

    Google Scholar 

  • Schulte, A. C., Easton, J. E., & Parker, J. (2009). Advances in treatment integrity research: multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38, 460–475.

    Google Scholar 

  • Scruggs, T. E., Mastropieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: methodology and validation. Remedial and Special Education, 8, 24–33.

    Article  Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  • Sidman, M. (2011). Can an understanding of basic research facilitate the effectiveness of practitioners? Reflections and personal perspectives. Journal of Applied Behavior Analysis, 44, 973–991.

    Article  PubMed Central  PubMed  Google Scholar 

  • Stoner, G., & Green, S. K. (1992). Reconsidering the scientist-practitioner model for school psychology practice. School Psychology Review, 21, 155–166.

    Google Scholar 

  • Turtura, J. E., Anderson, C. M., & Boyd, R. J. (2013). Addressing task avoidance in middle school students: academic behavior check-in/check-out. Journal of Positive Behavior Interventions, 42((6)), 1–9. doi:10.1177/1098300713484063.

    Google Scholar 

  • Urbina, S. (2004). Essentials of psychological testing. Hoboken: Wiley.

    Google Scholar 

  • Vaughn, S., & Fuchs, L. S. (2003). Redefining learning disabilities as inadequate response to instruction: the promise and potential problems. Learning Disabilities Research & Practice, 18, 137–146.

    Article  Google Scholar 

  • Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156–167.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Stephen P. Kilgus.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kilgus, S.P., Collier-Meek, M.A., Johnson, A.H. et al. Applied Empiricism: Ensuring the Validity of Causal Response to Intervention Decisions. Contemp School Psychol 18, 1–12 (2014). https://doi.org/10.1007/s40688-013-0009-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40688-013-0009-z

Keywords

Navigation