Abstract
School personnel make a variety of decisions within multitiered problem-solving frameworks, including the decision to assign a student to group-based support, to design an individualized support plan, or classify a student as eligible for special education. Each decision is founded upon a judgment regarding whether the student has responded to intervention. These and other conclusions are inherently causal, thus requiring that educators carefully consider the internal, construct, and conclusion validity of each decision to ensure its defensibility. Researchers have identified multiple variables that are likely to moderate these validities, including the integrity with which interventions are implemented, the psychometric adequacy of progress-monitoring tools, the extent to which interventions and supports are matched to a student's needs, and the approach to single-case research design. We therefore review each of these variables in the interest of assisting practitioners to design acceptable and valid multi-tiered frameworks of prevention and service delivery.
Similar content being viewed by others
References
Alberto, P. A., & Troutman, A. C. (2009). Applied behavior analysis for teachers (8th ed.). Upper Saddle River: Pearson.
Biggs, B. K., Vernberg, E. M., Twemlow, S. W., Fonagy, P., & Dill, E. J. (2008). Teacher adherence and its relation to teacher attitudes and student outcomes in an elementary school-based violence prevention program. School Psychology Review, 37, 533–549.
Brown-Chidsey, R., Bronaugh, L., & McGraw, K. (2009). RTI in the classroom: guidelines and recipes for success. New York: Guilford.
Burns, M. K., Ganuza, Z. M., & London, R. M. (2009). Brief experimental analysis of written letter formation: single-case demonstration. Journal of Behavioral Education, 18, 20–34.
Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: new directions for psychology and education (pp. 187–212). Hillsdale: Erlbaum.
Campbell, J. M. (2004). Statistical comparison of four effect sizes for single-subject designs. Behavior Modification, 28, 234–246.
Campbell, A., & Anderson, C. M. (2008). Enhancing effects of check-in/check-out with function-based support. Behavioral Disorders, 33, 233–245.
Chafouleas, S. M. (2011). Direct behavior rating: a review of the issues and research in its development. Education and Treatment of Children, 34, 575–591.
Chafouleas, S. M., Riley-Tillman, T. C., & Sugai, G. (2007). School-based behavioral assessment: informing intervention and instruction. New York: Guilford.
Cochrane, W. S., & Laux, J. M. (2008). A survey investigating school psychologists' measurement of treatment integrity in school-based interventions and their beliefs about its importance. Psychology in the Schools, 45, 499–507.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River: Pearson.
Crone, D. A., & Horner, R. H. (2003). Building positive behavior support systems in schools: functional behavioral assessment. New York: Guilford.
Daly, E. J., Witt, J. C., Martens, B. K., & Dool, E. J. (1997). A model for conducting a functional analysis of academic performance problems. School Psychology Review, 26, 554–574.
Drummond, T. (1994). The student risk screening scale. Grants Pass: Josephine County Mental Health Program.
Filter, K. J., & Horner, R. H. (2009). Function-based academic interventions for problem behavior. Education and Treatment of Children, 32, 1–19.
Fuchs, D. M., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness-to-intervention: definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18, 157–171.
Good, R. H., & Jefferson, G. (1989). Contemporary perspectives on curriculum-based measurement validity. In M. Shinn (Ed.), Advanced applications of curriculum-based measurement (pp. 61–88). New York: Guilford.
Gresham, F. M. (2010). Data-based decision making for students' social behavior. Journal of Evidence-Based Practices for Schools, 11, 149–168.
Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18, 37–50.
Hammond, D., & Gast, D. L. (2010). Descriptive analysis of single subject research designs: 1983–2007. Education and Training in Autism and Developmental Disabilities, 45, 187–202.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.
Kazdin, A. E., Kratochwill, T. R., & Vanden Bos, G. (1986). Beyond clinical trials: generalizing from research to practice. Professional Psychology: Research and Practice, 3, 391–398.
Knapp, T. J. (1983). Behavior analysts' visual appraisal of behavior change in graphic display. Behavioral Assessment, 5, 155–164.
Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., et al. (2010). Single-case designs technical documentation. Resource document. What Works Clearinghouse.: http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf. Accessed 18 Aug 2012
Kratochwill, T. R., Hoagwood, K. E., Kazak, A. E., Weisz, J. R., Hood, K., Vargas, L. A., et al. (2012). Practice-based evidence for children and adolescents: advancing the research agenda in schools. School Psychology Review, 41, 215–235.
Lane, K. L., Menzies, H. M., Oakes, W. P., & Kalberg, J. R. (2012). Systematic screenings of behavior to support instruction: from preschool to high school. New York: Guilford.
March, R. E., Horner, R. H., Lewis-Palmer, T., Brown, D., Crone, D., Todd, A. W., et al. (2000). Functional Assessment Checklist: Teachers and Staff (FACTS). Eugene, OR: Educational and Community Supports.
McIntosh, K., Campbell, A. L., Carter, D. R., & Dickey, C. R. (2009). Differential effects of a tier two behavior intervention based on function of problem behavior. Journal of Positive Behavior Interventions, 11, 82–93.
Mercer, S. H., & Sterling, H. E. (2012). The impact of baseline trend control on visual analysis of single-case data. Journal of School Psychology, 50, 403–419.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
National Association of School Psychologists (2010). Model for comprehensive and integrated school psychology services. Communique, 39(4), 1–6. Resource document.: http://nasponline.org/standards/2010standards/2_PracticeModel.pdf. Accessed 18 Aug 2012
Noell, G. H., & Gansle, K. A. (2006). Assuring the form has substance: treatment plan implementation as the foundation of assessing response to intervention. Assessment for Effective Intervention, 32, 32–39.
Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., et al. (2005). Treatment implementation following behavioral consultation in schools: a comparison of three follow-up strategies. School Psychology Review, 34, 87–106.
Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for single-case research. Exceptional Children, 75, 135–150.
Peterson, L., Homer, A., & Wonderlich, S. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477–492.
Riley-Tillman, T. C., & Burns, M. K. (2009). Evaluating educational interventions: single-case design for measuring response to intervention. New York: Guilford.
Ruble, L., McGrew, J. H., & Toland, M. D. (2012). Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 42, 1974–1983.
Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: introduction to the special series. School Psychology Review, 38, 445–459.
Sanetti, L. M. H., Fallon, L. M., & Collier-Meek, M. A. (2011). Treatment integrity assessment and intervention by school-based personnel: practical applications based on a preliminary study. School Psychology Forum, 5, 87–102.
Schulte, A. C., Easton, J. E., & Parker, J. (2009). Advances in treatment integrity research: multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38, 460–475.
Scruggs, T. E., Mastropieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: methodology and validation. Remedial and Special Education, 8, 24–33.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
Sidman, M. (2011). Can an understanding of basic research facilitate the effectiveness of practitioners? Reflections and personal perspectives. Journal of Applied Behavior Analysis, 44, 973–991.
Stoner, G., & Green, S. K. (1992). Reconsidering the scientist-practitioner model for school psychology practice. School Psychology Review, 21, 155–166.
Turtura, J. E., Anderson, C. M., & Boyd, R. J. (2013). Addressing task avoidance in middle school students: academic behavior check-in/check-out. Journal of Positive Behavior Interventions, 42((6)), 1–9. doi:10.1177/1098300713484063.
Urbina, S. (2004). Essentials of psychological testing. Hoboken: Wiley.
Vaughn, S., & Fuchs, L. S. (2003). Redefining learning disabilities as inadequate response to instruction: the promise and potential problems. Learning Disabilities Research & Practice, 18, 137–146.
Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156–167.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kilgus, S.P., Collier-Meek, M.A., Johnson, A.H. et al. Applied Empiricism: Ensuring the Validity of Causal Response to Intervention Decisions. Contemp School Psychol 18, 1–12 (2014). https://doi.org/10.1007/s40688-013-0009-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40688-013-0009-z