Abstract
Opportunities to evaluate strategies to create system-wide change in the child welfare system (CWS) and the resulting public health impact are rare. Leveraging a real-world, system-initiated effort to infuse the use of evidence-based principles throughout a CWS workforce, a pilot of the R3 model and supervisor-targeted implementation approach is described. The development of R3 and its associated fidelity monitoring was a collaboration between the CWS and model developers. Outcomes demonstrate implementation feasibility, strong fidelity scale measurement properties, improved supervisor fidelity over time, and the acceptability and perception of positive change by agency leadership. The value of system-initiated collaborations is discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A., Horowitz, J. D., Dlugosz, L. R., & Ehrhart, M. G. (2012). The role of organizational processes in dissemination and implementation research. In R. C. Brownson, G. A. Colditz, & E. K. Proctor (Eds.), Dissemination and implementation research in health: Translating science to practice (pp. 128–153). New York: Oxford University Press.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.
Adoption and Safe Families Act of 1997, Pub. L. No. 105-89, 111, Stat. 2115 (1997)
Altman, J. C. (2008). Engaging families in child welfare services: Worker versus client perspectives. Child Welfare, 87(3), 41–61.
Bandura, A. (1971). Vicarious and self-reinforcement processes. In R. Glaser (Ed.), The nature of reinforcement (pp. 228–278). New York: Academic Press.
Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30.
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
Chamberlain, P., Price, J., Reid, J. B., & Landsverk, J. (2008). Cascading implementation of a foster and kinship parent intervention. Child Welfare, 87(5), 27–48.
Cornerstones For Kids. (2006). The human services workforce initiative: Relationship between staff turnover, child welfare system functioning and recurrent child abuse. Houston, TX: Cornerstones For Kids.
Falander, C. A., & Shafranske, E. P. (2004). Clinical supervision. Washington, DC: American Psychological Association.
Forgatch, M. S., Patterson, G. R., & Gewirtz, A. H. (2013). Looking forward: The promise of widespread implementation of parent training programs. Perspectives on Psychological Science, 8, 682–694.
Institutes of Medicine. (2014). New directions in child abuse and neglect research. Washington, DC: National Academies Press.
Leve, L. D., Pears, K. C., & Fisher, P. A. (2002). Competence in early development. In J. B. Reid, G. R. Patterson, & J. Snyder (Eds.), Antisocial behavior in children and adolescents: A developmental analysis and model for intervention (pp. 45–64). Washington, DC: American Psychological Association.
Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85–106.
Linacre, J. M. (2015). WINSTEPS Rasch measurement computer program [Computer software and manual]. Beaverton, OR: Winsteps.com.
Marsh, J. C., Ryan, J. P., Choi, S., & Testa, M. F. (2006). Integrated services for families with multiple problems: Obstacles to family reunification. Children and Youth Services Review, 28, 1074–1087.
O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396–402.
Patterson, G. R., Reid, J. B., & Dishion, T. J. (1992). A social learning approach. IV. Antisocial boys. Eugene, OR: Castalia.
Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829–841.
Price, J. M., Chamberlain, P., Landsverk, J., Reid, J., Leve, L. D., & Laurent, H. (2008). Effects of a foster parent training intervention on placement changes of children in foster care. Child Maltreatment, 13, 64–75.
Ramsey, E., Walker, H. M., Shinn, M., O’Neill, R. E., & Stieber, S. (1989). Parent management practices and school adjustment. School Psychology Review, 18, 513–525.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: The Danish Institute of Educational Research.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
Reise, S. P. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544–559.
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667–696.
Schoenwald, S. K., Chapman, J. E., & Garland, A. F. (2014). Capturing fidelity in dissemination and implementation science. In R. S. Beidas & P. C. Kendall (Eds.), Dissemination and implementation of evidence-based practices in child and adolescent mental health (pp. 44–60). New York: Oxford University Press.
Schoenwald, S. K., Sheidow, A. J., & Chapman, J. E. (2009). Clinical supervision in treatment transport: Effects on adherence and outcomes. Journal of Consulting and Clinical Psychology, 77, 410–421.
Schumacker, R. E., & Smith, E. V, Jr. (2007). Reliability: A Rasch perspective. Educational and Psychological Measurement, 67, 394–409.
Shim, M. (2010). Factors influencing child welfare employee’s turnover: Focusing on organizational culture and climate. Children and Youth Services Review, 32, 847–856.
Smith, E. V, Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3, 205–231.
U.S. Department of Health and Human Services. (2014). Child welfare outcomes 2009–2012: Report to Congress. Washington, DC: U.S. Government Printing Office.
Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: Institute for Objective Measurement.
Wright, B. D., & Mok, M. (2000). Rasch models overview. Journal of Applied Measurement, 1, 83–106.
Acknowledgments
The authors would like to thank the New York City Administration for Children’s Services and the NYC supervisors and caseworkers for their collaboration in creating this model. We would also like to thank Courtenay Padgett for her project direction and Katie Lewis for her editorial assistance. This research was supported by the National Institute on Drug Abuse (P50 DA035763; R01 DA 032634; and 1R01DA040416-01A1) and the National Institute of Mental Health (R01 MH097748).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Saldana, L., Chamberlain, P. & Chapman, J. A Supervisor-Targeted Implementation Approach to Promote System Change: The R3 Model. Adm Policy Ment Health 43, 879–892 (2016). https://doi.org/10.1007/s10488-016-0730-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-016-0730-9