Abstract
In recent years, implementation researchers have focused on associations among organizational characteristics (inner settings), policy and funding drivers (outer setting), and implementation outcomes. The current study evaluated the influence of outer setting drivers on implementation of Wraparound care coordination for youth with complex behavioral health needs. Data were drawn from two sources. First, we examined the impact of outer setting drivers on Wraparound implementation by comparing Wraparound implementation in states that used Community Mental Health Centers (CMHCs) versus Care Management Entities (CMEs). Wraparound fidelity data were compiled for a sample of 1174 direct service providers within 9 states. Second, we compared workforce development efforts across CMHCs and CMEs within a separate sample of 813 administrators and practitioners. Results of multilevel models found that CMEs were associated with higher overall fidelity scores than CMHCs [b = .219, t(5.47) = 3.26, p = .020] even after accounting for state-level covariates and the clustering of individual practitioners within organizations. Furthermore, compared with CMHC staff, those employed by CMEs reported higher competence with the Wraparound model and attended more Wraparound-related trainings. Staff employed by CMHCs were more likely to change practices than their CME counterparts. These findings suggest possible influences of outer settings on Wraparound implementation. The unique policies and procedures associated with CMEs may have helped facilitate practice fidelity among staff, promoted competence, and required fewer practice changes to implement Wraparound. Such findings underscore the importance of policy and fiscal factors in implementation of Wraparound and other evidence-based strategies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274. https://doi.org/10.1146/annurev-publhealth-032013-182447.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7.
Aarons, G. A., & Sommerfeld, D. H. (2012). Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. Journal of the American Academy of Child and Adolescent Psychiatry, 51(4), 423–431. https://doi.org/10.1016/j.jaac.2012.01.018.
Anglin, G., Swinburn, A., Foster, L., Brach, C., & Bergofsky, L. (2014). Designing care management entities for youth with complex behavioral health needs (No. 8b26e5381126442589b3a4e7f0a701a3). Mathematica Policy Research.
Bond, G. R. (2018). Evidence-based policy strategies: A typology. Clinical Psychology: Science and Practice, 25(4), 1–14. https://doi.org/10.1111/cpsp.12267.
Bruns, E. J., Kerns, S. E., Pullmann, M. D., Hensley, S. W., Lutterman, T., & Hoagwood, K. E. (2016). Research, data, and evidence-based treatment use in state behavioral health systems, 2001–2012. Psychiatric Services, 67(5), 496–503. https://doi.org/10.1176/appi.ps.201500014.
Bruns, E. J., Parker, E. M., Hensley, S., Pullmann, M. D., Benjamin, P. H., Lyon, A. R., & Hoagwood, K. E. (2019). The role of the outer setting in implementation: Associations between state demographic, fiscal, and policy factors and use of evidence-based treatments in mental healthcare. Implementation Science, 14(1), 96. https://doi.org/10.1186/s13012-019-0944-9.
Bruns, E. J., Pullmann, M. D., Sather, A., Brinson, R. D., & Ramey, M. (2015). Effectiveness of wraparound versus case management for children and adolescents: Results of a randomized study. Administration and Policy in Mental Health and Mental Health Services Research, 42(3), 309–322. https://doi.org/10.1007/s10488-014-0571-3.
Bruns, E. J., Suter, J. C., & Leverentz-Brady, K. M. (2006). Relations between program and system variables and fidelity to the wraparound process for children and families. Psychiatric Services, 57(11), 1586–1593. https://doi.org/10.1176/ps.2006.57.11.1586.
Bruns, E. J., Walker, J. S., Zabel, M., Matarese, M., Estep, K., Harburger, D., Mosby, M., & Pires, S. A. (2010). Intervening in the lives of youth with complex behavioral health challenges and their families: The role of the wraparound process. American Journal of Community Psychology, 46(3–4), 314–331. https://doi.org/10.1007/s10464-010-9346-5.
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8, 117. https://doi.org/10.1186/1748-5908-8-117.
Coldiron, J. S., Bruns, E. J., & Quick, H. (2017). A comprehensive review of wraparound care coordination research, 1986–2014. Journal of Child and Family Studies, 26(5), 1245–1265. https://doi.org/10.1007/s10826-016-0639-7.
Coldiron, J. S., Walker, J., & Hensley, S. (2015, March). The revision and application of a training impact survey for Wraparound. In Paper presented at the 2015 annual research and policy conference on child, adolescent, and youth adult behavioral health, Tampa, FL.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50–64. https://doi.org/10.1186/1748-5908-4-50.
Damschroder, L. J., & Hagedorn, H. J. (2011). A guiding framework and approach for implementation research in substance use disorders treatment. Psychology of Addictive Behaviors, 25(2), 194–205. https://doi.org/10.1037/a0022284.
Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157. https://doi.org/10.1186/s13012-014-0157-1.
Estep, K., Matarese, M., & Hensley, S. (2014). Development, training, and initial statistical summary of a coaching measure for wraparound teams. In Presentation at 27th annual children’s mental health research and policy conference, Tampa, FL.
Field, A. (2017). Discovering statistics using IBM SPSS Statistics: North American edition (5th ed.). Sage Publications, Inc.
Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19(5), 531–540. https://doi.org/10.1177/1049731509335549.
Fixsen, D., Naoom, S., Blase, K., Friedman, R., & Wallace, F. (2005). Implementation research: A synthesis of the literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network.
Glisson, C., & Williams, N. J. (2015). Assessing and changing organizational social contexts for effective mental health services. Annual Review of Public Health, 36(1), 507–523. https://doi.org/10.1146/annurev-publhealth-031914-122435.
Ireys, H. T., Brach, C., Anglin, G., Devers, K. J., & Burton, R. (2018). After the demonstration: What states sustained after the end of federal grants to improve children’s health care quality. Maternal and Child Health Journal, 22(2), 195–203. https://doi.org/10.1007/s10995-017-2391-z.
Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Association for Supervision and Curriculum Development.
KFF. (2020). Status of state Medicaid expansion decisions: Interactive map. Retrieved December 8, 2020, from https://www.kff.org/medicaid/issue-brief/status-of-state-medicaid-expansion-decisions-interactive-map/.
Lewis, C. C., Weiner, B. J., Stanick, C., & Fischer, S. M. (2015). Advancing implementation science through measure development and evaluation: A study protocol. Implementation Science, 10(1), 1–10. https://doi.org/10.1186/s13012-015-0287-0.
Locke, J., Lee, K., Cook, C. R., Frederick, L., Vázquez-Colón, C., Ehrhart, M. G., Aarons, G. A., Davis, C., & Lyon, A. R. (2019). Understanding the organizational implementation context of schools: A qualitative study of school district administrators, principals, and teachers. School Mental Health, 11(3), 379–399. https://doi.org/10.1007/s12310-018-9292-1.
Luke, D. A. (2019). Multilevel modeling (Vol. 143). Sage Publications, Inc.
Lyon, A. R., Pullmann, M. D., Whitaker, K., Ludwig, K., Wasse, J. K., & McCauley, E. (2019). A digital feedback system to support implementation of measurement-based care by school-based mental health clinicians. Journal of Clinical Child and Adolescent Psychology, 48(sup1), S168–S179. https://doi.org/10.1080/15374416.2017.1280808.
Lyon, A. R., Stirman, S. W., Kerns, S. E., & Bruns, E. J. (2011). Developing the mental health workforce: Review and application of training approaches from multiple disciplines. Administration and Policy in Mental Health and Mental Health Services Research, 38(4), 238–253. https://doi.org/10.1007/s10488-010-0331-y.
Magnabosco, J. L. (2006). Innovations in mental health services implementation: A report on state-level data from the U.S. Evidence-Based Practices Project. Implementation Science, 1, 13–11. https://doi.org/10.1186/1748-5908-1-13.
McCarthy, P. T. (2014). The road to scale runs through public systems. Stanford Social Innovation Review, 12(2).
McMorrow, S., Gates, J. A., Long, S. K., & Kenney, G. M. (2017). Medicaid expansion increased coverage, improved affordability, and reduced psychological distress for low-income parents. Health Affairs, 36(5), 808–818. https://doi.org/10.1377/hlthaff.2016.1650.
Miles, P., & Brown, N. (2011). & The National Wraparound Initiative Implementation Work Group. . The wraparound implementation guide.
Mitchell, T., & Eisendrath, S. J. (2002). Community mental health centers. In L. Breslow (Ed.), Encyclopedia of public health (Vol. 1, pp. 259–260). Macmillan Reference USA. https://link.gale.com/apps/doc/CX3404000209/GVRL?u=wash_main&sid=GVRL&xid=5d60f08b.
Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons, G. A. (2019). Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implementation Science, 14(1), 1. https://doi.org/10.1186/s13012-018-0842-6.
National Conference of State Legislatures. (2020). State partisan composition. Retrieved December 8, 2020, from https://www.ncsl.org/research/about-state-legislatures/partisan-composition.aspx#.
National Council for Behavioral Health. (2013). Community Mental Health Act 50th Anniversary Timeline. https://www.thenationalcouncil.org/about/national-mental-health-association/overview/community-mental-health-act/50th-anniversary-timeline_final_for_mag-2/#foobox-1/0/50th-Anniversary-Timeline_final_for_mag.jpg.
Pires, S. A. (2010). How states, tribes and localities are re-defining systems of care. Evaluation and Program Planning, 33(1), 24–27. https://doi.org/10.1016/j.evalprogplan.2009.05.008.
Powell, B. J., Beidas, R. S., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., & Mandell, D. S. (2017). Methods to improve the selection and tailoring of implementation strategies. The Journal of Behavioral Health Services and Research, 44(2), 177–194. https://doi.org/10.1007/s11414-015-9475-6.
Powell, B. J., Hausmann-Stabile, C., & McMillen, J. C. (2013). Mental health clinicians’ experiences of implementing evidence-based treatments. Journal of Evidence-Based Social Work, 10(5), 396–409. https://doi.org/10.1080/15433714.2012.664062.
Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., Glass, J. E., & York, J. L. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2), 123–157. https://doi.org/10.1177/1077558711430690.
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(1), 1–14. https://doi.org/10.1186/s13012-015-0209-1.
Proctor, E., Hooley, C., Morse, A., McCrary, S., Kim, H., & Kohl, P. L. (2019). Intermediary/purveyor organizations for evidence-based interventions in the US child mental health: Characteristics and implementation strategies. Implementation Science, 14(1), 3. https://doi.org/10.1186/s13012-018-0845-3.
Purtle, J., Dodson, E. A., Nelson, K., Meisel, Z. F., & Brownson, R. C. (2018). Legislators’ sources of behavioral health research and preferences for dissemination: Variations by political party. Psychiatric Services, 69(10), 1105–1108. https://doi.org/10.1176/appi.ps.201800153.
Purtle, J., Nelson, K. L., Counts, N. Z., & Yudell, M. (2020). Population-based approaches to mental health: History, strategies, and evidence. Annual Review of Public Health. https://doi.org/10.1146/annurev-publhealth-040119-094247.
Purtle, J., Peters, R., & Brownson, R. C. (2016). A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014. Implementation Science. https://doi.org/10.1186/s13012-015-0367-1.
Ratzliff, A., Unützer, J., Katon, W., & Stephens, K. A. (2016). Integrated care: Creating effective mental and primary health care teams. Wiley.
Rieckmann, T. R., Kovas, A. E., Cassidy, E. F., & McCarty, D. (2011). Employing policy and purchasing levers to increase the use of evidence-based practices in community-based substance abuse treatment settings: Reports from single state authorities. Evaluation and Program Planning, 34(4), 366–374. https://doi.org/10.1016/j.evalprogplan.2011.02.003.
So, M., McCord, R. F., & Kaminski, J. W. (2019). Policy levers to promote access to and utilization of children’s mental health services: A systematic review. Administration and Policy in Mental Health and Mental Health Services Research. https://doi.org/10.1007/s10488-018-00916-9.
Stroul, B. A., & Blau, G. M. (Eds.). (2008). The system of care handbook: Transforming mental health services for children, youth, and families. Paul H Brookes Publishing.
Stroul, B. A., & Friedman, R. M. (1994). A system of care for children and youth with severe emotional disturbances. (Revised). Georgetown University Child Development Center.
Sugai, G., & Horner, R. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child and Family Behavior Therapy, 24(1–2), 23–50. https://doi.org/10.1300/j019v24n01_03.
United States Census Bureau. (2020). Income. Retrieved December 8, 2020, from https://www.census.gov/programs-surveys/acs/.
Walker, J. S. (2017, June 12). A study of turnover among Wraparound care coordinators and supervisors. National Wraparound Initiative. https://nwi.pdx.edu/a-study-of-turnover-among-wraparound-care-coordinators-and-supervisors/.
Walker, J. S. & Bruns, E. J. (n.d.). The impact of training and technical assistance for wraparound. http://www.nwi.pdx.edu/pdf/IOTTA-results.pdf.
Walker, J. S., & Bruns, E. J. (2006). Building on practice-based evidence: Using expert perspectives to define the wraparound process. Psychiatric Services, 57, 127–154. https://doi.org/10.1176/ps.2006.57.11.1579.
Watson, D. P., Adams, E. L., Shue, S., Coates, H., McGuire, A., Chesher, J., Jackson, J., & Omenka, O. I. (2018). Defining the external implementation context: An integrative systematic literature review. BMC Health Services Research, 18(1), 209. https://doi.org/10.1186/s12913-018-3046-5.
Williams, N. J., Ehrhart, M. G., Aarons, G. A., Marcus, S. C., & Beidas, R. S. (2018). Linking molar organizational climate and strategic implementation climate to clinicians’ use of evidence-based psychotherapy techniques: Cross-sectional and lagged analyses from a 2-year observational study. Implementation Science, 13(1), 85. https://doi.org/10.1186/s13012-018-0781-2.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest.
Rights and permissions
About this article
Cite this article
Olson, J.R., Azman, A., Estep, K.M. et al. Influences of Inner and Outer Settings on Wraparound Implementation Outcomes. Glob Implement Res Appl 1, 77–89 (2021). https://doi.org/10.1007/s43477-021-00008-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s43477-021-00008-1