Skip to main content

Advertisement

SpringerLink
Log in

Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians’ EBP Adoption Through Improved Organizational Culture and Capacity

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

The development of efficient and scalable implementation strategies in mental health is restricted by poor understanding of the change mechanisms that increase clinicians’ evidence-based practice (EBP) adoption. This study tests the cross-level change mechanisms that link an empirically-supported organizational strategy for supporting implementation (labeled ARC for Availability, Responsiveness, and Continuity) to mental health clinicians’ EBP adoption and use. Four hundred seventy-five mental health clinicians in 14 children’s mental health agencies were randomly assigned to the ARC intervention or a control condition. Measures of organizational culture, clinicians’ intentions to adopt EBPs, and job-related EBP barriers were collected before, during, and upon completion of the three-year ARC intervention. EBP adoption and use were assessed at 12-month follow-up. Multilevel mediation analyses tested changes in organizational culture, clinicians’ intentions to adopt EBPs, and job-related EBP barriers as linking mechanisms explaining the effects of ARC on clinicians’ EBP adoption and use. ARC increased clinicians’ EBP adoption (OR = 3.19, p = .003) and use (81 vs. 56 %, d = .79, p = .003) at 12-month follow-up. These effects were mediated by improvement in organizational proficiency culture leading to increased clinician intentions to adopt EBPs and by reduced job-related EBP barriers. A combined mediation analysis indicated the organizational culture-EBP intentions mechanism was the primary carrier of ARC’s effects on clinicians’ EBP adoption and use. ARC increases clinicians’ EBP adoption and use by creating proficient organizational cultures that increase clinicians’ intentions to adopt EBPs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  • Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274.

    Article  PubMed  PubMed Central  Google Scholar 

  • Aarons, G. A., Glisson, C., Green, P. D., Hoagwood, K., Kelleher, K. J., Landsverk, J. A., & The Research Networkon Youth Mental Health. (2012). The organizational social context of mental health services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science, 7(1), 56.

    Article  PubMed  PubMed Central  Google Scholar 

  • Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public sectors. Administration and Policy in Mental Health, 38, 4–23.

    Article  PubMed  Google Scholar 

  • Aarons, G. A., Sommerfeld, D. H., & Walrath-Greene, C. M. (2009). Evidence-based practice implementation: The impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implementation Science, 4(1), 83.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179–211.

    Article  Google Scholar 

  • Baer, J. S., Wells, E. A., Rosengren, D. B., Hartzler, B., Beadnell, B., & Dunn, C. (2009). Agency context and tailored training in technology transfer: A pilot evaluation of motivational interviewing training for community counselors. Journal of Substance Abuse Treatment, 37, 191–202.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bartholomew, N. G., Joe, G. W., Rowan-Szal, G. A., & Simpson, D. D. (2007). Counselor assessments of training and adoption barriers. Journal of Substance Abuse Treatment, 33, 193–199.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246.

    Article  Google Scholar 

  • Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013). Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science, 8(1), 22.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale: Erlbaum.

    Google Scholar 

  • Collins, P. Y., Patel, V., Joestl, S. S., March, D., Insel, T. R., Daar, A. S., et al. (2011). Grand challenges in global mental health. Nature, 475, 27–30.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Cooke, R. A., & Rousseau, D. M. (1988). Behavioral norms and expectations: A quantitative approach to the assessment of organizational culture. Group & Organization Studies, 13, 245–273.

    Article  Google Scholar 

  • Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fishbein, M., Triandis, H. C., Kanfer, F. H., Becker, M., Middlestadt, S. E., & Eichler, A. (2001). Factors influencing behavior and behavior change. In A. Baum, T. Revenson, & J. Singer (Eds.), Handbook of health psychology (pp. 3–17). Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Gioia, D., & Dziadosz, G. (2008). Adoption of evidence-based practices in community mental health: A mixed methods study of practitioner experience. Community Mental Health Journal, 44, 347–357.

    Article  PubMed  Google Scholar 

  • Glisson, C., Dukes, D., & Green, P. (2006). The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse and Neglect, 30, 855–880.

    Article  PubMed  Google Scholar 

  • Glisson, C., Green, P., & Williams, N. J. (2012a). Assessing the organizational social context (OSC) of child welfare systems: Implications for research and practice. Child Abuse and Neglect, 36, 621–632.

    Article  PubMed  Google Scholar 

  • Glisson, C., Hemmelgarn, A., Green, P., Dukes, D., Atkinson, S., & Williams, N. J. (2012b). Randomized trial of the availability, responsiveness, and continuity (ARC) organizational intervention with community-based mental health programs and clinicians serving youth. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 780–787.

    Article  PubMed  Google Scholar 

  • Glisson, C., Hemmelgarn, A., Green, P., & Williams, N. J. (2013). Randomized trial of the availability, responsiveness and continuity (ARC) organizational intervention for improving youth outcomes in community mental health programs. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 493–500.

    Article  PubMed  Google Scholar 

  • Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., et al. (2008a). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health, 35, 98–113.

    Article  PubMed  Google Scholar 

  • Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78, 537–550.

    Article  PubMed  PubMed Central  Google Scholar 

  • Glisson, C., Schoenwald, S. K., Kelleher, K., Landsverk, J., Hoagwood, K. E., Mayberg, S., et al. (2008b). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy In Mental Health, 35, 124–133.

    Article  PubMed  Google Scholar 

  • Glisson, C., Williams, N. J., Green, P., Hemmelgarn, A., & Hoagwood, K. (2014). The organizational social context of mental health Medicaid waiver programs with family support services: Implications for research and practice. Administration and Policy In Mental Health, 41, 32–42.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hartnell, C. A., Ou, A. Y., & Kinicki, A. (2011). Organizational culture and organizational effectiveness: A meta-analytic investigation of the competing values framework’s theoretical suppositions. Journal of Applied Psychology, 96, 677–694.

    Article  PubMed  Google Scholar 

  • Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, 24, 1918–1927.

    Article  PubMed  Google Scholar 

  • Hofstede, G., Neuijen, B., Ohayv, D. D., & Sanders, G. (1990). Measuring organizational cultures: A qualitative and quantitative study across twenty cases. Administrative Science Quarterly, 35, 286–316.

    Article  Google Scholar 

  • Hogue, A., Dauber, S., Henderson, C. E., & Liddle, H. A. (2014). Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy In Mental Health, 41, 697–705.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health, 37, 230–244.

    Article  PubMed  Google Scholar 

  • Insel, T. R. (2009). Translating scientific opportunity into public health impact. Archives of General Psychiatry, 66, 128–133.

    Article  PubMed  Google Scholar 

  • Jaccard, J., Litardo, H. A., & Wan, C. K. (1999). Subjective culture and social behavior. In J. Adamopoulos & Y. Kashima (Eds.), Social psychology and cultural context (pp. 95–106). Thousand Oaks: Sage publications.

    Chapter  Google Scholar 

  • James, L. R., Demaree, R. G., & Wolf, G. (1993). Rwg: An assessment of within-group agreement. Journal of Applied Psychology, 78, 306–309.

    Article  Google Scholar 

  • Jensen-Doss, A., Hawley, K. M., Lopez, M., & Osterberg, L. D. (2009). Using evidence-based treatments: The experiences of youth providers working under a mandate. Professional Psychology: Research and Practice, 40, 417–424.

    Article  Google Scholar 

  • Larson, E. L., Early, E., Cloonan, P., Sugrue, S., & Parides, M. (2000). An organizational climate intervention associated with increased handwashing and decreased nosocomial infections. Behavioral Medicine, 26, 14–22.

    Article  CAS  PubMed  Google Scholar 

  • LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852.

    Article  Google Scholar 

  • MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104.

    Article  PubMed  PubMed Central  Google Scholar 

  • MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128.

    Article  PubMed  PubMed Central  Google Scholar 

  • Michie, S., van Stralen, M. M., & West, R. (2011). The behavior change wheel: A new method for characterizing and designing behavior change interventions. Implementation Science. doi:10.1186/1748-5908-6-42.

    Google Scholar 

  • National Institute of Mental Health. (2008). National Institute of Mental Health strategic plan. Washington, DC: U. S. Department of Health and Human Services.

    Google Scholar 

  • Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 1009–1025.

    Article  PubMed  PubMed Central  Google Scholar 

  • Olin, S. S., Williams, N., Pollock, M., Armusewicz, K., Kutash, K., Glisson, C., & Hoagwood, K. E. (2014). Quality indicators for family support services and their relationship to organizational social context. Administration and Policy in Mental Health and Mental Health Services Research, 41, 43–54.

    Article  PubMed  Google Scholar 

  • Parmelli, E., Flodgren, G., Beyer, F., Baillie, N., Schaafsma, M. E., & Eccles, M. P. (2011). The effectiveness of strategies to change organizational culture to improve healthcare performance: A systematic review. Implementation Science, 6(1), 33.

    Article  PubMed  PubMed Central  Google Scholar 

  • Podsakoff, P. M., MacKenzie, S. B., Lee J., & Podsakoff, N. P. (2003). Common method bias in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903.

    Article  PubMed  Google Scholar 

  • Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., et al. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69, 123–157.

    Article  PubMed  Google Scholar 

  • Powell, B. J., Proctor, E. K., & Glass, J. E. (2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research on Social Work Practice, 24, 192–212.

    Article  PubMed  Google Scholar 

  • Preacher, K. J., & Kelley, K. (2011). Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological Methods, 16, 93–115.

    Article  PubMed  Google Scholar 

  • Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6, 77–98.

    Article  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., & Hensley, M. (2010). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38, 65–76.

    Article  PubMed Central  Google Scholar 

  • Raghavan, R., Inkelas, M., Franke, T., & Halfon, N. (2007). Administrative barriers to the adoption of high-quality mental health services for children in foster care: A national study. Administration and Policy in Mental Health and Mental Health Services Research, 34, 191–201.

    Article  PubMed  Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis (2nd ed.). Thousand Oaks: Sage publications.

    Google Scholar 

  • Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

    Google Scholar 

  • Schein, E. H. (2004). Organizational culture and leadership. San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., et al. (2008). A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Administration and Policy In Mental Health, 35, 84–97.

    Article  PubMed  Google Scholar 

  • Schulte, M., Ostroff, C., Shmulyian, S., & Kinicki, A. (2009). Organizational climate configurations: Relationships to collective attitudes, customer satisfaction, and financial performance. Journal of Applied Psychology, 94, 618–634.

    Article  PubMed  Google Scholar 

  • Sheeran, P. (2002). Intention-behavior relations: A conceptual and empirical review. In W. Stroebe & M. Hewstone (Eds.), European review of social psychology (Vol. 12, pp. 1–36). London: Wiley.

    Google Scholar 

  • Taylor, A. B., MacKinnon, D. P., & Tein, J. Y. (2008). Tests of the three-path mediated effect. Organizational Research Methods, 11, 241–269.

    Article  Google Scholar 

  • Trice, H. M., & Beyer, J. M. (1993). The cultures of work organizations. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • Walrath, C. M., Sheehan, A. K., Holden, E. W., Hernandez, M., & Blau, G. (2006). Evidence-based treatments in the field: A brief report on provider knowledge, implementation, and practice. Journal of Behavioral Health Services Research, 33, 244–253.

    Article  PubMed  Google Scholar 

  • Williams, N. J. (2015). Multilevel mechanisms of implementation strategies in mental health: Integrating theory, research, and practice. Administration and Policy in Mental Health and Mental Health Services Research,. doi:10.1007/s10488-015-0693-2.

    PubMed Central  Google Scholar 

  • Williams, N. J., & Glisson, C. (2013). Reducing turnover is not enough: The need for proficient organizational cultures to support positive youth outcomes in child welfare. Children and Youth Services Review, 35, 1871–1877.

    Article  Google Scholar 

  • Williams, N. J., & Glisson, C. (2014a). Testing a theory of organizational culture, climate, and youth outcomes in child welfare systems: A United States national study. Child Abuse and Neglect, 38, 757–767.

    Article  PubMed  Google Scholar 

  • Williams, N. J., & Glisson, C. (2014b). The role of organizational culture and climate in the dissemination and implementation of empirically-supported treatments for youth. In R. S. Beidas & P. C. Kendall (Eds.), Dissemination and implementation of evidence based practices in child and adolescent mental health (pp. 61–81). New York: Oxford University Press.

    Google Scholar 

  • Zhang, Z., Zyphur, M. J., & Preacher, K. J. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695–719.

    Article  Google Scholar 

  • Zohar, D., & Polachek, T. (2014). Discourse-based intervention for modifying supervisory communication as leverage for safety climate and performance improvement: A randomized field study. Journal of Applied Psychology, 99, 113–124.

    Article  PubMed  Google Scholar 

Download references

Funding

This study was funded by grants from the National Institute of Mental Health to NJW (F31MH099846) and CG (R01MH084855). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health.

Author information

Authors and Affiliations

  1. School of Social Work, Boise State University, 1910 University Dr., Boise, ID, 83725-1940, USA

    Nathaniel J. Williams

  2. Children’s Mental Health Services Research Center, University of Tennessee, Knoxville, TN, USA

    Charles Glisson, Anthony Hemmelgarn & Philip Green

Authors
  1. Nathaniel J. Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charles Glisson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anthony Hemmelgarn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip Green
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nathaniel J. Williams.

Ethics declarations

Conflict of interest

Nathaniel Williams declares that he has no conflict of interest. Charles Glisson declares that he has no conflict of interest. Anthony Hemmelgarn declares that he has no conflict of interest. Philip Green declares that he has no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Williams, N.J., Glisson, C., Hemmelgarn, A. et al. Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians’ EBP Adoption Through Improved Organizational Culture and Capacity. Adm Policy Ment Health 44, 269–283 (2017). https://doi.org/10.1007/s10488-016-0742-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-016-0742-5

Keywords

Search

Navigation