Interagency Collaborative Team model for capacity building to scale-up evidence-based practice

Highlights

• A process model for evidence-based practice implementation is presented.

• Qualitative research examines the implementation of SafeCare in one large county.

• The implementation model generated planned structural supports for implementation.

• Stakeholder interviews clarified strengths and limitations in the implementation model.

• Integrated steps at system and practice levels are key to effective implementation.

Abstract

Background

System-wide scale up of evidence-based practice (EBP) is a complex process. Yet, few strategic approaches exist to support EBP implementation and sustainment across a service system. Building on the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework, we developed and are testing the Interagency Collaborative Team (ICT) process model to implement an evidence-based child neglect intervention (i.e., SafeCare®) within a large children's service system. The ICT model emphasizes the role of local agency collaborations in creating structural supports for successful implementation.

Methods

We describe the ICT model and present preliminary qualitative results from the use of the implementation model in one large scale EBP implementation. Qualitative interviews were conducted to assess challenges in building system, organization, and home visitor collaboration and capacity to implement the EBP. Data collection and analysis centered on EBP implementation issues, as well as the experiences of home visitors under the ICT model.

Results

Six notable issues relating to implementation process emerged from participant interviews, including: (a) initial commitment and collaboration among stakeholders, (b) leadership, (c) communication, (d) practice fit with local context, (e) ongoing negotiation and problem solving, and (f) early successes. These issues highlight strengths and areas for development in the ICT model.

Conclusions

Use of the ICT model led to sustained and widespread use of SafeCare in one large county. Although some aspects of the implementation model may benefit from enhancement, qualitative findings suggest that the ICT process generates strong structural supports for implementation and creates conditions in which tensions between EBP structure and local contextual variations can be resolved in ways that support the expansion and maintenance of an EBP while preserving potential for public health benefit.

Introduction

Introduction of evidence-based practices (EBPs) can lead to substantial public health benefits. However, the implementation process can shape whether intended outcomes are actually achieved (Aarons and Palinkas, 2007, Allen et al., 2012, Crea et al., 2008, Fixsen et al., 2005, Greenhalgh et al., 2004, Palinkas and Aarons, 2009). Well-established practice models, implemented poorly or not sustained, will fail to achieve intended goals despite research evidence supporting their clinical effectiveness (Backer, 2000, Bond et al., 2009). Thus, an effective implementation approach is often as important as the practice to be utilized.

Several conceptual models describe factors that can influence implementation effectiveness. Some models emphasize structural features hypothesized to be core components of effective implementation (Aarons et al., 2011, Damschroder et al., 2009, Feldstein and Glasgow, 2008, Greenhalgh et al., 2004, Mendel et al., 2008). Other models emphasize implementation processes, outlining key steps (and their timing) hypothesized to contribute to successful implementation of service innovations (Glisson and Schoenwald, 2005, Sosna and Marsenich, 2006, Stetler et al., 2008). Structural and process implementation models are often conceptually aligned. For example, both types of models address the central importance of issues such as strong and effective leadership to support change initiatives, establishing a strong fit between change efforts and organizational and service system culture and values, creating methods for ensuring quality program delivery (i.e., fidelity), and clarifying/addressing financial supports for a change initiative.

This paper describes the Interagency Collaborative Team (ICT) implementation process model. The ICT model provides an approach to support successful roll-out of human service innovations in large geographic areas, particularly change efforts involving EBPs. It is directly relevant to improving outcomes of service enhancements in child and family service systems. The ICT model is designed to enable organizations to work together in ways that generate the structural and process supports associated with successful implementation and sustainment of innovations. We discuss some core areas of difference and similarity between the ICT model and other implementation strategies, connecting core features to one structural implementation framework, the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework (Aarons et al., 2011). Qualitative data from the scale-up of an EBP in one large county illustrate areas of strength and some limitations in the ICT model and provide perspective on other process models of EBP implementation.

Like a number of implementation frameworks, the EPIS framework summarizes variables that can positively or negatively affect the implementation of an evidence-based practice. The EPIS framework is unusual in identifying key variables thought to particularly affect implementation efforts during each of four major implementation stages in public sector child welfare and mental health settings. For example, some key variables identified as influencing the preparation and early implementation stages of a quality improvement effort include strength of the leadership supporting change (Aarons, 2006, Edmondson, 2004, Klein et al., 2001), the degree of fit of an innovation with the service system context (Klein & Sorra, 1996), clarity of financial support for proposed changes (Aarons et al., 2009, Frambach and Schillewaert, 2002), level of involvement of practice developers in the implementation process (Aarons et al., 2011), and the presence of cross-organizational knowledge of and commitment to the new practice (Glisson and Schoenwald, 2005, Sosna and Marsenich, 2006).

The ICT implementation process model outlines steps designed to lead directly to the kinds of key implementation supports described in the EPIS framework. The model takes its name from the fact that it emphasizes the key role of collaboration among stakeholders and staff members at the system level, from multiple partnering organizations, and of developing or utilizing a local “seed” team to embody and support promotion and maintenance of expertise and ongoing fidelity in the practice to be implemented. Inter-agency collaboration and willingness to share expertise is central to multiple steps in the implementation process and across organizational levels. Conceptually, the ICT model has much in common with other implementation process models (Chamberlain et al., 2008, Glisson and Schoenwald, 2005, Sosna and Marsenich, 2006), which describe logically ordered sets of activities designed to create a context in which EBP implementation occurs effectively and intended public health benefits are realized.

Fig. 1 provides a graphical representation of key implementation processes included in the ICT model, with the stages of the EPIS framework listed temporally down the left side of the figure. In the ICT model, a process is considered to be a goal-driven domain of focus that extends over a period of time within the longer implementation effort. For example, the initial EBP education and stakeholder development and alignment processes involve an initial phase of identifying community-based stakeholders with interests in a particular practice change effort, and discussions and education efforts designed to lead to joint selection of and commitment to a common practice change initiative. The practice fit assessment process involves a careful analysis by key stakeholders at system and organizational levels of EBPs under consideration to identify aspects of practices that fit with existing policies, contracting, and service routines and those where modifications might be required. Brief descriptions of each ICT process are provided at the bottom of Fig. 1.

Specific ICT model action steps are listed in Table 1 that animate the processes shown in Fig. 1. Their contributions to each implementation process are noted in the figure. For example, the Initial EBP Education process occurs as part of ICT action steps A (convening of stakeholders) and B (soliciting expertise). Education about the EBP becomes an intense process focus that occurs in the context of meetings among interested stakeholders, supplemented by expertise about the EBP solicited from appropriate sources. Sources may be multiple, including EBP developers, other users of the EBP, researchers having familiarity with the practice, and/or materials available from sources such as journals or intermediary organizations that summarize information about EBPs. Structural supports designed to arise from the ICT processes are represented as planks beneath the model processes that generate them. We represent the ICT model in this manner because it is best conceptualized as a series of major actions that address core implementation processes. Specific action steps animate these processes and give rise to or strengthen key structural supports viewed as creating an environment that can sustain an innovative practice as it is scaled up.

The ICT model initially revolves around a service system and multi-agency commitment to invest in the long-term viability of an EBP-centered quality improvement initiative, with an ultimate goal to improve selected client level outcomes. Partnering agencies may include a range of stakeholder organizations, but particularly involve funding, administration, and service delivery organizations from the outset. During an initial exploration phase, stakeholders convene and meet to discuss need for a practice change effort that involves investment by multiple individuals and organizations. Although no specified leader is required to initiate such meetings, it is expected that one local or regional organization will often take responsibility for convening and leading such discussions. For example, a health and human service administration may convene discussions around maltreatment prevention, reduction in delinquency, or some other practice change effort. Within an ICT model-guided implementation, convening of stakeholders should include efforts to identify those stakeholders with substantial interests in the identified substantive area (e.g., child neglect).

A second important step in the process of exploring a possible practice change involves concentrated efforts to obtain wide-ranging factual information about the costs, benefits, and tradeoffs associated with specific practice changes. Outside expertise is identified and sought to help answer questions and reduce uncertainty about the change effort under discussion. The joint process of participating in education about possible practice change efforts and discussing the advantages and disadvantages of various options is aimed at developing a shared commitment and direction among stakeholders at an inter-agency level to a jointly supported EBP implementation.

Once a specific EBP is selected as the focal point for a broad practice change effort, stakeholders in the ICT process initiate implementation of the EBP by creating a formative interagency collaborative “seed” team (or ICST), which may consist of employees from several different local organizations that form a core unit of expertise in the selected service model. A seed team intentionally involves multiple organizations in the maintenance of innovation expertise to build broader investment in, commitment to, and communication about an innovation among invested stakeholders and subsequently trained practitioners. The seed team becomes a repository of local expertise for an EBP. It is designed to serve as the ongoing support structure for continued EBP training, coaching, and roll-out across a geographic area and as a facilitator of minor practice adjustments that help to fit a practice to diverse regional contexts (Aarons et al., 2012). Members of the seed team maintain a central liaison role between the EBP developer and other actors at multiple levels within the service system so that issues, decisions, and adaptations can be negotiated during initial experimentation with implementation.

Following inception of the seed team and initial EBP training, certification, and service delivery, the team is then responsible for training and supporting additional teams of individuals that can then implement the selected EBP as it is scaled up across a service system. The seed team assumes responsibility for ongoing training of new teams of practitioners that may consist of employees from several local non-profit organizations, hereafter referred to as Interagency Collaborative Teams (ICTs). These ICTs form for the express purpose of learning and mastering delivery of the EBP to be implemented, under the guidance of the original seed team. Although members include staff from multiple organizations, they meet together with a seed team coach during a supervision and knowledge transfer phase. ICTs trained by the seed team are responsible for the primary delivery of the EBP. This implementation structure, with regional teams having interagency composition, results in a network of local providers that allows for high inter-agency communication, and information and possible workload sharing.

The seed team maintains relationships with ICTs following EBP training. In order to maintain and continually enhance quality delivery of the EBP, newly trained providers continue to receive constructive support and feedback in the form of supervision and coaching from seed team members for a defined period of time, which may vary by practice or situation. In the case of SafeCare, ongoing fidelity monitoring and coaching are integral components of the EBP. Organizing supervision and coaching through the seed team has many potential benefits in the short and longer term. This structure is designed to provide a clear source of leadership and information to newly trained providers. Ideally having the seed team serve as the source of ongoing training and coaching facilitates a gradual reduction of EBP developer involvement whereby the local service system and its contracted agencies become the repository of expertise in the practice being adopted. Based on its initial key role within a multi-organizational implementation effort, the seed team continues to serve as a locus of information about needs for adaptations to make a practice work within a particular local context. Such adaptations may involve changes to aspects of the intervention itself or to the structure of the service setting in which the EBP is delivered (e.g., Finno-Velasquez, Fettes, Aarons, & Hurlburt, under review). Cross-organizational membership on the seed team contributes to ensuring a continuing locus of expertise available to all organizations within the ICT partnership, reducing the kinds of expertise loss that regularly occur within individual organizations and agencies due to staff turnover and organizational changes.

As noted above, the ICT model seeks to foster implementation supports in the areas of practice fit, leadership, communication, expertise distribution, EBP quality (fidelity) management, effective developer involvement, and program adaptation. A large California county used the ICT model to implement an evidence-based neglect prevention program throughout the county. Qualitative inquiry into this effort helps to illustrate several key elements of the implementation approach.

Implementation of the SafeCare neglect prevention model occurred in one large California county. With approximately 3 million total residents, the county population is similar in size or larger than that of many smaller U.S. states. The county encompasses urban, semi-urban, and rural areas that are home to a diverse cultural mix of residents, including significant Mexican-American and Native American populations. Planning for many aspects of public human services in the county is organized into local planning regions, each with some of its own local history, demographic and cultural characteristics, climate, and topography. Implementation of any new practice at a county level represents a large-scale system and organizational change effort that occurs across the planning regions.

In 2007, the Department of Health and Human Services (DHHS) agency and the local chapter of a national foundation embarked upon an effort to transition one category of county maltreatment prevention services toward an EBP. County DHHS leaders (responsible for child welfare services), members of the local branch of a national foundation, and research partners convened to consider three different child focused EBPs to improve outcomes for children and families involved with the child welfare system. After consideration of research evidence, programmatic fit, and financial resources required, SafeCare®, an evidence-based child neglect prevention program utilizing home visiting (Chaffin et al., 2012, Lutzker et al., 1998), was selected for implementation.

Qualitative interviews and focus groups allowed us to document the roll-out of SafeCare, provided insight into how the ICT model generated key structural supports for implementation, and helped to identify process issues worthy of more careful consideration. The following section summarizes at a general level what we learned from that qualitative work about themes related to implementation process, including: (a) initial commitment and collaboration among stakeholders, (b) leadership, (c) communication, (d) practice fit with local context, (e) ongoing negotiation and problem solving, and (f) early successes.