Introduction

There are two fundamental purposes of clinical supervision in the social work field: to safeguard the client’s welfare, and to promote the professional development of supervisees (Bogo and McKnight 2006; Kadushin and Harkness 2014). These goals are accomplished by attending to the three functions that typically occur during clinical supervision: administrative, educational, and supportive functions (Kadushin and Harkness 2014). In order to have success in clinical supervision, supervisors must be aware of and elicit feedback about the supervisory working alliance from their direct reports or supervisees (Bordin 1983). The term “working alliance” refers to supportive relationships formed in helping professions, and is defined as mutual respect, appreciation, and trust between service providers and individuals in care (Bordin 1979). Although Bordin’s initial work examined the therapeutic relationship between service providers and individuals in care, his model was later extended to include the supervisory relationship (Bordin 1983), which is particularly applicable for social workers. In this model, the working alliance is composed of three components: the extent to which there is agreement on goals, agreement on the tasks necessary to reach the goals, and the bond that develops between two people working together (Bordin 1983). Research on the working alliance indicates that healthy relationships between service providers and the people they serve leads to improved treatment outcomes (Horvath 2006), and these improved outcomes occur regardless of the type of service provided or modality of treatment. The same phenomenon for the supervisory working alliance has been examined in both social work and counseling fields to determine the impact on supervisee outcomes (Bennett et al. 2013; Ladany and Friedlander 1995).

Supervisory Working Alliance and Worker Outcomes

Due to the relationship between the working alliance and improved client outcomes, researchers have also examined the relationship between the supervisory working alliance and improvements in worker outcomes. Researchers have focused on outcomes that are related to the worker’s responsibilities, such as disclosing information, adhering to treatment models, and understanding their job roles. There has also been a focus on outcomes that benefit clients such as seeing improved outcomes for those who are provided services as well as outcomes that benefit the organization and the worker, such as satisfaction, retention, self-efficacy, stress, and burnout. The relevance for examining these worker outcomes in the context of the supervisory working alliance and the results from previous research are highlighted in this section.

Disclosure of Information

A critical component for social workers in their professional role and to aid in ongoing professional development is their willingness to disclose information to supervisors about what supervisees still need to learn (Kadushin and Harkness 2014). However, researchers have found supervisees vary in their willingness to reveal their shortcomings and other counseling-related observations (Ladany et al. 1996). Having a stronger supervisory working alliance predicts supervisees’ willingness to report sensitive material to their supervisors (Webb and Wheeler 1998). Failing to disclose information not only hinders the supervisees’ learning process, but it also can put the supervisor at risk due to vicarious liability whereby the supervisor is liable for any of the supervisees’ unethical or illegal activities (Bernard and Goodyear 2009; Kadushin and Harkness 2014).

Treatment Model Adherence

Researchers have also found therapists typically dislike manualized treatments and social workers have reservations about the relevance, usability, and applicability of evidence based practices (Gray et al. 2014); however, fluctuations in the supervisory alliance account for a substantial portion of the fluctuations in adherence to treatment protocols (Patton and Kivlighan 1997). Therefore, those who have a stronger working alliance with their supervisor are more likely to use a treatment model as it is prescribed with their clients.

Role Clarity

Another area of importance is the impact of the supervisory working alliance on supervisees’ role conflict and ambiguity, which is specifically apparent during the onboarding process of new supervisees. According to role theory, role conflict and ambiguity occurs when supervisees perceive their role and have a set of expectations that differ from what the supervisor or organization defines and expects of their role (Beehr 1995). Supervisees who have a greater emotional bond with their supervisor experience less role conflict (Ladany and Friedlander 1995), and overall working alliance predicts lower levels of role conflict and role ambiguity (Son et al. 2007).

Impact on Client Outcomes

Supervision is a unique intervention that involves administrative, educational, and supportive functions using the context of a positive supervisor–supervisee relationship, but also includes a phenomenon referred to as the parallel process (Kadushin and Harkness 2014). The parallel process occurs when supervisors model their interactions with social workers in the way they should ideally interact with clients (Shulman 2010). Research has examined the relationship between providing clinical supervision and its impact on positive client outcomes due to the parallel process (Crowe et al. 2011). If the supervisory working alliance is strong between supervisor and supervisee, the working alliance between the supervisee and client may also be strong. This relational domino effect has been hypothesized to have a positive influence on the outcomes a client experiences. However, a systematic review in child welfare found the evidence for supervision outcomes is lacking due to weak study designs, limited samples, and varying definitions of supervision (Carpenter et al. 2013).

Satisfaction and Retention

Job turnover is a topic of concern in the child welfare and social work field, as turnover can have an impact on continuity and quality of care which can then have an effect on an organization’s effectiveness (Chen and Scannapieco 2010; Cho and Song 2017). Therefore, organizations have searched for effective solutions for staff retention, and improving job satisfaction has been used as a solution for retention (Chen and Scannapieco 2010). One important factor is the relationship between having a strong supervisory working alliance and greater satisfaction with supervision (Ladany et al. 1996), as well as overall job satisfaction and well-being (Mor Barak et al. 2009). The supportive relationship offered through supervision has been found to improve job satisfaction and intention to stay with an organization (Kim and Lee 2009).

Self-Efficacy

Although previous research has focused on the importance of job satisfaction for worker retention, there has been less focus on the interaction effect between job satisfaction and other retention relevant factors (Chen and Scannapieco 2010). For example, having high self-efficacy where supervisees believe they are capable of performing professional work roles can have an impact on ameliorating burnout (Chen and Scannapieco 2010). Researchers have found that a supervisor’s support was particularly important to retain workers of low self-efficacy, and that improving job satisfaction may not be a universal approach for worker retention due to the influence of worker’s self-efficacy (Chen and Scannapieco 2010). In addition, when supervisors are socially and emotionally supportive to supervisees, the supervisee’s self-efficacy is related to intention to stay with an organization (Cho and Song 2017).

Stress and Burnout

Stress and burnout have also been related to supervisee’s self-efficacy in that having high self-efficacy toward work is related to lower emotional exhaustion which in turn was found to be an important factor related to job exits (Sterner 2009). Among supervisees, having a positive supervisory alliance has been related to experiencing less stress at work (Gnilka et al. 2012; Sterner 2009). Previous studies have found that supervisory variables, including supportive communication from supervisors and high-quality supervisory relationships, buffer the effect stress can have on emotional exhaustion (Kim and Lee 2009). Burnout has been found to be associated with a supervisee’s perception of the supervisory relationship, not just whether they received help or support (Mena and Bailey 2007). When supervisees reported feeling a negative supervisory relationship, this was associated with both emotional exhaustion and depersonalization, where supervisees can feel detached, devalued, and focus less on their personal needs (Mena and Bailey 2007).

Supervisory Activities and Supervisory Working Alliances

The following supervisory activities are more effective in building a strong supervisory working alliance and reducing the negative effects of stressful work environments that exist in the social service field (Beidas et al. 2013). For example, organizational commitment to provide supervision is essential and can be reinforced through a written policy that incorporates input from all levels of the organization (Hoge et al. 2014). Additional activities have also been recommended in order to improve the quality of supervision, such as (a) adopting an informed consent process that describes the purpose and frequency of supervision, (b) outlining best practices for documenting supervision sessions, (c) specifying the minimal requirements for supervision by job categories, (d) establishing qualifications and preparation for supervisors, and (e) developing procedures for assessing supervision quality (Hoge et al. 2014).

Additional techniques to help overcome common challenges inherent in the supervisory relationship include supervision contracts, role induction procedures, collaborative goal setting, giving feedback, and evaluating the supervisory relationship (Bernard and Goodyear 2009; Shulman 2010). Supervision contracts, similar to informed consent procedures, can help to set expectations and clarify roles during supervision sessions. Role induction procedures can also be used to clarify supervisees’ expectations and understanding of the types of information that should be shared during supervision. Collaborative goal setting helps to understand the professional development needs of supervisees and also reduces resistance to the supervision process. Supervisee resistance can be problematic and can manifest in several ways during supervision through a supervisee resisting the supervisor’s influence, the experience itself, the tasks related to the supervisory process, or the plan for implementing particular client interventions (Bernard and Goodyear 2009; Munson 2012). Therefore, there should also be opportunities to give and receive feedback about the effectiveness of supervision sessions in order to assess supervision quality and to proactively address any conflicts or uncertainties about the process.

To help supervisors gather feedback about supervision sessions, Pressley Ridge developed the Alliance Building: Learning to Engage (ABLE) model. ABLE was developed to be used by any person in the helping professions, regardless of the type of service provided. The goal of ABLE in clinical supervision is to elicit feedback from supervisees on the supervisory working alliance using a short feedback tool to facilitate a constructive dialogue. Using ABLE requires supervisors to routinely assess supervisees’ perceptions regarding their agreement with the focus and format of the current supervision session using a supervision tool (ABLE-S). This feedback is then used to refine or revise current supervision experiences in real time. The intended end result is an open and continual dialogue that fosters buy-in towards supervision and may help to reduce supervisee resistance to the supervision process.

Current Study and Hypotheses

While social workers and researchers acknowledge the importance of a positive working alliance between service providers and clients, less is known about mechanisms for developing working alliances between supervisors and supervisees. The majority of studies examining the working alliance use lengthy assessments to examine the supervisory working alliance that are not conducive to having a dialogue about the scores in supervision sessions (Bennett et al. 2013; Palomo et al. 2010; Pearce et al. 2013; Smith et al. 2002), with only one study using a short feedback tool only for purposes of psychometric validation (Wainwright 2010). No studies have examined the use of a short, real-time feedback tool about the supervisory working alliance in supervision sessions and its impact on supervisee outcomes. In addition, the majority of studies focus on counselor trainees in university settings with minimal research on paid social workers and counseling staff who are working in social service organizations (e.g., Ladany and Friedlander 1995; Mehr et al. 2010).

Therefore, this two-part study aims to determine (a) if a short feedback tool is a valid and reliable measurement of the supervisory working alliance, and (b) whether the use of a short feedback tool increased supervisees’ outcomes by examining their occupational self-efficacy, and professional quality of life (compassion satisfaction, and decreased burnout and traumatic stress) compared to a control group. These two outcomes were chosen based on research findings that tied these outcomes to decreasing staff turnover, and the organization’s interest in retaining staff through examining the supervisory relationship as a key factor in impacting turnover. It was hypothesized that the use of the feedback tool would have a positive effect on these outcomes over time compared to the control group that did not use the feedback tool.

Study Design

A multimethod approach was used for this study. First, we collected data in order to assess the reliability and validity of the ABLE-S by comparing items on the ABLE-S against similar constructs of the Working Alliance Inventory for supervisees (WAI-T; Bahrick 1989). Second, a randomized control trial design was used to determine if the use of ABLE in supervision improves professional quality of life and occupational self-efficacy. Participation in both studies was voluntary; potential participants were given the choice to refuse participation before completing surveys and at any point before submitting responses. By submitting responses either electronically or on paper, participants consented to participate in the study. Those participating were also informed via a cover letter that all data would be kept confidential and used to help researchers better understand the supervisory working alliance.

Study Setting

The study occurred at Pressley Ridge, a multi-state, multi-service organization founded in 1832. Pressley Ridge has a long and distinguished history of serving youth, families, and adults with complex mental or behavioral health problems across the Northeastern United States. Pressley Ridge operates over 70 programs in the following six states: Pennsylvania, Delaware, Maryland, Ohio, Virginia, and West Virginia. With over 1000 employees, Pressley Ridge currently cares for over 7000 individuals annually through the following core services: residential treatment, foster care, community based (e.g., family-based mental health, family preservation services, service coordination, and wraparound), and education.

Study 1 Methods

Procedure

Supervisees from 22 programs across Pressley Ridge were asked to anonymously complete two forms (the Alliance Building: Learning to Engage: Supervisor Alliance Form [ABLE-S] and the WAI-T version) about their supervisor in order to determine the validity and reliability of the short feedback tool (ABLE-S) in comparison to the long version of an assessment used to rate the working alliance (the WAI-T; Bahrick 1989). The forms were completed in staff meetings in order to capture a representative sample of supervisees across the organization and were sent to the Social Research and Innovation Center at Pressley Ridge for data analysis.

Measures

Alliance Building: Learning to Engage—Supervisor Alliance Form (ABLE-S)

The ABLE-S tool was developed by Pressley Ridge’s Organizational Performance Department as a brief measurement of the working alliance between a supervisee and supervisor to provide real-time feedback about a supervision session. The first three questions were developed to correspond with the three working alliance components (goals, tasks, and bond), and the fourth question addresses the impact of the working alliance on the supervisee’s progress. Participants completed the form using a scale from 0 (strongly disagree) to 6 (strongly agree) to statements such as “My supervisor and I have the same ideas about my goals in supervision,” and “My supervisor cares about me.” Higher scores indicate a more favorable alliance. Several versions of the form were also created to be used with: youth, adults, families, foster parents, youth in foster care to rate their foster parents, and teams, but are not the focus of the current study.

Working Alliance Inventory-Trainee Version (WAI-T)

The WAI-T version (Bahrick 1989) was used to measure supervisees’ perceptions of the strength of their working alliances with supervisors. The WAI-T is an adapted version of the original Working Alliance Inventory (Horvath and Greenberg 1986), a measure used to assess the strength of the working alliance between therapists and clients. Items on the WAI-T were simply reworded to fit the supervisor–supervisee relationship context. The WAI-T measures components of the supervisory alliance based on Bordin’s (1979) initial conceptualization of the alliance, including agreement on workplace goals, agreement on tasks, and the emotional bond between supervisor and supervisee.

The WAI-T is a self-report survey consisting of 36 items rated on a 7-point Likert scale ranging from 1 (never) to 7 (always). The WAI-T includes three subscales, each containing 12 items: Agreement on Goals (e.g., “My supervisor accurately perceives what my goals are”), Agreement on Tasks (e.g., “I am clear on what my responsibilities are in supervision”), and Emotional Bond (e.g., “I believe my supervisor likes me”). Items in each subscale are summed to create subscale scores, with higher values indicating more favorable impressions of the supervisory alliance. Previous research (Ladany and Friedlander 1995) has shown that the WAI-T demonstrates high external validity and strong internal consistency (α > .91 for all subscales).

Data Analysis Plan

To assess convergent validity of the ABLE-S, we examined correlations between ABLE-S items and corresponding WAI-T domains. We also completed an exploratory factor analysis to determine if all ABLE-S items made a unique contribution to operationalize the working alliance construct. Finally, we wanted to explore if items are related to each other, and their degree of reliability. The data were analyzed using the Statistical Package for the Social Sciences (SPSS) for Windows, version 25.0.

Study 1 Results

Demographics

A total of 256 supervisees from 22 programs participated in the survey. Participants were diverse in their role within the multi-service organization, with 54% working in community or home based programs, 34% in treatment foster care, 6% in youth residential treatment, and 6% in administrative support. The majority of participants were female (81%), and the average age was 37 years old. The racial breakdown of participants is as follows: 80% White/Caucasian, 12% Black/African American, and 8% other races. Surveys were completed anonymously and therefore did not ask for identifying information such as degree or job title, however supervisees working in participating programs include bachelors and masters level social workers, counselors, teachers, and mental health professionals. Table 1 includes means and standard deviations for each of the four items on the ABLE-S tool.

Table 1 Means and standard deviations for ABLE-S items
Full size table

Convergent Validity and Reliability Results

The total score on the ABLE-S tool was significantly related to the total score on the WAI-T (r = 0.79, p < .001). Additionally, the individual items on the ABLE-S indicated strong correlation with associated WAI-T subscales measuring goals (see Table 2) (r = 0.33–0.60, p < .001), tasks (r = 0.34–0.68, p < .001), and bond (r = 0.33–0.76, p < .001). Finally, the ABLE-S assessing progress was strongly correlated with all items on the WAI-T (r = 0.26–0.70, p < .001). Next, we calculated Cronbach’s alpha for the ABLE-S tool to determine reliability of the form. Results indicated high internal consistency (α = .94) (Cronbach 1951).

Table 2 Correlation of ABLE-S questions with items on the WAI-T
Full size table

Exploratory Factor Analysis

An exploratory factor analysis indicated a one factor model containing all four of the ABLE-S items, explaining 84.0% of response variance. This one factor model means that all four questions help to define and operationalize one singular construct, the working alliance. In addition, factor loadings for each of the four items were high: .91, .94, .89, and .94 respectively. High factor loadings suggest stronger factor contributions and mean that the four items are influential in explaining the variation in the factor. This means that all four items should be retained in the ABLE-S tool because these items are making a unique contribution to explaining the working alliance construct.

Study 2 Methods

Procedure

To identify programs for inclusion in the study, a master list of programs (N = 66) was created. Most programs were available to be included in the study, but some programs were excluded prior to random assignment. Programs with five or fewer staff members and programs with previous experience using the ABLE model (e.g., participated in study one or helped to pilot previous versions of the form) were excluded from this study. We identified two programs from each service type that were similar in program size and client demographics based on their primary service type (residential, foster care, education, community-based). This resulted in four program pairings for inclusion in the study (n = 8 programs). Finally, random assignment was used to identify which program in each service pairing would serve as the intervention site. This process mitigated sampling bias by having randomization into the group assignments, thus resulting in all individuals who are in the selected program sites being eligible for the study.

Supervisors and supervisees working in the control sites were provided with a prerecorded online training about the study’s purpose, time frame, measurements, participant rewards, and eligibility requirements. Staff in the intervention sites received the same training, but also viewed an additional training that explained the purpose of ABLE and demonstrated the use of the ABLE-S tool using video case examples. The intervention group also received a set of implementation resources including activity sheets, reminder cards, and guides, and was also asked to use the ABLE-S tool with their supervisees during weekly or bi-weekly supervision sessions. Baseline measures were collected simultaneously for all participating staff members prior to the start of the intervention itself. All staff members were eligible to participate in the study, but participation was not mandatory. All staff members who elected to participate could earn up to $30 in gift cards for completing study measures. Participating supervisors could earn an additional amount of gift card funds equal to the number of times they used the ABLE-S during supervision sessions.

Data were collected from both intervention and control sites at baseline, after 3 months, and once again after 6 months had passed. All data were collected using a battery of survey measures imported into the SurveyMonkey online survey administration software. At the start of each data collection period, all staff members received an email containing an individualized link to participate in the surveys online. Reminder emails were sent 1 and 2 weeks after the initial emails. At the conclusion of the study, all quantitative data were imported from SurveyMonkey into IBM’s Statistical Package for the Social Sciences (SPSS) for analysis.

Measures

Occupational Self-Efficacy Scale (OSES)

A shortened version of the OSES (Rigotti et al. 2008) was used to measure supervisees’ feelings of self-efficacy in the workplace. Occupational self-efficacy refers to the level of confidence that employees have in their individual ability to successfully complete work related tasks. Previous studies have empirically demonstrated that occupational self-efficacy is related to desirable job-related outcomes such as increased job satisfaction (Judge and Bono 2001) and performance (Stajkovic and Luthans 1998).

The short version of the OSES contains six items rated on a six-point Likert scale ranging from 1 (not at all true) to 6 (completely true). Higher values are indicative of higher levels of self-efficacy, and item values may be summed to produce a total score. Example items include “Whatever comes my way in my job, I can usually handle it” and “I feel prepared for most of the demands of my job.” A validation study conducted with participants from five countries demonstrated that the short version of the OSES has sufficient item-level internal consistency (α = .85–.90) and construct validity (Rigotti et al. 2008).

Professional Quality of Life Scale (ProQOL)

The second edition of the Professional Quality of Life Scale (Stamm 2010) was used to measure supervisees’ experience of compassion satisfaction and compassion fatigue. Compassion satisfaction refers to the positive aspects of working in the helping professions. These positive aspects are generally equated to a staff member feeling that they have the ability to help others and to make a difference in the lives of others through their work. Compassion fatigue refers to the negative aspects of the working professions, and is further categorized into burnout and secondary traumatic stress (STS). Burnout is associated with feelings of hopelessness, stress, and the idea that one’s efforts are insignificant or worthless. STS refers to negative feelings and behaviors resulting from exposure to work related trauma. The technical manual describes the ProQOL as having high construct validity, citing evidence from over 200 studies, but how these studies examined construct validity is not described. Stamm does, however, provide evidence that the scales are unique, citing low levels of shared variance across constructs (Stamm 2010).

The ProQOL measure results in three subscale scores: Compassion Satisfaction, Burnout, and STS. Each subscale contains ten items requiring a Likert scale response ranging from 1 (never) to 5 (very often). Scores from each subscale are summed as raw scores then standardized and converted to t scores for interpretation and comparison purposes. The Compassion Satisfaction subscale contains items such as “I feel connected to others.” The Burnout subscale includes items such as “I feel bogged down by the system.” The STS subscale includes items such as “I am not productive at work because I am losing sleep over traumatic experiences of a person I help.” Higher scores are indicative of more positive feelings for the Compassion Satisfaction subscale, and lower scores indicate fewer symptoms in the Burnout and STS subscales.

ABLE Model Fidelity

Upon conclusion of the study, supervisors in the intervention group were asked to complete a fidelity survey about the ABLE model developed by Pressley Ridge’s Social Research and Innovation Center. This survey is a six item measure that uses a four point Likert scale ranging from 1 (minimally) to 4 (fully). The first item asks supervisors to rate how often they used the ABLE model by rating the question “I used ABLE during each scheduled meeting with my supervisees.” The remaining five items of the survey were designed to assess whether supervisors were employing the foundational components and mechanisms of ABLE when using it with supervisees. Example items include “I inquire with individuals as to how I can improve their ratings and I check with them to determine if I understood their ratings clearly,” and “I integrate feedback and areas discussed into the next scheduled meeting with supervisees.” Higher scores on the survey are indicative of individuals who report using ABLE as intended.

Data Analysis Plan

We were interested in examining differences between the intervention and control group in changes in staff occupational self-efficacy and professional quality of life across three distinct time points. Because there were missing time points in the dataset, we used a Linear Mixed Model (LMM) analysis. The LMM provides more flexibility and power when handling unbalanced data (e.g., missing data or inconsistent time intervals), because it does not have the limitation of other conventional statistical techniques (e.g., repeated measures ANOVAs) that will drop the entire participant’s data if one time point is missing as well as addressing a lack of independence of observations (Shek and Ma 2011).

Study 2 Results

Demographics

A total of 93 supervisees from eight programs participated in the study. Supervisees included bachelors and masters level social workers, counselors, teachers, and mental health professionals. Of these, 42 or 45% (n = 24 for control, and n = 18 for intervention) had complete data for all three time points (initial, 3-month, and 6-month). Additionally, 89% (n = 43 for control; n = 41 for intervention) completed pre-test measures, 69% (n = 35 for control; n = 30 for intervention) completed 3-month measures, and 57% (n = 30 for control; n = 23 for intervention) completed 6-month measures. Data were restructured to perform LMM analysis, resulting in 138 data points for the control sites (46 at each of the three time points) and 141 data points for the intervention site (47 at each of the three time points).

Supervisees were diverse in their role within the multi-service organization with 39% in education programs, 33% in youth residential treatment, 15% in treatment foster care, and 14% working in community or home based programs. The majority of participants were female (67%) and white (84%). Table 3 provides results for demographics variables, and there were no statistically significant differences in these variables between the two groups.

Table 3 Pilot study demographics by group
Full size table

Results

Changes in Occupational Self-Efficacy

In order to assess changes to staff occupational self-efficacy over time between the intervention and control groups, a LMM was conducted. The LMM analysis did not indicate any significant changes in occupational self-efficacy over time between the intervention and control group (see Table 4). A compound symmetry covariance structure was used in order to minimize error in the model. Mean scores for each of the time points for both groups can be found in Table 5.

Table 4 Results of LMM for outcome variables by time point
Full size table
Table 5 Means and standard deviations for outcome variables by group and time point
Full size table

Changes in Professional Quality of Life

In order to assess changes to compassion satisfaction and compassion fatigue (burnout and STS) over time between the intervention and control groups, a second LMM was completed. A compound symmetry covariance structure was used in order to minimize error in the model. The LMM analysis did not indicate any significant changes in any of the three variables over time (see Table 4). However, as seen in Table 5, there were differences between the groups in compassion satisfaction (t = 3.33, p = .001), STS (t = − 2.37, p = .02), and burnout (t = − 2.55, p = .01). These results show the control group generally reported better scores overall than the intervention sites, including starting the study with more favorable baseline scores than the intervention group. The intervention group started with less favorable scores and experienced incremental change at 3 months although the change was not statistically significant.

ABLE Model Implementation Fidelity

Descriptive results from the ABLE model implementation fidelity measure demonstrate that supervisors may have struggled to implement ABLE effectively. Overall, 13 supervisors completed the survey, and the average score on the measure was 13.38. While there are no defined scoring bands or markers for fidelity, the range of available scores is 6–24, with higher scores indicating better implementation fidelity. The range of actual scores received was 6–22, suggesting a wide discrepancy between some supervisors’ ability or willingness to implement the intervention as designed.

Discussion

Past research demonstrates that a strong supervisory working alliance can help to attain desirable employee outcomes such as adherence to treatment protocols (Patton and Kivlighan 1997), lower levels of role conflict and ambiguity (Son et al. 2007), and improved job satisfaction, well-being, and retention (Mor Barak et al. 2009). Creating a strong supervisory alliance can be achieved by offering supervisees opportunities for collaborative discourse and feedback focused on individualized goal setting, task monitoring, and attention to the bond or rapport within the supervisory relationship. This paper introduces a new model (ABLE) that uses feedback mechanisms to help foster a healthy working alliance between supervisors and their supervisees. We conducted a two-part pilot study to assess the validity of the ABLE-S rating form and to explore the impact of using the ABLE model in clinical supervision.

The first part of the study showed that the ABLE-S is a reliable and valid measure of the supervisory working alliance and could be used to elicit real-time feedback from supervisees due to its brevity. To examine the potential impact of the ABLE model on supervisee outcomes, for the second part of the study we conducted a multi-site randomized control trial pilot. Programs from a mid-sized child welfare agency were randomly selected and assigned into intervention and control groups, and over the course of 6 months, the supervisors in programs designated as intervention sites were asked to use ABLE with supervisees. Results from a LMM analysis indicated no significant differences in occupational self-efficacy, burnout, compassion satisfaction, or STS over time. Based on this initial analysis, it appears that the use of ABLE did not improve these outcomes for supervisees working in the intervention sites.

It is possible that the use of ABLE in this particular sample did not have a significant effect on any of the intended supervisee outcomes. However, several significant confounds and outstanding circumstances are likely to have influenced the results of this study. The first confound is the low adoption rate of the ABLE model itself within the intervention sites. According to the results of the ABLE model fidelity survey, more than half of the supervisors in the intervention sites (53.89%) reported that they used ABLE minimally during the course of the study; no supervisors indicated that they used ABLE to the full extent. This makes interpreting the results of the presented statistical analyses challenging, as no supervisors used ABLE during each supervision session as intended, and more than half indicated that they rarely used ABLE when meeting with supervisees. While ongoing consultation was available to each of the participating program sites, at no point in time did any supervisors indicate that they were not using ABLE.

The low adoption rate experienced in this study is unfortunately not a new phenomenon in public and behavioral health service settings. Research has demonstrated that the uptake of promising or even evidence-based managerial strategies (including performance feedback) in these settings is consistently low (Rousseau 2006) due to a range of factors including time, competing priorities, and cultural fit. Other research identifies lack of knowledge and motivation as additional barriers (Tacia et al. 2015). It is also possible that supervisors were not using the ABLE model often because supervision was occurring infrequently. While this study did not include a measure of supervision frequency, past research has noted that busy practitioners may not always receive supervision (Issurdatt 2010). Therefore, future research examining the ABLE model should be careful to include additional measures to assess whether supervision is occurring and also whether the strategy is being used over the course of the study. For example, researchers could ask participants to complete fidelity assessments during the intervention phase instead of at the end of the study. Doing so would give consultants timely information to assist in targeted training for the intervention. In addition, organizational level supports that can facilitate the adoption of practice should be leveraged to maximize the probability that the intervention will be used. These supports include employing knowledge brokers, improving technical infrastructure, offering capacity building resources such as full-time trainers, and aligning the organization’s culture and climate with the strategy or intervention itself (Chuang et al. 2017).

The second significant confound is that supervisors also indicated that they did not always implement ABLE with fidelity when they did use it during supervision. Items two through six of the ABLE fidelity survey assess the implementation fidelity of the strategy. The resulting average item score of 2.3 is indicative of partial implementation fidelity to the ABLE model, with more supervisors selecting “minimal” implementation fidelity over “full” implementation fidelity. This suggests that many supervisors struggled to implement the model correctly, and may have required additional training or ongoing, on-site consultation from an experienced user. The online training provided as part of this pilot study was designed to briefly demonstrate how to use the ABLE model and to explain data collection requirements. This training did not include any form of assessment to identify whether attendees understood the material or could implement ABLE effectively. The training was purposefully designed in this way so that we could test the effectiveness of the ABLE model by itself with little training or support. However, the ABLE model relies heavily on supervisors’ ability to facilitate constructive dialogue and to respond to constructive criticism. The ABLE-S tool is merely a tool for initiating and framing these conversations. In practice, it is likely that the skills required to use ABLE effectively (e.g., reflective listening, collaborative goal setting) do not come naturally for all supervisors. Given that supervisors struggled to implement the intervention, future evaluations should include more intensive and perhaps live training measures to ensure that participants can demonstrate that they can successfully use these skills in practice.

While there were no significant differences in supervisee outcomes for those who were in the intervention group, the control sites generally faired better than the intervention sites at the end of the study. However, supervisees in the control sites started the study with better baseline scores on each outcome measure. The intervention sites began the study with lower baseline scores, but experienced some improvement during the first 3 months of the pilot study. Unfortunately, the supervisees in these same sites reported less favorable results at 6 months than at 3 months. The inconsistency in outcome gains observed in the intervention sites may be partially explained by attrition, as 18 fewer participants (49% attrition rate) provided 6 month ratings of the study’s outcome measures compared to the control sites (28% attrition rate).

Lastly, given these attrition rates and the infrequent use of ABLE at the intervention sites, it is possible that other influences or existing differences in the programs themselves accounted for results across the study outcomes. For example, one program in the intervention group was short staffed throughout the course of the study. When a program has open positions, current employees often have to take on a greater caseload and work longer hours, including overtime shifts. This may have accounted for some of the poorer scores provided by supervisees in this program. Other examples of program level variables that may have influenced results include supervision frequency and quality, staff experience, financial health, program census, commitment towards participating in the study, and more. While it is impossible to account for each of these program-level variables, future research could use a more advanced sampling technique to match programs based on these characteristics, if known. We are currently planning a larger scale evaluation study which will include measurement of important characteristics such as supervision quality and frequency to account for these program-level variables. Future evaluations should also take steps to account for the five dimensions associated with successful implementation of public health interventions: reach, efficacy, adoption, implementation, and maintenance (Glasgow et al. 1999).

Implications for Social Work

Although results of this pilot study were not in the hypothesized direction, the underlying theory of eliciting feedback from supervisees still should be considered an important component of the supervisory process. Oftentimes supervision sessions focus more on administrative functions and performance (paperwork, productivity, and caseloads) rather than a deliberate focus on the working alliance between supervisor and supervisee. Particularly for newer supervisors who feel they need to understand the administrative aspects and responsibilities of child protection before they can integrate the clinical and educational pieces of supervision (Dill and Bogo 2009). The use of the ABLE-S tool, since it was found to be valid and reliable in this study, can provide a tool necessary to facilitate the conversation around the supervisory working alliance. Previous research in other fields has found several benefits to focusing on the working alliance in supervision, and so child welfare organizations and the social work field should make a concerted effort to train supervisors in ways to build working alliances with their supervisees. With turnover and retention as common issues in the social work field (Sullivan et al. 2015), having a strong supervisory working alliance can help to overcome the stressors of working with clients who have emotional and behavioral issues. For example, previous research has found that the supervisory interpersonal interaction, which focuses on the quality of the relationship, has been linked to supervisees’ sense of empowerment and a reduction in emotional exhaustion and depersonalization that can occur when working directly with clients (Mor Barak et al. 2009).

Organizations that are dealing with high turnover and retention issues may benefit from training supervisors in eliciting feedback from supervisees, preferably using a real-time feedback tool such as ABLE-S. Growing empirical and professional literature around clinical supervision has identified the lack of supervisory training as the critical factor for supervisee accountability and ensuring high quality care for clients; therefore, specialized trainings for supervisors should be prioritized in both child welfare and the social work field (Kraemer Tebes et al. 2011). Organizations should also prioritize instituting a policy around the frequency of supervision sessions as these tend to be cancelled or re-scheduled due to time constraints and staffing issues. By creating a culture of feedback that places emphasis on the working alliance, a parallel process can occur with the clients who are receiving services as well which in turn would result in not only positive staff outcomes but also positive client outcomes.

Conclusion

We focused on examining a new model (ABLE) by testing the model’s measurement tool and examining worker outcomes in the field, which comes with its own unique set of challenges (such as low adoption rates of models, staff turnover, missing data, and the burden of implementing a new model and collecting data on top of heavy caseloads and other responsibilities). Despite these challenges, we did find the supervision tool itself (the ABLE-S) is a reliable and valid tool that can be used to elicit real-time feedback about the supervisory working alliance during supervision sessions. We also were able to implement a randomized study in the field and examine outcomes that are of interest to organizations that are focused on decreasing staff turnover. Results from the second part of the study did not indicate significant differences between the intervention and control sites in occupational self-efficacy, burnout, compassion satisfaction, or STS over time. Even though the results were not in the hypothesized direction, we learned several valuable lessons that we will incorporate into future studies in order to make sure we are controlling for confounding variables through expanded measurements and data collection, and ensuring proper implementation of new models or tools. This study provides an example of translating a theoretical model into practice and the pitfalls and successes that were experienced collecting data around a new model and measurement tool.