Introduction

Several studies have shown beneficial effects of Shared Decision-Making (SDM) in mental health care [1,2,3,4,5]. Patients report that they are better informed and more actively engaged in treatment, experience higher satisfaction, and reduced decisional conflict, and patients show more treatment adherence. This can lead to better treatment outcomes. SDM is the collaborative approach in which patients and clinicians share available information from both perspectives and where patients are supported to participate actively in decision-making about treatment [6]. To participate in clinical decision-making, patients need access to information. Clinical feedback from outcome measurements in everyday practice, so-called Routine Outcome Monitoring (ROM), could be helpful in decision-making about treatment [7,8,9]. ROM is a personalised source of information and implies regular measurements of clinical outcomes during treatment. It provides feedback on the patients’ progress [10, 11] and has the potential to be a useful tool to involve patients in their treatment process [7, 8, 12, 13]. Research has shown that ROM can enhance the communication between patients and clinicians, and, when patients and clinicians both are provided with this feedback information, ROM can have beneficial effects on mental health status, especially for patients who are not responding to treatment favourably [11, 13,14,15,16].

Hence, integrating systematic clinical feedback derived from ROM within a SDM framework could be considered a promising approach to improve the communication between patients and clinicians, encourage the empowerment of patients, and reduce decisional conflict (DC) [8]. DC refers to the degree to which patients were engaged in and felt comfortable about important clinical decisions [17,18,19]. In general, patients with less DC experience a better decision-making process. This leads to higher patient satisfaction with the clinician and can have positive effects on treatment engagement, well-being, and treatment outcome, whereas high DC has negative consequences in terms of decisional delay, regret, treatment dropout, worse clinical outcomes, and a higher intention to complain about treatment [17, 19,20,21,22]. Therefore, in the National Quality Improvement Collaborative for the clinical use of ROM, we invested in the implementation and evaluation of SDM using ROM (SDMR) in Dutch specialist mental health care. Although scientific knowledge about SDM [3, 5] and ROM [11, 13,14,15,16] as separate initiatives is expanding, no previous randomised-controlled trial (RCT) has investigated a combined SDMR approach in mental health care. As far as we know, only one pilot study [7] on the combination of SDM and ROM was conducted. This study showed the feasibility and utility of ROM as a personalised source of information while making shared decisions about treatment. To test the effects of this combined approach, an RCT was recommended. Here, we present the results of a multi-centre two-arm cluster RCT aimed to investigate the effects of this approach on DC, the working alliance between patients and clinicians, and treatment outcomes. We compared SDMR with Decision-Making As Usual (DMAU) and hypothesized that SDMR: (1) diminishes patients’ perception of decisional conflict, (2) improves the working alliance between patients and clinicians, and (3) leads to better treatment outcomes.

Methods

Study design and participants

This study was a multi-centre two-arm match-paired cluster randomised-controlled trial (RCT) conducted in Dutch specialist mental healthcare in which the application of Shared Decision-Making using Routine Outcome Monitoring (SDMR) was compared with decision-making as usual (DMAU). 14 teams (7 intervention, 7 control) of 4 specialist mental health care organisations were recruited to participate in this trial. The design of this study has been described in more detail elsewhere [8].

SDMR was intended to be suitable for a broad population of patients treated in mental health care. Therefore, the trial involved teams treating patients with various diagnoses. The teams came from several catchment areas across the country (urban, semi-urban, and rural). To keep contamination to a minimum, we used a cluster design at the team level with pairs of two similar teams from the same mental health organisation being randomised.

Two complementary methods for recruiting patients were employed. First, clinicians of the intervention and control group consecutively invited each new patient to participate in the study. Second, in long-term treatment teams, patients were asked to participate at their annual treatment evaluation date. Patients were excluded from participating if they did not speak or read the Dutch language or were incapable of completing questionnaires because of cognitive functioning or an ongoing crisis. Patients enrolled after receiving face to face and written information about the study and after giving written informed consent.

Results are presented in accordance with the CONSORT statement for cluster randomised-controlled trials [23]. The trial was registered with The Netherlands Trial Registry (NTR5262).

Randomisation and blinding

After enrolment, random allocation of pairs of teams to either the experimental or control condition took place at cluster level by an independent statistician generating a random number (0 = DMAU, 1 = SDMR) by a SPSS syntax. The pairs of teams of the same mental health organisation treated a similar population of patients in the same geographical catchment area. Seven teams were allocated to the intervention group and seven teams to the control group. Due to the cluster randomisation and the nature of the intervention, blinding of the clinicians and patients was not possible. To reduce the risk of bias, data collection was carried out by research assistants, independent of the research team and participating teams, with separate instruments, which were not part of the intervention.

Intervention

The RCT was part of a National Quality Improvement Collaborative (QIC), aimed to implement Routine Outcome Monitoring (ROM) in every day clinical practice [8, 9]. The intervention teams, which participated for a full year in the QIC program, implemented a model with five steps to apply SDMR as a personalised source of information [8]. An essential part of this intervention was the implementation of ROM, tailored to the patient group, in routine clinical practice [9]. In addition, prior to the study, the clinicians of the intervention teams underwent a 1-day training in applying SDMR in clinical practice. Over the course of this study, the clinicians of all the intervention teams received one central booster session, and additionally intervention teams organised their own local, regular supervision sessions. The researcher attended the supervision sessions of each team twice, aiming to monitor intervention integrity. A comprehensive description of the intervention can be found elsewhere [8, 9].

Data collection

Patients completed self-report questionnaires just after the start of SDMR (T0) and at 6-month follow-up (T1). Clinicians answered questions regarding their patients at 6 months (T1). Respondents received a link by email to complete the questionnaires and if necessary received one or more reminders by email, mail, or phone. If patients did not use Internet, they received paper questionnaires by mail.

Outcomes

Primary outcome

Decisional conflict was measured in patients using the revised, validated decisional conflict scale (DCS) [17], which was translated into Dutch [8, 18]. Each of the 16 items is scored from 0 (strongly agree) to 4 (strongly disagree). Besides a total score, the DCS includes five dimensions (information, support, clarification of values, certainty, and decision quality). Higher scores indicate more decisional conflict, which means that patients experience less information, less support, less clarification, less certainty, and poorer decision quality about decision-making. To calculate the total scale and scores of the five dimensions, the item scores were summed, divided by the number of items, and multiplied by 25. The scores thus range from 0 to 100 [17]. In general, the internal consistency of the total score and five dimensions of the Dutch version of the DCS calculated in this study population is good [18].

To compare patients’ and clinicians’ views on Decisional Conflict, additionally, DC was measured on a Visual Analogue (VAS) 10-point scale (item: ‘to what extent do you agree with the decision taken?’), filled out at T1 by both patients and clinicians (regarding their patients).

Secondary outcomes

We used the following validated self-report questionnaires to measure the secondary outcomes: Working Alliance Inventory Short Form (WAI-S) [24], Outcome Questionnaire (OQ-45) [25], and Manchester Short Quality of Life Measurement (MANSA-VN-16) [26]. At T1, the WAI-S was filled out by both patients and clinicians. Measurement of treatment outcomes was tailored to the patient group. Therefore, the OQ-45, which measures symptom severity and functioning, was completed by patients in short-term treatment, and the MANSA-VN-16, which gives insight in the patients’ perspective on the quality of life, by those in long-term treatment. Cronbach’s alphas of these questionnaires calculated in this study population were: WAI-S α = 0.93; MANSA-VN-16 α = 0.89; OQ-45 α = 0.93.

Process outcomes

The process outcomes gave information about the intervention integrity, which tests whether SDMR was properly applied in clinical practice. We tested this both in the experimental and in the control teams, because (elements of) SDMR have been adopted by many clinicians already. The SDM process was measured with the validated SDM-Q-9 [27], and the application of ROM with four questions about the clinical use of ROM feedback [8]. At T1, both questionnaires were completed by patients and clinicians. The calculated Cronbach’s alphas in this study population were: SDM-Q-9 α ≥ 0.91 and ROM α ≥ 0.77.

Patients’ characteristics

At baseline, socio-demographics (gender, age, and educational level) and clinical characteristics (setting, primary diagnoses, and locus of control) [28] were collected.

Statistical analysis

Analyses were based on the intention-to-treat principle. Socio-demographical and clinical characteristics were calculated using descriptive statistics. We also tested for possible differences in these characteristics between the two arms at T0 with independent samples t tests for the continuous and Chi-square tests for the categorical variables [8]. To test for selective dropout before first or follow-up measurement, differences in baseline characteristics were assessed by logistic regression analyses. If participants had answered at least 80% of the items on a questionnaire, missing items were imputed with the mean value of the completed items. To take into account cluster effects and to handle missing data [29], the analyses of the primary and secondary outcomes and process parameters were performed using linear multilevel regression analysis (MLA) with three levels (teams, clinicians and patients). The analyses of the outcomes were based on the initial hypotheses performed with a two-tailed 0.05 significance level. We were also interested in the effect sizes (Cohen’s d) of the differences between the two arms, which were calculated by dividing the between-group difference by the pooled SD. The thresholds for interpreting the effect sizes were: small 0.00–0.32, medium 0.33–0.55, and large ≥ 0.56 [30]. Furthermore, the differences in patients’ and clinicians’ views were tested. Finally, two post hoc analyses were performed for the primary outcome DC. First, we assessed potential effect modification by primary diagnoses and the degree of DC at T0. Interactions were tested for significance at a p value of 0.1. Second, we performed, in line with the initial hypotheses, per-protocol analyses by investigating the influence of intervention integrity on DC and the association between DC and treatment outcomes. MLA was performed in MLwiN 2.21 software. All other analyses were performed in SPSS 22.0.

Sample size

This study was powered to detect a medium-effect size of d = 0.5 on the primary outcome between intervention and control group. A sample size of 65 patients per arm was needed to obtain β = 0.80 power with alpha set at 0.05 (two-tailed). To correct for clustering, we used an expected intra-cluster correlation coefficient (ICC) of 0.05 and calculated [23, 31] that a sample size of 136 patients per arm was needed. Taking into account a dropout rate of 25%, we intended to include 182 patients per arm.

Results

Participant flow

Inclusion took place between October 2015 and July 2016. All measurements were completed in March 2017. 14 teams (7 intervention and 7 control) of 4 organisations for specialist mental health care and 55 clinicians participated in the trial. As shown in Fig. 1, 227 patients gave informed consent and 186 patients (94 intervention, 51%; 92 control, 49%) responded to the first measurement (T0). In a total 158 patients, 81 intervention (51%) and 77 control (49%) completed the follow-up measurement at 6 months (T1). The range of participating patients per team was 4 to 28 (mean 13.3), and per clinician, the range was 1 to 11 patients (mean 3.38). The number of patients was in balance between the pairs of teams in the two arms. Clinicians completed an assessment at 6 months for 173 patients: 90 (52%) from the intervention and 83 (48%) from the control group. The dropout rate of patients between inclusion and T0 was 18%, and between T0 and T1, it was 11%. Numbers of dropout were not significantly different between the intervention and control group. In addition, the dropout rate was not significantly associated with any socio-demographic and most clinical characteristics. Only for locus of control was a significant difference found. Participants who dropped out between first and follow-up measurement reported significantly higher locus of control (p = 0.034) compared to patients who completed the follow-up measurement. Reasons for dropout were: (1) withdrawal of informed consent (57%), (2) inability to complete questionnaires (23%), because of decreased cognitive capacity, current crisis, or death, and (3) patients not responding to reminders (20%).

Fig. 1
figure 1

Flow diagram RCT shared decision-making and ROM (Consort 2010)

Full size image

Patients’ characteristics

Table 1 shows a complete overview of the socio-demographics (gender, age, and educational level) and clinical characteristics (setting, primary diagnosis, and locus of control) of the study population, which were balanced between the two arms. Patients with the primary diagnoses depressive (26%) and personality disorders (20%) were relatively the largest groups, followed by, respectively, psychotic (17%), anxiety (15%), bipolar (13%), and other* (9%) disorders. Patients’ characteristics showed the variety of participating patients receiving different kinds of long-term and short-term treatment. All patients were treated in outpatient settings, but followed different types of treatment (for example: different kinds of individual or group psychotherapy, medication treatment, or support and guidance in daily life given by nurses). We could not extract information from electronic records at patient level about who exactly received which kinds of treatment. Because of these varieties in patients and treatment characteristics, patients who participated in this trial also had to make decisions about various issues such as: diagnostic research, type of therapy, choices in medication treatment, personal treatment goals, or daytime activities.

Table 1 Participants’ characteristics
Full size table

Effectiveness of the intervention

In the intention-to-treat analyses, no significant differences between SDMR and DMAU on the primary outcome DC were found (Table 2), either for the total DC scale, or on the subdomains. We only found cluster effects within clinicians (ICC 0.2) and did not find evidence for clustering of the results at team level (ICC 0.0), which was also the unit of randomisation. Regarding the DC VAS scale, no significant differences were found for the patient-reported version. Only, for the clinician, completed version was a difference found. Compared to the control group, clinicians in the intervention group reported on this VAS scale significantly more ‘agreement with the decision taken’ (β = 0.710, p = 0.018, 95%CI: 0.122–1.298) with a medium-effect size (d = 0.45). In addition, we found no significant effects of SDMR on patients’ reported secondary outcomes: working alliance, clinical outcome, and quality of life. According to clinicians, the working alliance was better in the intervention group as compared to the control group (β = 0.254, p = 0.026 95%CI: 0.031–0,477; d = 0.45). Finally, although the application of SDM reported by patients and clinicians did not differ between conditions, both patients and clinicians experienced a significantly better usage of ROM in the intervention group compared to the control group (patients β = 10.980, p = 0.013, 95%CI: 2.289–19.671; clinicians: β = 25.558, p = 0.000, 95%CI: 17.930-33.186). The effect sizes were medium (patients: d = 0.39) to large (clinicians: d = 1.31).

Table 2 Differences between intervention and control group on the primary and secondary outcomes and process parameters
Full size table

When comparing patients’ with clinicians’ reported measurements, we found, irrespective of the condition, that clinicians, compared to patients, scored significantly higher (more positive) on the DC VAS scale (meandiff. 1.55, sd 2.96, p = 0.000, 95%CI: 1.069–2.028), WAI-S (meandiff. 0.39, sd 0.98, p = 0.000, 95%CI: 0.240–0.539), SDMQ9 (meandiff. 7.79, sd 32.46, p = 0.004, 95%CI: 2.531–13.040), and ROM questions (meandiff. 10.64, sd 35.19, p = 0.000, 95%CI: 4.941–16.33).

Subgroup analyses

Post hoc analyses for the primary outcome DC (DCS total scale change T1−T0) showed significant interaction effects between primary diagnosis and trial condition, which means that the effect of SDMR on the primary outcome was significantly influenced by the primary diagnosis as effect modifier. Table 3 demonstrates the intervention effects for each group of primary diagnoses. Patients with the primary diagnoses’ depression and anxiety demonstrated a reduced degree of DC in the intervention group at T1, whereas the other diagnosis groups showed an increased level of DC at follow-up. For depressed patients, SDMR had a large effect (d = − 0.69) and was significantly more effective than DMAU (β = − 11.173, p = 0.047, 95%CI: − 22.176 to − 0.170). Furthermore, the level of experienced DC at T0 was also an effect modifier for the DCS total change score (Table 3). Compared to the control group, patients in the intervention group with a lower level of DC at T0 reported more reduction in DC, while patients who experienced a high level of DC at T0 showed less reduction in DC at T1.

Table 3 Results of subgroup analyses: differences in change scores (T1–T0) on the primary outcome DC in the intervention group for diagnosis groups and level of DC at T0
Full size table

Associations between process and outcome parameters

As part of the post hoc analyses, we explored the association between the process parameters SDMQ9 and ROM with the primary outcome DC (Table 4). We found that a better application of SDM and ROM, according to patients, led to significantly less DC (SDM β = − 0.217, p = 0.000, 95%CI: − 0.321 to − 0.113; ROM β= − 0.111, p = 0.021, 95%CI: -0.205 to − 0.017). The effect sizes were small (ROM d = − 0.32)-to-medium (SDM d = − 0.45). When clinicians reported a better SDM and ROM application, we did not find significantly different DC scores in patients. Thus, in contrast to the clinicians’ reported level of SDM and ROM, when patients experienced a better application of SDM and ROM, this was associated with significantly decreased DC scores. Finally, we explored the influence of DC on treatment outcomes (Table 4). We found significant associations (medium-effect sizes) between reduced DC and better clinical outcomes (β = 0.344, p = 0.001, 95% CI 0.134 to 0.554, d = 0.40,) and a higher quality of life (β = − 0.011, p = 0.006, 95% CI − 0.019 to − 0.0032, d = − 0.54).

Table 4 Associations of process parameters with the primary outcome DC and the influence of DC change score on treatment outcomes
Full size table

Discussion

This study presents the results of a cluster randomised-controlled trial, testing the application of Shared Decision-Making using Routine Outcome Monitoring (SDMR) in specialist mental health care. Compared to decision-making as usual (DMAU), we found no significant influence of SDMR on the primary outcome reduction of Decisional Conflict (DC) reported by patients. Furthermore, there were no significant differences in the patients’ rated secondary outcomes (working alliance, clinical outcome, and quality of life) between the two arms. In contrast to the patients’ rated outcome parameters, the clinicians in the intervention group reported a significantly better working alliance and experienced more agreement with the clinical decisions that were taken. Subgroup analyses showed that that effect of the intervention was significantly modified by diagnosis. Depressed patients reported a significant and large effect of SDMR on reduction of DC. Looking at important process parameters, both patients and clinicians in the intervention group reported a significantly better application of ROM. Patients reported medium effects, while clinicians reported very large effects. The SDM process rated by clinicians and patients did not differ between the groups. In general, irrespective of the condition, clinicians’ views about the outcome and process parameters were significantly more positive than patients’ rated parameters. Exploring the influence of the process parameters on DC demonstrated that a better implementation of SDMR was associated with significantly less DC experienced by patients. Thus, when, according to patients, SDMR was applied well, patients reported a reduction of DC, which had a positive influence on clinical outcomes and quality of life.

Interpretation and clinical implications

To our knowledge, this is the first RCT to rigorously test the effectiveness of a combined SDM with ROM initiative in mental health care. In contrast with our hypotheses and previous research [1,2,3,4,5, 11, 13,14,15,16], we found no significant differences between SDMR and DMAU regarding the primary and secondary outcomes reported by patients. Interestingly, the results of the clinicians on DC and working alliance in the intervention group were significantly more positive compared to the control group. A possible explanation for the lack of differences between the conditions on the patients’ rated outcome parameters is that SDMR was not optimally implemented. First, this could be evidenced by the fact that, although clinicians in the intervention group reported change, patients did not notice much difference. A second argument is that, despite the investment in training, booster, and supervision sessions, the application of SDM, reported by patients and clinicians, did not differ between the two arms. Only the usage of ROM was significantly better in the intervention group according to both patients and clinicians. When looking at the association between the process parameters and the primary outcome, we found, in line with our hypotheses and previous research [1,2,3,4,5, 11, 13,14,15,16], that a better application of SDMR led to less DC. Thus, if clinicians were able to adequately apply SDMR during treatment consultations with their patients, this approach resulted in positive effects for patients. Less DC was, in turn, associated with better clinical outcomes and better quality of life.

Moreover, the subgroup analyses indicated that the effect of the intervention differed between subgroups of patients, with a significantly positive effect for depressed patients. This is in line with previous research which revealed that shared decision-making had a positive impact on depressed patients in terms of satisfaction and treatment adherence [3]. Previous work also demonstrated that feedback during treatment led to better outcomes in patients with mood disorders [16, 32]. This may imply that the intervention was too generic for the variety of patients treated in specialist mental health care. In designing the study, we hoped to be able to arrive at a generic method to improve SDM across mental health, and we approached teams treating very different types of patients. However, tailoring the intervention to the patient group and its decision topics may be necessary to improve results. Previous work has shown that the success of SDM depends on the patients’ ability to participate actively [33, 34]. A limitation of our intervention is that we focused our training on clinicians. Future interventions should targeting both patients and clinicians [33, 34].

Strengths and limitations

Our study had various strengths. First, due to the matched pair randomisation between similar teams, the risk of confounding was reduced. This was evidenced by the fact that no differences were found in patients’ characteristics and dropout rates between the two arms. Second, because this study was conducted in real-world clinical practice with a heterogeneous patient population, including various ages, diagnostic groups, and treatment settings, the study results were generalizable to a large group of mental health care teams, and thus had good external validity. Because of this heterogeneity, we were also able to investigate differences in effects for subgroups of patients. Third, the independent data collection with separate research instruments apart from the intervention prevented socially desirable scoring of patients and undesired influence of the research team or clinicians on the results. In addition, it enhanced the uniformity and quality of the collected data. Furthermore, at T1, we had insight into both patients’ and clinicians’ views on the application of SDMR. Finally, we used multilevel analyses to correct for the variability of the results at clinician level.

This study also had a number of limitations. Despite the extra efforts (recruiting an additional pair of two teams, checking if all eligible patients were asked to participate in the study and extending the inclusion period by a maximum of 5 months), the study population of 186 patients with a first measurement was smaller than intended. Reasons for this smaller population were difficulties in recruiting patients due to the lower inflow of patients receiving treatment from the participating teams and fewer patients who were willing to participate in the study. Because of the end date of the national collaborative and its funding, we could not extend the inclusion period further. Although the estimated sample size was not reached, the study population of 186 patients, with more than 90 patients per arm and the balance between arms in patients’ characteristics and dropout rates, proved sufficient to detect medium-effect sizes. Looking at the effect sizes of the primary analyses, these were extremely small, suggesting that power was not a deciding factor in our negative results of the primary outcome. Furthermore, although we adopted a cluster randomised-controlled design to reduce the risk of contaminating the two conditions, contamination cannot be ruled out altogether and may have weakened the effects found. Finally, treatment outcomes were evaluated, tailored to the patient groups, with two different measurement instruments. Patients who received short-term treatment filled out the OQ-45, which measures symptom severity and functioning. Patients who followed long-term treatment completed the MANSA-VN-16 which measured quality of life. Therefore, we have partially insight into the effects of SDMR on symptom severity, merely for patients who followed short-term treatment.

Conclusions

To our knowledge, this is the first randomised-controlled trial (RCT) which investigated the combination of Shared Decision-Making using Routine Outcome Monitoring (SDMR) in specialist mental health care.

In general, this study showed no effects of SDMR on primary and secondary outcomes rated by patients. However, subgroup analyses revealed a positive effect of SDMR on DC for depressed patients. According to clinicians, SDMR has a positive impact on DC and the working alliance. Furthermore, in the intervention group, both patients and clinicians reported a better ROM application. Finally, taking participants of both arms into account, a higher level of SDMR led to less DC, which, in turn, had a positive influence on treatment outcomes. Although this is a negative trial, the results are encouraging for further investments in the development, implementation, and investigation of this approach in clinical practice.