BACKGROUND

Up to 90% of patient encounters with health professionals occur in the primary health care setting.1,2 Performance and quality of primary health care has therefore substantial impact on public health.3,4 Although the generalisability of clinical practice guidelines has recently been challenged,5,6 they are still important for decision-making in primary care. Guidelines offer a synthesis of the current evidence and recommendations for action. However, their use is still not comprehensively accepted by primary care providers (PCPs).7,8 From the first comprehensive framework of facilitators and barriers9 to recent progresses in this field,1 a wide range of internal and environmental factors were mapped, challenging guideline adherence. Each of these barriers can be addressed by well-targeted intervention methods.10 The vast majority of intervention efforts focused on changing the individual behaviour of the practicing professional, e.g. aiming to increase the PCPs’ knowledge and/or skills by educational meetings or outreach visits; or to motivate by involving local opinion leaders, or auditing the PCP practice,11,12,–13 without convincing breakthrough. External barriers can be managed, e.g. by changing the regulatory environment or the method of financing, thus creating interest in guideline adherence; or by organisational interventions11,12,–13 like improving facilities or referral possibilities. However, in spite of all intervention efforts, guideline adherence still cannot be regarded as universal.1,5,7,12,13,14,–15

This systematic review aimed to evaluate the effectiveness of interventions to improve guideline adherence of PCPs in the primary care setting, focusing on recent results.

METHODS

Eligibility Criteria

All publications of original research were included if they referred to guidelines of non-communicable diseases for adult (≥ 18 years) patient populations and described an intervention performed in the setting of primary health care, targeting the PCP. General quality control measures without direct guideline reference were excluded.

For the sake of comparability, only studies performed in developed countries were considered, according to the categorisation of the United Nations Statistics Division (UNSD).16

Search Strategy and Information Sources

This review followed the PRISMA statement17 for systematic reviews. Electronic search was performed in Medline and Embase for studies published between November 2009 and October 2017, focusing on recent results in this field, with the following search strategy: ((“Physicians, Family”[Mesh] OR “Family Practice”[Mesh] OR “General Practitioners”[Mesh] OR “General Practice”[Mesh] OR “Primary Health Care”[Mesh]) AND ((guideline*[Title/Abstract]) OR (“Evidence-Based Medicine”[Mesh]) OR (“Practice Guidelines as Topic”[Mesh])) AND (implement* OR disseminat* OR diffusion OR “knowledge translation”)) NOT (“Nursing”[Mesh] OR “Developing Countries”[Mesh]).

The inclusion was restricted to those studies which were published in English or German. Full-text papers were retrieved via the library service of the university.

Data Extraction

Two reviewers (EK and EG) independently screened the retrieved studies for inclusion criteria based on title and abstract. The included studies were assessed likewise based on full text. Agreement on the selection was achieved by consensus. If disagreement could not be resolved by discussion, a third independent researcher (MM) made the decision.

A Microsoft Access-based data extraction form was developed (RS) and pre-tested on five studies (EK and EG). Information regarding the targeted guideline, the characteristics of studies, the applied intervention methods, outcome indicators and results were collected with the help of this form by one reviewer (EK) and randomly tested by the second reviewer (EG). All quality criteria and results were controlled by a second researcher (RS).

Intervention Methods

We categorised interventions according to the Cochrane Effective Practice and Organisation of Care Group (EPOC)11 taxonomy of interventions (1): professional interventions targeting the health care provider directly, (2) financial interventions addressing either the provider or the patient by various means of incentives, (3) organisational interventions aiming to support the desired behaviour by modifying the setting and (4) regulatory interventions which introduce changes on the level of legislation. The EPOC taxonomy further divides the1 professional interventions into (1.1) distribution of educational materials, (1.2) educational meetings, (1.3) local consensus processes, (1.4) educational outreach visits, (1.5) involvement of local opinion leaders, (1.6) patient-mediated interventions, (1.7) audit and feedback, (1.8) reminders, (1.9) marketing and (1.10) mass media.

Intervention schemes were categorised according to the number of applied intervention methods as a main criterion (i.e. single or multifaceted approach). Further refinement of this categorisation was based on the above types of intervention methods.

Outcome Indicators

For the classification of the outcome indicators, we followed the method of Grimshaw12 and included all measures of the process of care (i.e. the PCPs’ activities) and/or the outcome of care (i.e. patient-level results). We further categorised the process of care indicators as prescription, diagnostic behaviour, patient counselling and knowledge level.

Synthesis of the Results

Harvest Plots

Harvest plots18 provide an alternative graphical method for displaying combined data when the complexity of the intervention schemes, the diversity of the applied indicator sets and the inhomogeneity of the outcomes prevent direct comparison using traditional meta-analytic methods such as the forest plot. Harvest plots show both the effect size and other parameters of interest such as study quality.

All outcome indicator categories (both process of care and outcome of care) were included in this analysis. Though some studies indicated main outcome(s) and secondary outcome(s), each of them was handled equally. Outcome indicators within a study were grouped into the outcome categories of prescription, diagnostic behaviour, counselling, knowledge and patient-level outcome of care; thus, a study could report up to five outcome groups. Outcome groups were displayed by bar charts, with each bar referring to one outcome group of a study, so a study could be represented by up to five bars corresponding to the number of the covered outcome categories.

Level of effect was estimated by the proportion of improved outcome indicators and classified as “no effect” (the study has not reported any significant improvement among the outcome indicators), “moderate effect” (≤ 50% of the reported outcome indicators within a category improved significantly) and “strong effect” (> 50% of the reported outcome indicators of a category improved significantly). This categorisation was chosen because it reflects the distribution of the proportions and also allows to visualise this outcome in a harvest plot. The categorisation does not consider level of significance or absolute magnitude of effect.

Study quality was assessed by four of the five criteria of the Cochrane risk of bias tool19 (random allocation to the interventions, blinding of the outcome assessors, completeness of the outcome data, avoiding selective reporting), except for blinding of participants which is rarely possible in health services research and thus commonly left out.20 The fulfilment of each of these four criteria was rated as no bias (3 points); most likely no bias (2 points); and no info or a reported problem (1 point). Overall study quality was then summarised as the average of the four ratings. Sample size was considered relevant for the reliability of results; however, the actual power of each study was not reported in most of the studies. Thus, instead of power, we chose to display sample size instead, i.e. an artificial threshold of involving at least 100 PCPs was chosen to display more reliable results.

Results were visualised in a matrix where each row corresponds to an intervention scheme and each column to an effect size category. Within each cell of this matrix, bar charts display the two quality parameters of the included studies where the height of each bar indicates the quality of the study and the colour the sample size.

Forest Plot

We included controlled trials and controlled before-after studies with either a dichotomous or continuous outcome. We displayed the main outcome of each study, if explicitly stated in the article. If the main outcome was not clearly defined, we decided to display all outcomes. To make outcomes comparable across studies, effect measures were transformed to a common scale centred to zero, i.e. absence of an intervention effect would be represented by a zero, a beneficial effect of the intervention by a value greater than zero, and a harmful effect by a value less than zero. Due to the heterogeneity of the included studies and their different outcomes, we used a random effects model rather than a fixed effect model to estimate a summary measure of the pooled outcomes. Regarding transformation of effect measures, see details in electronic supplementary material (ESM 1).

For calculating the forest plot and the summary measures, we used the function “rma” from the “metafor”-package21 running R 3.0.3.22

To test for asymmetry in the funnel plot, we applied the Egger test.23

Data Availability

The dataset analysed during the current study is available from the corresponding author on reasonable request.

RESULTS

Selected Studies

We identified 1103 records from the database search, from which 211 records qualified for full-text assessment, 38 records were decided by discussion (EK, EG) and 27 records involved the third researcher (MM) in the decision process. Applying the inclusion criteria, 36 studies were eligible for harvest plot analysis. Among these, 21 were included in the forest plot. A flow chart of study selection according to PRISMA requirements and detailed reasoning for exclusion is displayed in Figure 1.

Figure 1
figure 1

Flowchart of paper selection.

Full size image

The studies (Table 1) represented a wide geographical coverage from three continents. Controlled studies (n = 17) where randomisation was typically on the cluster level due to the organisational requirements of these types of intervention, and controlled before-after studies (n = 11) were included in the analysis. The uncontrolled before-after studies (n = 8) were included only in the harvest plot. General quality of the studies was moderate with a median quality score of 2 out of 4 (Table 1). Disagreement regarding quality criteria was solved by discussion in six cases (EK, RS). The funnel plot has not revealed major concern regarding publication bias. Egger’s test for asymmetry resulted in a p value of 0.0921 (ESM 2).

Table 1 Characteristics of Selected Papers
Full size table

Methods of Intervention

The 36 retrieved studies covered six intervention schemes (Table 1). Among the single interventions, (1) three studies tested the passive distribution of the guideline or educational materials.37,49,50 (2) Seven studies applied educational meetings, eventually including supplementary tools of the same methodological approach, e.g. distribution of materials applied on the meeting or e-learning sessions.34,40,44,47,54,57,58 (3) Seven studies used diverse single approaches including audit,25,31,43 motivational interview,53 reminder26,32 and patient-mediated intervention.41 Multifaceted interventions mainly applied a combination of two methods, one of them a kind of training approach, extended by a second methodologically different method. From these, (4) the combination of educational meeting and audit/feedback was the most frequent with four studies.27,46,52,56 (5) Seven studies applied educational meetings supported by diverse additional methods such as reminders,29,48 networking33 or distribution55; or performed the educational component in the form of an outreach visit, extended by distribution24 or audit.42,45 (6) Eight studies used a multifaceted approach, combining more than two methods: distribution of materials, outreach visit and reminders59; educational meetings, outreach visit and audit28,36; outreach visit, audit and practice facilitation35; educational meetings, audit and reminders38; educational meetings, dissemination and networking30; educational meetings, motivational interview and networking51; educational meetings, reminders, audit and feedback, opinion leaders, financial reimbursement, patient-medicated interventions and distribution of educational material.39

Three studies reported explicitly relying on a theory with regard to the tested intervention: the Theoretical Domains Framework and the Theory of Planned Behaviour,34 the knowledge translation framework by the Canadian Institutes of Health Research41 and the RE-AIM framework.59

Indicators

The range of the number of indicators within a study was one to 43 (Table 1); in total, 303 outcome indicators were reported in the 36 studies. Eighteen studies applied outcome indicators belonging to one single outcome category, i.e. either prescription, diagnostic behaviour, counselling, knowledge or patient-level outcome of care; the other 18 studies covered up to three outcome categories in individual combinations (Table 1). In total, 62 groups of outcome indicators were reported by the 36 studies, referring to knowledge transfer (in four studies, with a range of one to nine individual indicators), diagnostic behaviour (in 22 studies, with a range of one to 43 individual indicators), prescription (in 19 studies, with a range of one to 16 individual indicators), counselling activity (in five studies, with each one indicator) and patient-level outcomes (in 12 studies, with a range of one to 15 individual indicators). With regard to effectiveness, 27 outcome groups showed no effect, 14 a moderate and 21 a strong effect.

Impact of Intervention

The analysis according to the intervention schemes was performed both by a harvest plot (Fig. 2) and forest plot (ESM 3) demonstrating that more complex methods are not necessarily more effective. Diverse single intervention methods such as audit, reminder, motivational interview or patient-mediated intervention showed the largest effect (73% of outcome indicators being effective; 0.48 [0.38, 0.58]) covering all outcome categories except counselling. A similar rate of success was observable for educational meetings with 67% of indicators being effective (0.18 [0.06, 0.31]) and multifaceted interventions combining more than two methods (with 65% of indicators being effective; 0.11 [0.01, 0.20]).

Figure 2
figure 2

Impact of intervention (harvest plot). Height of the bar: quality of the paper. Colour of the bar: number of PCPs in the intervention (grey < 100, black ≥ 100). Category of the outcome: D: diagnostic behaviour, P: prescription, C: counselling, K: knowledge level, O: outcome of care on patient level, : theory-based intervention.

Full size image

In contrast, the least effective method was the passive distribution of materials not implying any organised activity from the target group, showing a moderate effect only in the knowledge transfer category.

The effectiveness of multifaceted interventions applying two methods lay in between the two above approaches. Combining the educational meeting with audit almost halved the effectiveness (38% of the indicators showing any effect) compared to the single educational meeting, while the effectiveness of other two-method combinations was better but still not achieving the result of single-method active interventions.

The effectiveness of all single interventions even including the distribution of materials was larger (0.27 [0.17, 0.38]) (ESM 4) than the effectiveness of all multifaceted interventions applying more than one method (0.13 [0.06, 0.19]) (ESM 5).

The random effects model of quantitative outcomes based on 73 indicators of 21 studies covering five outcome categories shown in the forest plot (Fig. 3) resulted in a moderate overall positive summary effect measure (0.22 [0.15, 0.29]).

Figure 3
figure 3

Impact of intervention (forest plot) separated for outcome categories. The summary measure for each outcome category is shown at the end of each category; the summary measure over all studies at the bottom. Size of box proportional to log of sample size.

Full size image

Theory-based interventions (Fig. 2) did not show better performance, compared to those without explicitly referring to theory.

Analysing the effectiveness according to the outcome categories, knowledge transfer showed the largest improvement (0.39 [0.05, 0.73]). For the outcome category of diagnostic behaviour, a moderate positive effect was detected (0.22 [0.14, 0.31]). Prescription showed no significant improvement (0.11 [− 0.01, 0.24]), likewise the patient-based outcomes (0.14 [− 0.02, 0.30]). The analysis of the counselling activity was based on two indicators where the achievement of a single study contributed to the improvement (0.30 [0.14, 0.45]), while in the harvest plot, only one of the five indicators showed an improvement.

DISCUSSION

This study demonstrated that, among a wide span of interventions targeted at guideline implementation in the primary care setting, single-component interventions were equally effective as complex multifaceted intervention schemes in improving process of care and outcome of care.

When promoting a guideline, it seems obvious that educational material such as leaflets, folders or posters should be distributed. However, our analysis could show that passively receiving educational materials was least effective. Benefits of traditional printed educational materials are modest and short term.12 Arguably, the modest effect of passive distribution of educational material could be improved, e.g. by wisely choosing the channel and method of delivery13 or by the design of the material.60

An intervention should not stop at this step: also educational meetings, when remaining passive and didactic, could not improve the practice, regardless of whether it improved knowledge or not.61 In contrast, approaches implying active participation in the educational process have been reported as a key factor for success.62 This aspect is reflected when looking at the details explaining the difference between the high effectiveness of educational meetings found in our study and the controversial opinions regarding the effectiveness of educational meetings reported in the literature.7,63 Interactive education, preferably combined with supportive measures,61 was found to be superior to didactic and passive education,13 being able to improve either the process of care performed by the PCPs or the health outcomes of the patients.13,61 However, the effect sizes are typically small,12 because education alone has a limited effect on modifying complex behaviours.61

These above methods represent the two ends of the spectrum of the effectiveness detected in this study. In between, several single-intervention methods and their combinations may demonstrate effectiveness or lack of it, depending on the adjustment to the current circumstances. The generally low impact of audit and feedback12,13,64 can be improved by methodological refinement64 like choosing a respected authority as a source of providing repeated feedback in both verbal and written forms, or applying management tools of clear target setting and an action plan for achieving them. Reminders have a moderate effect on guideline implementation12,65,66 either as a single approach or in combination with educational materials or meetings as part of a multifaceted intervention.12 No consistent characteristics could be identified to increase their effect;65 however, some studies found a better performance of computer-based reminders compared to paper-based.13 In our analysis, only one study41 applied patient-mediated intervention with convincing effect on prescription and also on outcomes on patient level. Literature12 also suggests moderate to large improvement.

In our study, multifaceted interventions did not demonstrate a direct relationship between the number of intervention components and effect size, the same result confirmed by literature.12,67 The benefit of multifaceted interventions is ambiguous: some reviews did not support the commonly held assumption that multifaceted interventions are more effective than single-component interventions,12 while others favoured the multifaceted approach.13,62

These results suggest possibilities for further improvement. Our search hits covered an almost complete range of professional interventions, but it is equally important what was not covered: no example for organisational interventions could be identified. The literature confirms that this approach is sparse12,13: most of the interventions focus on the providers’ behaviour on individual level in spite of the fact that organisational interventions could give a powerful support for behaviour change63 or limit its required extent.68 Addressing the context comprehensively to increase the effectiveness could be supported by relying on theoretical frameworks.69 However, with respect to the small sample size in our review of studies utilising a theory, no definite conclusion could be drawn regarding the performance of a theory-based approach. The literature also reports70,71 on the small number of studies applying a theoretical background and encourages its use tailored to the setting and to the aim of intervention.70 Economic evaluation of guideline implementation interventions is rarely reported,12,13 our hits demonstrating no exception: three studies31,35,41 presented cost-benefit or cost-effectiveness information. This indicates that research should focus on adding this essential aspect in favour of comprehensive comparability.72 Finally, including the above aspects already in the guidelines73 and addressing the needs of the providers and settings74 as well as those of the patients75 could represent a powerful tool for increasing implementation effectiveness.

LIMITATIONS

Among several interventions targeting the PCP, our search strategy was directed to the studies which explicitly stated guideline implementation in the title or abstract. Search was performed in two major literature databases; however, relevant studies still may have been missed. Applying the UNSD categorisation for the selection of the developed countries may not cover all aspects of interest regarding the developmental level of the health care systems.

The diverse quality of studies was indicated in the analysis. The selected studies were heterogeneous with regard to or have not reported in-depth the non-clinical factors potentially influencing adherence to the guidelines of evidence-based practice; e.g. patient-related factors such as socioeconomic status, attitude, or preferences; physician’s personal characteristics, qualification, or work overload; and practice-related factors such as availability of resources or policies and reimbursement model.76

To account for the heterogeneity of the included studies regarding outcomes, study design and settings, we chose a random effects model and all effect measures were transformed to a similar scale. Still, the results should be interpreted with care and we utilised the forest plot rather as a tool to give an overview on the different effects, rather than to draw conclusions due to the summary measures. According to our best knowledge, applying forest plot and harvest plot on the same dataset to display and compare the information extracted by two means of synthesis of different level is a novelty.

CONCLUSION

Our systematic review demonstrated that all elements of the PCPs’ activities can be successfully improved by intervention; however, the most effective method cannot be simply and consequently linked with the category of the targeted outcome. The best result could be detected when knowledge, a basic prerequisite for change, was provided by an interactive method, and the intervention scheme addressed the detected needs of the involved actors and the barriers of the setting. Organising a change from outside requires a deliberate balance between addressing the detected needs and minimising the load on the target group, resulting in an intervention scheme as complex as required but as simple as possible.