Skip to main content
Download PDF
Download ePub
  • Systematic review
  • Open access
  • Published:

Implementability of healthcare interventions: an overview of reviews and development of a conceptual framework

Implementation Science volume 17, Article number: 10 (2022) Cite this article

Abstract

Background

Implementation research may play an important role in reducing research waste by identifying strategies that support translation of evidence into practice. Implementation of healthcare interventions is influenced by multiple factors including the organisational context, implementation strategies and features of the intervention as perceived by people delivering and receiving the intervention. Recently, concepts relating to perceived features of interventions have been gaining traction in published literature, namely, acceptability, fidelity, feasibility, scalability and sustainability. These concepts may influence uptake of healthcare interventions, yet there seems to be little consensus about their nature and impact. The aim of this paper is to develop a testable conceptual framework of implementability of healthcare interventions that includes these five concepts.

Methods

A multifaceted approach was used to develop and refine a conceptual framework of implementability of healthcare interventions. An overview of reviews identified reviews published between January 2000 and March 2021 that focused on at least one of the five concepts in relation to a healthcare intervention. These findings informed the development of a preliminary framework of implementability of healthcare interventions which was presented to a panel of experts. A nominal group process was used to critique, refine and agree on a final framework.

Results

A total of 252 publications were included in the overview of reviews. Of these, 32% were found to be feasible, 4% reported sustainable changes in practice and 9% were scaled up to other populations and/or settings. The expert panel proposed that scalability and sustainability of a healthcare intervention are dependent on its acceptability, fidelity and feasibility. Furthermore, acceptability, fidelity and feasibility require re-evaluation over time and as the intervention is developed and then implemented in different settings or with different populations. The final agreed framework of implementability provides the basis for a chronological, iterative approach to planning for wide-scale, long-term implementation of healthcare interventions.

Conclusions

We recommend that researchers consider the factors acceptability, fidelity and feasibility (proposed to influence sustainability and scalability) during the preliminary phases of intervention development, evaluation and implementation, and iteratively check these factors in different settings and over time.

Peer Review reports

Background

Implementation science aims to identify and address care gaps, support practice change and enhance quality and equity of health care. Building a robust and generalizable evidence base to inform implementation practice is the objective of implementation research. Implementation research can also play a critical role in efforts to reduce research waste, in that it can provide evidence about the strategies that are effective for translating the findings of clinical research into enhanced healthcare practice and thus improved health outcomes [1,2,3]. Identifying the factors important for translation of an effective intervention or innovation from the research setting to routine clinical practice can arguably contribute to reducing the estimated annual US$85 billion, globally, wasted in health research [1, 2, 4].

Most implementation investigations focus on one of two approaches to achieve change. First, implementation activities consist of either “top down” processes (e.g. governance arrangements, national policies and guidelines, continuing medical education, incentivisation systems) [5,6,7], or more granular “bottom-up” processes that consider views of healthcare workers: their perceived barriers and enablers to specific elements of practice change at the level of healthcare teams and individual clinicians [8,9,10,11]. The second approach considers features of healthcare contexts (including organisational factors and the wider health system context) that might interact with the implementation activities to enable or impede practice change [12,13,14,15,16].

The current paper considers a third lever for achieving implementation: the perceived features of healthcare interventions themselves (in addition to effectiveness). An early theory, Diffusion of Innovations [17], identified six features of innovations that make their adoption more or less likely, namely, relative advantage, compatibility with the existing system, complexity, trialability, potential for reinvention and observed effects, where trialability refers to being able to test the innovation or intervention on a small scale, such as a pilot study. The more recent Consolidated Framework of Implementation Research (CFIR) proposed seven attributes of interventions, namely, intervention source, evidence strength and quality, relative advantage, adaptability, trialability, complexity and design quality and packaging, which refers to the presentation of the intervention, such as how it is bundled and user accessibility [12]. A recently published review identified 28 implementation frameworks and models, including the CFIR, which were synthesised into a number of core phases and components [18]. The authors suggest there is a need for an overarching framework that can guide researchers from intervention development to sustainable practice change.

Uptake of an intervention by both providers and recipients also depends crucially on their perceptions of the intervention. The COVID pandemic of 2020–2021 exemplifies this point. Even though approved vaccine interventions have substantial evidence of a positive benefit-to-risk ratio, the speed of uptake in many countries has been dependent on the perceptions of politicians, service providers and members of the public regarding the necessity, urgency and benefits of vaccine programs. It seems that, independent of the objective features of an intervention, stakeholder perceptions about the intervention will radically influence implementation and uptake at many levels. Furthermore, these perceptions may change over time and during roll-out of an intervention. We refer to these perceptions as views about the “implementability” of an intervention. We define “implementability” as the likelihood that an intervention will be adopted into routine practice and into health consumer behaviours across settings and over time. Several concepts related to implementability of healthcare interventions are gaining traction in the implementation science literature and appear to be primarily focused on the earlier stages of intervention development or latter stages of evaluation. These are acceptability, fidelity, feasibility, scalability and sustainability. A search of the health sciences literature (conducted 25th February 2021) for studies published in the last 20 years containing one or more of these five concepts in the title, revealed that the annual frequency of usage has steadily increased. Figure 1 shows that, from a relatively low baseline in the year 2000, these concepts appeared in titles in 2020, respectively, > 900, > 450, > 4500, > 350 and > 650 times.

Fig. 1
figure 1

Annual frequency of the five key concepts in publications indexed to PubMed

Full size image

Where there was an explicit rationale in the included studies, authors noted that the concept under investigation was likely to influence engagement, adoption and ongoing use. For example, a systematic review exploring videoconferencing in an orthopaedic setting [19] found that “acceptability of service users (both patients and clinicians) is a key factor for the uptake of telemedicine in clinical practice” (p.184). Hence, it is plausible that these concepts, individually and collectively, influence intervention implementability. To identify whether other implementation-related concepts were also appearing in the literature, we conducted a further illustrative search of literature published in the last 20 years (conducted on 29th August 2021) relating to specific interventions (using the phrase “of [intervention]”). We selected three healthcare interventions for which there were published reviews including one or more of the five concepts. The majority (> 90%) of reviews focused on clinical effectiveness or evaluation of outcomes. No additional concepts related to implementability, other than the five considered in our proposed framework, were evident.

There seems to be little consensus about the nature of these concepts, appropriate measurement strategies and how they might be related to one another. Without consistent definitions or reliable measurement approaches, it is not possible to test assumptions or predictions about whether these features indeed influence the implementability of healthcare interventions.

Implementability has been previously explored in the published literature, but this has focused on the implementability of clinical practice guidelines [20,21,22,23] or, more recently, the implementability of late-phase clinical trials [24]. The current paper focuses more broadly on prospective implementability of healthcare interventions, particularly at the early stages of development and evaluation, during scale-up, and over time.

The aim of this paper is to report the development of a testable conceptual framework of implementability that includes acceptability, fidelity, feasibility, scalability and sustainability.

Methods

A multifaceted approach was used to develop and refine the framework of implementability of healthcare interventions. We use the World Health Organization definition of healthcare interventions as “an act performed for, with or on behalf of a person or population whose purpose is to assess, improve, maintain, promote or modify health, functioning or health conditions.” [25]

Step 1: Overview of reviews

A preliminary exploratory search indicated a large volume of systematic reviews on the aforementioned five concepts within published literature on healthcare interventions. We therefore decided to conduct an overview of reviews [26], to answer the following questions:

Have the five concepts (acceptability, fidelity, feasibility, scalability and sustainability) been defined, operationalised and/or theorised in systematic reviews (SR) on healthcare interventions?

Have the five concepts been combined in any publications, frameworks or models used in published literature on healthcare interventions?

Search strategy

Systematic reviews published from January 2000 to March 2021 were identified and retrieved by one author (MK). Searches were structured by combining relevant review filters (Additional file 1: search strategy), with the appearance of the truncated term for each concept (“acceptab*”, “fidelity”, “feasib*”, “sustainab*” and “scalab*”) in article titles. Searching for these terms in article titles ensured that the review had a primary focus on the concept. The term feasibility was combined with “or process evaluat*” but papers were considered for inclusion only if the term feasibility was in the abstract. We did not include a synonym search as we were specifically interested in the use of the particular concept terms in the literature.

Multiple databases were searched using OVID (Medline and Embase) and EBSCO (PsycINFO) and restricted to publications in English.

Screening citations

Duplicates were removed using the deduplicate option in the OVID and EBSCO search engines and the remaining citations were imported into EndNote X9 [27] where further duplicates were manually identified and removed. Reviews were considered eligible for inclusion if they met the criteria detailed in Table 1. Screening was a two-step process, commencing with an initial review of the abstracts by one author (MK) to determine eligibility. If there was insufficient information in the abstract to make a decision regarding inclusion, the full paper was retrieved, and the methods section was screened. For example, drug-development reviews with acceptability or feasibility in the title but a focus on cost-effectiveness acceptability curves were excluded.

Table 1 Inclusion and exclusion criteria for the overview of reviews
Full size table

Full-text review

The full articles for all citations that met the inclusion criteria were retrieved by one author (MK) with an additional author (SK) independently reviewing a random selection of 10% of the retrieved papers. Data were extracted using a form developed by the research group and included if and how the concept was defined, how it was operationalised or measured, use or development of a framework and overall outcome of the review, i.e. had the healthcare intervention achieved acceptability, fidelity, feasibility, scalability or sustainability? A review was considered to have used a framework if it was mentioned in the methods and data were synthesised using the components of the stated framework. Primary studies included in the reviews were not individually reviewed as this was considered to be outside of the scope of this overview. Data extracted from the reviews were summarised descriptively.

Assessment of quality

Assessment of quality was not conducted as the aim of the study was to explore how the implementability concepts were defined and conceptualised, rather than the quality of the information related to the health intervention being delivered.

Step 2: Development of the preliminary framework

A preliminary framework (Fig. 2) was developed in parallel with the overview of reviews, integrating the five concepts that appeared to be mostly investigated in isolation in the published reviews on healthcare interventions, and based on the research experiences and theory-development expertise of two authors (JF and MK). This framework was then presented to the group of experts (co-authors), as described below.

Fig. 2
figure 2

Initial framework of implementability of healthcare interventions

Full size image

Step 3: Modified Nominal Group Technique

The Nominal Group Technique (NGT) is a facilitator-led, structured process for obtaining information and arriving at a decision with a target group who have some association or experience with the topic [28]. Various adaptations of the NGT have been used in conceptual studies that focus on framework development [29,30,31,32,33]. Recently, an additional pre-meeting, information-giving step has been suggested to enable more time for participants to consider their contribution to the topic [34, 35]. The adapted NGT process utilised in this study was as follows: (i) identification of group members, to include experts with depth and diversity of experience [36]. All authors on this paper were invited by e-mail to attend an online group meeting. They were purposively identified at the start of this study for their knowledge and expertise in the fields of implementation science, theory development, biomedical informatics and clinical research across a broad range of fields; (ii) provision of information prior to the meeting, including a PowerPoint presentation, findings of the overview of reviews and objectives of the meeting. Five authors with extensive clinical research backgrounds were asked to prepare a clinical scenario on one concept for sharing at the group meeting. The intention of this exercise was to discuss the fit between a real-world example of a study that explored one of the concepts and the proposed framework; (iii) meeting conducted online and facilitated by one author (JF) who has extensive experience in consensus panel processes. Following presentation of the meeting materials, including the preliminary framework, group members were instructed to silently consider the framework and generate ideas and critiques; iv) round-robin process with participants sharing their ideas and critiques; v) clarification process where participants shared their clinical scenario on a concept and discussed the fit with components of the initial framework, and vi) voting and/or agreement on the preliminary framework.

Results

Step 1: Overview of reviews

The database searches initially identified a total of 839 references across all five concepts (acceptability = 224, fidelity = 281, feasibility = 253, scalability = 37 and sustainability = 44). Following removal of 317 duplicates and screening of titles and abstracts, 301 full texts were sought for retrieval. Two were not retrieved as they were not available in English. Of the remaining 299 reports assessed for eligibility, 43 were excluded due to being unrelated to the concept (e.g. fidelity of DNA) and four were excluded as they focused on psychometric testing of a measure rather than a health intervention. The final number of publications included in this review was 252, of which 22 papers discussed more than one concept. As we were considering the concepts separately, these 22 were treated as separate investigations, resulting in a total of 274 investigations (Additional file 2: reviews included in the overview, consisting of acceptability = 132, fidelity = 41, feasibility = 65, scalability = 11 and sustainability = 25), with the stages of the search process presented in Fig. 3.

Fig. 3
figure 3

PRISMA [37] flow chart of included reviews for search completed in March 2021

Full size image

Characteristics of the included studies

Of the 252 studies in the overview, 30% included a meta-analysis and 19% used a mixed-methods approach that incorporated both quantitative and qualitative data from empirical research. The healthcare interventions that were a focus of the review were broad ranging and included psychological/psychiatric/psychosocial interventions (20%), technology-based interventions (5%), physical activities (6%), pharmacological and alternative interventions. The number of studies included in reviews ranged from 4 to 296, with the majority of studies (63%) having no setting exclusions (or not reported). Acceptability and fidelity were assessed using highly variable measurement approaches, so it was not appropriate to summarise the findings across reviews. The intervention under investigation was reported to be feasible in 32 (49%) reviews, sustainable in 1 (4%) review and successfully scaled up in 1 (9%) review (Additional file 3).

Definition and measurement (question a) and frameworks (question b) for the five concepts

A total of 1096 items of information were extracted from the 274 investigations by the first reviewer (MK). The second reviewer (SK), double extracted information from 10% of the reviews (total 32 papers double reviewed), which were compared with the first reviewer to assess reliability.

Of these, 100% agreement was achieved for definition of the concept and outcome of the review (e.g. acceptability or fidelity or feasibility or scalability or sustainability was/was not achieved). There was 96% agreement on the use of a framework and 89% agreement on the constructs measured. One reviewer failed to identify a framework in one of the studies resulting in 96% (out of 32 studies double reviewed) agreement. For constructs measured, reviewers did not identify the same constructs for two papers, resulting in 89% agreement. The two reviewers discussed these differences and agreed on a decision.

Twenty-two publications included two concepts in their review, of which 20 considered acceptability and feasibility [38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57] in exploring implementation of a healthcare intervention, and two considered scalability and sustainability [58, 59].

Acceptability

Of the reviews exploring acceptability of a healthcare intervention, only 13 provided an a priori definition and they largely focused on whether those receiving a healthcare intervention found it to be “appropriate” and “fair” and “reasonable” [42, 48, 60,61,62,63,64,65,66]. Four reviews considered acceptability from the perspective of those delivering a healthcare intervention [61, 62, 65, 66].

The majority of reviews measured one variable in evaluating acceptability, and this was predominantly either dropout rates (33%) or user perceptions of the intervention (30%). Twenty-three reviews measured three or more component variables, most commonly a combination of participant dropouts, recruitment rates, perceptions of users such as satisfaction measures, adherence to the study protocol and adverse events (3%).

Six reviews used a framework to define and measure acceptability, as described in Table 2. The Theoretical Framework of Acceptability [67] was used in four of these reviews and defined acceptability as “a multi-faceted construct that reflects the extent to which people delivering or receiving a healthcare intervention consider it to be appropriate, based on anticipated or experiential cognitive and emotional responses to the intervention”. (p.8) This framework consists of seven constructs related to acceptability including affective attitude, burden, perceived effectiveness, ethicality, intervention coherence, opportunity costs and self-efficacy.

Table 2 Definitions, frameworks and commonly measured components in systematic reviews on acceptability of healthcare interventions (n = 132)
Full size table

Fidelity

Of the 41 reviews exploring fidelity of a healthcare intervention, 35 included an a priori definition. Almost all reflected a dictionary definition of fidelity [68] with terms such as “integrity” and “delivered as intended” and “accuracy and consistency” included in their descriptions.

Thirty-six reviews measured four or more components as part of their assessment of fidelity of a healthcare intervention including adherence to the protocol (76%), dose delivered and received (76%) and provider training (49%). Thirty-four publications on fidelity of a healthcare intervention explicitly used a framework to guide their review, as described in Table 3. The framework used in 19 of these cases was from the National Institute of Health Behavior Change Consortium (NIHBCC) [69] which includes measures of study design, training, delivery of treatment, receipt of treatment and enactment of treatment. The NIHBCC describe fidelity as “the methodological strategies used to monitor and enhance the reliability and validity of behaviour interventions”. (p.443)

Table 3 Definitions, framework and commonly measured components in systematic reviews on fidelity of healthcare interventions (n = 41)
Full size table

Feasibility

Thirteen of 65 reviews defined feasibility using terms such as “practicality” and “ease of delivery” and “possible to undertake”. Seven of these 13 papers noted the importance of context and broader system factors when considering the “possibility of what could be done”, such as physical space, ongoing funding and political support [39, 70,71,72,73,74,75].

The two most frequently measured constructs within the concept of feasibility were adherence to the study protocol (34%) (i.e. the same as the operationalisation of fidelity) and perceptions of key stakeholders (33%), including those providing and receiving the healthcare intervention (23%). There was a significant overlap between feasibility and acceptability with a number (22%) of feasibility reviews incorporating acceptability as a construct to be measured within feasibility.

Five reviews referred to a feasibility framework (Table 4) with the most commonly used being Bowen et al’s publication on designing feasibility studies [76]. This highly cited publication does not provide a definition of feasibility but does identify eight areas of focus that should be addressed in feasibility studies including acceptability, demand, implementation, practicality, adaptation, integration, expansion and limited efficacy testing. Bowen et al. define implementation as the extent to which the intervention can be delivered as planned which is synonymous with fidelity.

Table 4 Definitions, frameworks and commonly measured constructs in systematic reviews on feasibility of healthcare interventions (n = 65)
Full size table

Almost a third (32%) of the reviews found the healthcare intervention to be feasible, with the remaining being a mixture of not feasible or unable to establish feasibility due to lack of information in the empirical studies.

Scalability

Healthcare interventions may be scaled up to different populations and/or settings. Eleven reviews explored scalability of a healthcare intervention with nine presenting an a priori definition including terms such as “deliberate efforts” and “expanding or increasing the impact”. Six of these definitions also included the need for a healthcare intervention to be proven effective prior to scaling up.

Five reviews measured four or more constructs with organisation, community and sociocultural factors being the most frequently reported measures (45%) followed by resources, economic viability (18%) and adaptation of the intervention (18%). Only one of the reviews definitively found that the healthcare intervention had been successfully expanded across different settings or populations [77]. The majority were unable to reach a conclusion due to lack of data in the included studies.

Four different scalability frameworks (Table 5) were described and used in four of the 11 reviews [59, 78,79,80]. These were the World Health Organisation ExpandNet Scaling-Up Framework [81], the Intervention Scalability Assessment Tool (ISAT) [82], the Assess, Innovate, Develop, Engage, Devolve (AIDED) model [83] and the Non-adoption, Abandonment, Scale-up, Spread, Sustainability (NASSS) framework [84]. Commonalities across the four frameworks include the intervention, the strategic/political context to support scale-up and resources to support and sustain the scale-up process. Scalability was defined in a similar way within these frameworks as the capacity or ease with which an intervention or innovation that had been proven effective could be expanded to other settings or populations [81, 82].

Table 5 Definition, frameworks and commonly measured constructs in systematic reviews on scalability of healthcare interventions (n = 11)
Full size table

Sustainability

Of the 25 reviews on this concept, 15 included a definition with common use of terms such as “continuation” and “extended period of time”. Seven of these definitions also included the notion that sustainability is about the maintenance of the intervention or program after initial funding or implementation efforts have ceased [58, 85,86,87,88,89,90].

Constructs typically measured in reviews of sustainability of healthcare interventions included organisation- or community-specific factors (36%), continuation of the intervention beyond a specified period of time (24%), established collaborations or partnerships (12%) and resources (8%). Only one of the reviews [91] found that sustainability had been achieved for the healthcare intervention under investigation. A number of other reviews [92,93,94] were unable to draw a definitive conclusion due to inconsistent definitions and measures of sustainability within the empirical literature in the reviews.

Seven different sustainability frameworks (Table 6) were referred to in nine of the systematic reviews. The two most frequently used frameworks included ongoing maintenance of benefits from the intervention, capacity building and integration of the intervention or program within the organisation [95, 96]. Moore et al. developed a comprehensive definition of sustainability based on five constructs including continuation of a program or intervention of implementation strategies or individual behaviour change, after a defined period of time, with or without adaptations but continuing to produce benefits for the individual and/or systems [95].

Table 6 Definitions, frameworks and commonly measured constructs in systematic reviews on sustainability of healthcare interventions (n = 25)
Full size table

In summary, two key findings emerged from the overview of reviews. First, the current literature suggests that the concepts are related, although there was some variation in the terms used. For example, reviews on feasibility of a healthcare intervention measured ‘implementation’ of the intervention, defined as the extent to which the intervention can be delivered as planned, also known as fidelity. Similarly, reviews on sustainability, also known as maintenance, measured resources, funding and organisational factors, which were all measures frequently included within reviews on feasibility of a healthcare intervention. The second key finding is that although acceptability appeared to be an important factor contributing to fidelity, additional factors were required, for example provider training. Similarly, although fidelity was an important factor contributing to feasibility, additional factors were required, for example funding and other resources such as physical space.

Step 2: Modified Nominal Group Technique

The first group meeting took place online using ZOOM [97] in March 2021. Three themes were identified from the group discussion, as follows:

Theme 1: It is plausible that the concepts influence implementability of a healthcare intervention.

Following presentation of the findings of the overview of reviews on each concept, participants agreed the theoretical plausibility of a framework of implementability for healthcare interventions that includes all five concepts. This was further consolidated through sharing their own real-world research and clinical experiences as described in Tables 7, 8, 9, 10 and 11.

Table 7 Scenario illustrating influence of acceptability on implementability
Full size table
Table 8 Scenario illustrating influence of fidelity on implementability
Full size table
Table 9 Scenario illustrating influence of feasibility on implementability
Full size table
Table 10 Scenario illustrating influence of fidelity and feasibility on sustainability of a healthcare intervention
Full size table
Table 11 Hypothetical scenario illustrating influence of acceptability, fidelity and feasibility on scalability of a health intervention
Full size table

Theme 2: The concepts appear to be related to one another.

Participants were asked to consider the question “in your view, is it plausible that the concepts are related to each other?” All agreed that it was possible that the concepts were interdependent and should be considered together when developing and implementing a healthcare intervention. They suggested that individual concepts were necessary but insufficient on their own to ensure implementability of a healthcare intervention. Participants also identified that in many of the clinical scenarios they shared, often more than one concept was involved in the final outcome. For example, it was identified that the hypothetical scalability scenario included issues with feasibility of a device, acceptability of an online system and lack of procedural information relevant for fidelity.

Theme 3: The preliminary implementability framework could be amended to better represent the relationships between the concepts.

Although all participants agreed that the concepts were related, they were not in agreement that the preliminary framework adequately represented the nature of their interdependence. Some participants felt more detail was needed to explain how the concepts related whilst other participants felt that some concepts required greater representation than others. All participants were asked to further consider the framework over the following week and provide feedback and revisions.

Step 3: Revising the framework

Following the first group discussion and feedback from the participants, the preliminary framework was revised, resulting in three options presented to consensus panel participants as three figures. These three draft frameworks included the revisions requested by the participants such as annotations on the first framework for option one, different graphics representing different relationships between the concepts on the second option and a combination of options one and two for the third framework option. Participants were asked to consider the three options and silently vote on their preferred framework and provide any further feedback via return email. The feedback was considered and integrated by two authors (MK and JF) resulting in a further version of the framework (Fig. 4). All participants agreed this final version.

Fig. 4
figure 4

Conceptual framework of implementability of healthcare interventions

Full size image

The framework as depicted in Fig. 4 is designed to guide research activities, chronologically and iteratively, from left to right and from bottom to top. Commencing in the research context, it is proposed that acceptability is the first concept to assess, during intervention development and work-up of supporting documentation and resources (intervention protocol, training manual, patient information leaflet, data and technology requirements, validation of digital components, etc.). If acceptability is adequate to providers and potential recipients, it is appropriate to deliver the intervention to assess fidelity as delivered and as received. Without adequate acceptability, providers and recipients are unlikely to engage with the intervention, and hence, fidelity will be low. Adequate fidelity will also require other enabling factors such as provider training and confirmed information flow. Without adequate fidelity, it would be wasteful to conduct a feasibility study. Factors such as appropriate resources, workforce, technology, and management will be required for feasibility. If feasibility (supported by acceptability and fidelity) in the research context is adequate, it is appropriate to consider testing acceptability, fidelity and feasibility in the healthcare context. If adopted consistently in one healthcare context, it is appropriate to consider scaling the intervention to other settings, provider groups and patient groups. In each new setting, it would be wise to re-assess acceptability, fidelity and feasibility as described above because adequate feasibility in one setting at one time, whilst a positive sign, is not a guarantee that the intervention will be feasible in other settings. Similarly, over time, the factors that support feasibility may change, thus threatening sustainability. It would therefore be prudent to continue to assess the factors affecting feasibility over time to detect any problems that need to be addressed.

Discussion

Based on the findings from the overview of reviews and the group consensus process, we propose a framework of implementability of healthcare interventions which includes five key concepts, namely, acceptability, fidelity, feasibility, scalability and sustainability. The framework illustrates the interrelationship between the concepts and chronology, with acceptability, fidelity and feasibility requiring investigation during early stages of the development of a healthcare implementation, including during proof-of-principle studies and pragmatic evaluations of intervention effectiveness at one point in time and in one specific context. Acceptability is a necessary but not sufficient condition for fidelity, and similarly, fidelity is a necessary but not sufficient condition for feasibility. All three concepts are context- and population-dependent and will require reinvestigation as the healthcare intervention is scaled to different settings and populations and over time. We argue that there is an association between the concepts, with acceptability, fidelity and feasibility influencing the scalability and sustainability of a healthcare intervention. We are not suggesting that there is a causal relationship between the concepts, rather, scalability and sustainability depend on the pre-conditions acceptability, fidelity and feasibility of the healthcare intervention, and these concepts should be re-examined over time, and as the healthcare intervention is implemented with different populations or in different settings. This argument is consistent with other frameworks on scalability and sustainability, including the Intervention Scalability Assessment Tool [82] and the Dynamic Sustainability Framework [98], both of which suggest that feasibility, acceptability and fidelity must be considered in the planning for scaling up and sustainability of a healthcare intervention. The Consolidated Framework for Implementation Research [12] suggests that the outer setting, including the social and economic context, can influence implementation. Our proposed framework encourages the researcher to prospectively assess acceptability, fidelity and feasibility in both the inner and outer contexts as the key stakeholders are likely to be different.

From the 252 reviews identified in the overview, the majority did not provide a definition of the concept. Rather, the reviews used measurement approaches which implied a definition. For example, feasibility was typically defined by measuring components such as compliance to the intervention, dropouts, recruitment rates and adverse events. There was conflating of terms, with feasibility and safety used interchangeably in several reviews, particularly drug feasibility reviews. Although frameworks were identified for all five concepts, they were not frequently used in the reviews identified in the overview. Most of the frameworks included conceptual definitions and operationalisation of the concept, but these varied between frameworks. It is difficult to test the influence of these concepts on the implementability of a healthcare intervention without consistent definitions, descriptions, operationalisation and measurement approaches for these concepts.

Of all the concepts explored in the systematic overview, scalability and sustainability of healthcare interventions were not often achieved. These findings suggest that healthcare interventions may be found to be effective, acceptable and feasible in the development or pilot phase, but this does not guarantee successful scale-up or sustainment of the intervention over time. We propose that the framework of implementability can provide a dynamic, longitudinal perspective of intervention development where researchers consider acceptability, fidelity and feasibility during the earlier phases of intervention development and implementation, and iteratively re-evaluate these factors as the healthcare intervention is scaled to different settings and over time.

Other rigorous frameworks, such as RE-AIM and the Implementation Outcomes Framework, propose that sustainable adoption and implementation of healthcare interventions require consideration of many like concepts such as acceptability, reach, effectiveness, adoption, implementation and maintenance [99, 100]. Whilst there are some similarities in the concepts contained in these frameworks and our proposed framework of implementability, the latter is explicitly concerned with the prospective and ongoing identification of factors that will influence scalability and sustainability of a healthcare intervention. It should also be noted that prospective identification of factors that influence implementability of healthcare interventions is receiving growing attention in the literature, particularly in relation to reducing avoidable research waste [101]. A recent publication developed a framework to assist researchers to prospectively cost the different phases of healthcare interventions. The authors argue that implementation costs are often underestimated or not included in cost-effectiveness analyses. This in turn contributes to research waste which may have been avoided through a more systematic and earlier approach to identifying the factors that support the translation of effective interventions into real-world settings, and prospectively costing the implementation of these.

It has been argued that involving end-users in the development and implementation of healthcare interventions may improve outcomes through enhanced relevance, acceptability and feasibility [102]. We propose that our framework of implementability could be used to test these assumptions. We recommend that researchers prospectively set criteria to inform the decision about whether to abandon, amend or proceed with the intervention, depending on the outcomes of the feasibility study [103]. Table 12 illustrates how the framework of implementability could be used to prospectively guide the ongoing implementation activities of healthcare interventions. Enabling factors were identified from published frameworks and reviews included in step 1, though this is not an exhaustive list and is likely to be influenced by context. For example, scalability of healthcare interventions to low- and middle-income countries may be more, or less, enabled by factors that differ from those in high-income countries [104]. The framework of implementability of healthcare interventions may be particularly helpful in guiding effectiveness-implementation hybrid designs, which aim to simultaneously evaluate effectiveness of the intervention in the real-world context and the implementation strategy [105,106,107].

Table 12 Example plan for using the framework of implementability
Full size table

We propose that empirical investigation of the framework of implementability is required to answer the following questions:

What is the nature of the relationship between the key concepts? (e.g. linear, curvilinear or threshold?)

Do acceptability, fidelity and feasibility predict scalability and sustainability as proposed in the framework?

Can identified deficits in constructs of the framework be addressed to enhance the implementability of effective interventions?

Strengths and limitations

To our knowledge, this is the first review that considers all five key concepts in published reviews on healthcare interventions. The overview collated and described important information on concepts that are increasingly being assumed to influence implementation of healthcare interventions. The development of a framework utilising well-established consensus methods is another strength. Although one author was responsible for most of the screening, data extraction and coding, independent extraction by a second author of 10% of the reviews confirmed reliability of the extraction process.

In order to the make the overview of reviews feasible, we only focused on publications that had one or more of the concepts in the title and/or abstract. Therefore, it is possible we may not have identified some reviews that were relevant. It must also be noted that the framework of implementability of healthcare interventions is untested. We propose that it articulates some of the untested assumptions in the current literature on implementation science and have suggested some approaches for empirical evaluation of the framework.

We do not propose that interventions with high implementability will automatically result in high uptake. As we argued in the background to this paper, the features of interventions may interact with top-down and bottom-up implementation activities, and with contextual factors, to achieve consistent uptake into routine practice. Our argument is that these implementation activities are more likely to be effective if implementability of the intervention is high.

Conclusions

The framework developed in this study can inform research that aims to prospectively and iteratively identify the likely implementability of evidence-based healthcare interventions. We suggest that the framework be tested empirically through studies that examine the actual uptake of interventions, across settings and over time, compared with prospective assessments of the independent variables (acceptability, fidelity and feasibility) and the outcome variables (scalability and sustainability) in the framework. We recommend that, to avoid research waste, implementability should be assessed, and enhancements made, during the clinical evaluation stages of the development of interventions [1, 3]. This would potentially accelerate their uptake into clinical practice.

Availability of data and materials

All data generated during this study are included either within the text or as an additional file.

Abbreviations

AIDED:

Assess, Innovate, Develop, Engage and Devolve

CFIR:

Consolidated Framework of Implementation Research

FAME:

Feasibility, Appropriateness, Meaningfulness and Effectiveness

IOF:

Implementation Outcomes Framework

ISAT:

Intervention Scalability Assessment Tool

ITIPS:

Implementation of Treatment Integrity Procedures Scale

NASSS:

Non-adoption, Abandonment, Scale-up, Spread and Sustainability

NGT:

Nominal Group Technique

NIHBCC:

National Institute of Health Behavior Change Consortium

PRISMA:

Preferred Reporting Items for Systematic Reviews

RE-AIM:

Reach, Effectiveness, Adoption, Implementation and Maintenance

SAFE:

Structured Assessment of Feasibility

SR:

Systematic review

WHO:

World Health Organisation

References

  1. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet (London, England). 2009;374(9683):86–9.

    Article  Google Scholar 

  2. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Glasziou P, Chalmers I. Research waste is still a scandal—an essay by Paul Glasziou and Iain Chalmers. Bmj. 2018;363:k4645.

    Article  Google Scholar 

  4. Ivers NM, Grimshaw JM. Reducing research waste with implementation laboratories. Lancet (London, England). 2016;388(10044):547–8.

    Article  Google Scholar 

  5. Davey P, Marwick CA, Scott CL, Charani E, McNeil K, Brown E, Gould IM, Ramsay CR, Michie S. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev. 2017;2(2):CD003543. https://doi.org/10.1002/14651858.CD003543.pub4.

  6. Flodgren G, Eccles MP, Shepperd S, Scott A, Parmelli E, Beyer FR. An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes. Cochrane Database Syst Rev. 2011;2011(7):CD009255. https://doi.org/10.1002/14651858.CD009255.

  7. Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;2009(2):CD003030. https://doi.org/10.1002/14651858.CD003030.pub2. Update in: Cochrane Database Syst Rev. 2021 Sep 15;9:CD003030.

  8. Cane J, OʼConnor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37.

    Article  PubMed  PubMed Central  Google Scholar 

  9. French SD, Green SE, OʼConnor DA, McKenzie JE, Francis JJ, Michie S, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7(1):38.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Presseau J, McCleary N, Lorencatto F, Patey AM, Grimshaw JM, Francis JJ. Action, actor, context, target, time (AACTT): a framework for specifying behaviour. Implement Sci. 2019;14(1):102.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Squires JE, Graham I, Bashir K, Nadalin-Penno L, Lavis J, Francis J, et al. Understanding context: A concept analysis. J Adv Nurs. 2019;75(12):3448–70.

    Article  PubMed  Google Scholar 

  14. Squires JE, Graham ID, Hutchinson AM, Michie S, Francis JJ, Sales A, et al. Identifying the domains of context important to implementation science: a study protocol. Implement Sci. 2015;10(1):135.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4(1):67.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Rogers EM, Cartano DG. Methods of measuring opinion leadership. Public Opin Q. 1962;26(3):435–41.

    Article  Google Scholar 

  18. Huybrechts I, Declercq A, Verté E, Raeymaeckers P, Anthierens S. The Building Blocks of Implementation Frameworks and Models in Primary Care: A Narrative Review. Front Public Health. 2021;9:675171. https://doi.org/10.3389/fpubh.2021.675171.

  19. Gilbert AW, Jaggi A, May CR. What is the patient acceptability of real time 1:1 videoconferencing in an orthopaedics setting? A systematic review. Physiotherapy. 2018;104(2):178–86.

    Article  PubMed  Google Scholar 

  20. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011;6:26.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, et al. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005;5(1):23.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Kastner M, Bhattacharyya O, Hayden L, Makarski J, Estey E, Durocher L, et al. Guideline uptake is influenced by six implementability domains for creating and communicating guidelines: a realist review. J Clin Epidemiol. 2015;68(5):498–509.

    Article  PubMed  Google Scholar 

  23. Shekelle P, Woolf S, Grimshaw JM, Schünemann HJ, Eccles MP. Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012;7(1):62.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Cumpston MS, Webb SA, Middleton P, Sharplin G, Green S, Australian Clinical Trials Alliance Reference Group on I, et al. Understanding implementability in clinical trials: a pragmatic review and concept map. Trials. 2021;22(1):232.

    Article  PubMed  PubMed Central  Google Scholar 

  25. World Health Organization. (n.d.). International Classification of Health Interventions (ICHI). WHO. Retrieved October 2021 from https://www.who.int/standards/classifications/international-classification-of-health-interventions

  26. Pollock A, Campbell P, Brunton G, Hunt H, Estcourt L. Selecting and implementing overview methods: implications from five exemplar overviews. Syst Rev. 2017;6(1):145.

    Article  PubMed  PubMed Central  Google Scholar 

  27. The EndNote Team. EndNote. EndNote. X9 ed. Philadelphia: Clarivate; 2013.

    Google Scholar 

  28. Van de Ven AH, Delbecq AL. The nominal group as a research instrument for exploratory health studies. Am J Public Health. 1972;62(3):337-42. https://doi.org/10.2105/ajph.62.3.337.

  29. Hussainy SY, Crum MF, White PJ, Larson I, Malone DT, Manallack DT, Nicolazzo JA, McDowell J, Lim AS, Kirkpatrick CM. Developing a Framework for Objective Structured Clinical Examinations Using the Nominal Group Technique. Am J Pharm Educ. 2016;80(9):158. https://doi.org/10.5688/ajpe809158.

  30. McMillan SS, Kelly F, Sav A, Kendall E, King MA, Whitty JA, Wheeler AJ. Using the Nominal Group Technique: how to analyse across multiple groups. Health Services and Outcomes Research Methodology. 2014;14(3):92-108.

  31. Pan H, Norris JL, Liang YS, Li JN, Ho MJ. Building a professionalism framework for healthcare providers in China: a nominal group technique study. Med Teach. 2013;35(10):e1531–6.

    Article  PubMed  Google Scholar 

  32. Rubin G, De Wit N, Meineche-Schmidt V, Seifert B, Hall N, Hungin P. The diagnosis of IBS in primary care: consensus development using nominal group technique. Fam Pract. 2006;23(6):687-92. https://doi.org/10.1093/fampra/cml050. Epub 2006 Oct 24.

  33. Sarre G, Cooke J. Developing indicators for measuring Research Capacity Development in primary care organizations: a consensus approach using a nominal group technique. Health Soc Care Community. 2009;17(3):244-53. https://doi.org/10.1111/j.1365-2524.2008.00821.x.

  34. Olsen J. The Nominal Group Technique (NGT) as a tool for facilitating pan-disability focus groups and as a new method for quantifying changes in qualitative data. Int J Qual Methods. 2019;18:1609406919866049.

    Article  Google Scholar 

  35. Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract. 2012;18(2):188–94.

    Article  PubMed  Google Scholar 

  36. Leape LL, Park RE, Kahan JP, Brook RH. Group judgments of appropriateness: the effect of panel composition. International J Qual Health Care. 1992;4(2):151–9.

    CAS  Google Scholar 

  37. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol. 2021;134:103–12.

    Article  PubMed  Google Scholar 

  38. Forbes CC, Finlay A, McIntosh M, Siddiquee S, Short CE. A systematic review of the feasibility, acceptability, and efficacy of online supportive care interventions targeting men with a history of prostate cancer. J Cancer Survivorship : Res Pract. 2019;13(1):75–96.

    Article  Google Scholar 

  39. Pham MD, Agius PA, Romero L, McGlynn P, Anderson D, Crowe SM, et al. Acceptability and feasibility of point-of-care CD4 testing on HIV continuum of care in low and middle income countries: a systematic review. BMC Health Serv Res. 2016;16(a):343.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Skea ZC, Aceves-Martins M, Robertson C, De Bruin M, Avenell A. Acceptability and feasibility of weight management programmes for adults with severe obesity: a qualitative systematic review. BMJ Open. 2019;9(9):e029473.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Marshall S, Vahabi M, Lofters A. Acceptability, Feasibility and uptake of HPV self-sampling among immigrant minority women: a focused literature review. J Immigr Minor Health. 2019;21(6):1380–93.

    Article  PubMed  Google Scholar 

  42. Babiano-Espinosa L, Wolters LH, Weidle B, Op de Beek V, Pedersen SA, Compton S, et al. Acceptability, feasibility, and efficacy of Internet cognitive behavioral therapy (iCBT) for pediatric obsessive-compulsive disorder: a systematic review. Syst Rev. 2019;8(1):284.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Gilbey D, Morgan H, Lin A, Perry Y. Effectiveness, Acceptability, and feasibility of digital health interventions for LGBTIQ+ young people: systematic review. J Med Internet Res. 2020;22(12):e20158.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Sheffield KM, Woods-Giscombé CL. Efficacy, Feasibility, and Acceptability of perinatal yoga on womenʼs mental health and well-being: a systematic literature review. J Holist Nurs. 2016;34(1):64–79.

    Article  PubMed  Google Scholar 

  45. GL D, De Crescenzo F, Minozzi S, Morgano GP, Mitrova Z, Scattoni ML, et al. Equity, acceptability and feasibility of using polyunsaturated fatty acids in children and adolescents with autism spectrum disorder: a rapid systematic review. Health Qual Life Outcomes. 2020;18(1):101.

    Article  Google Scholar 

  46. Li Y, Coster S, Norman I, Chien WT, Qin J, Ling Tse M, et al. Feasibility, acceptability, and preliminary effectiveness of mindfulness-based interventions for people with recent-onset psychosis: A systematic review. Early Interv Psychiatry. 2021;15(1):3–15.

    Article  CAS  PubMed  Google Scholar 

  47. Fish AF, Christman SK, Frid DJ, Smith BA, Bryant CX. Feasibility and acceptability of stepping exercise for cardiovascular fitness in women. Appl Nurs Res. 2009;22(4):274–9.

    Article  PubMed  Google Scholar 

  48. Heynsbergh N, Heckel L, Botti M, Livingston PM. Feasibility, useability and acceptability of technology-based interventions for informal cancer carers: a systematic review. BMC Cancer. 2018;18(1):244.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Hadgraft NT, Brakenridge CL, Dunstan DW, Owen N, Healy GN, Lawler SP. Perceptions of the acceptability and feasibility of reducing occupational sitting: review and thematic synthesis. Int J Behav Nutr Phys Act. 2018;15(1):90.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Pierret ACS, Anderson JK, Ford TJ. Burn A.-M. Review: Education and training interventions, and support tools for school staff to adequately respond to young people who disclose self-harm – a systematic literature review of effectiveness, feasibility and acceptability. Child Adolesc Ment Health. 2021. https://doi.org/10.1111/camh.12436

  51. Brooke-Sumner C, Petersen I, Asher L, Mall S, Egbe CO, Lund C. Systematic review of feasibility and acceptability of psychosocial interventions for schizophrenia in low and middle income countries. BMC Psychiatry. 2015;15:19.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Moulton-Perkins A, Moulton D, Cavanagh K, Jozavi A, Strauss C. Systematic review of mindfulness-based cognitive therapy and mindfulness-based stress reduction via group videoconferencing: Feasibility, acceptability, safety, and efficacy. Journal of Psychotherapy Integration. Advance online publication. 2020. https://doi.org/10.1037/int0000216.

  53. Shek AC, Biondi A, Ballard D, Wykes T, Simblett SK. Technology-based interventions for mental health support after stroke: A systematic review of their acceptability and feasibility. Neuropsychol Rehabil. 2021;31(3):432–52.

    Article  PubMed  Google Scholar 

  54. Padmanathan P, De Silva MJ. The acceptability and feasibility of task-sharing for mental healthcare in low and middle income countries: a systematic review. Soc Sci Med. 1982;2013(97):82–6.

    Google Scholar 

  55. Griffiths H. The acceptability and feasibility of using text messaging to support the delivery of physical health care in those suffering from a psychotic disorder: a review of the literature. Psychiatry Q. 2020;91(4):1305–16.

    Article  Google Scholar 

  56. Stephen C, McInnes S, Halcomb E. The feasibility and acceptability of nurse-led chronic disease management interventions in primary care: an integrative review. J Adv Nurs. 2018;74(2):279–88.

    Article  PubMed  Google Scholar 

  57. Tough D, Robinson J, Gowling S, Raby P, Dixon J, Harrison SL. The feasibility, acceptability and outcomes of exergaming among individuals with cancer: a systematic review. BMC Cancer. 2018;18(1):1151.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Pallas SW, Minhas D, Perez-Escamilla R, Taylor L, Curry L, Bradley EH. Community health workers in low- and middle-income countries: what do we know about scaling up and sustainability? Am J Public Health. 2013;103(7):e74–82.

    Article  PubMed  PubMed Central  Google Scholar 

  59. James HM, Papoutsi C, Wherton J, Greenhalgh T, Shaw SE. Spread, scale-up, and sustainability of video consulting in health care: systematic review and synthesis guided by the NASSS framework. J Med Internet Res. 2021;23(1):e23775.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Qiu D, Hu M, Yu Y, Tang B, Xiao S. Acceptability of psychosocial interventions for dementia caregivers: a systematic review. BMC Psychiatry. 2019;19(1):23.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Bautista T, James D, Amaro H. Acceptability of mindfulness-based interventions for substance use disorder: a systematic review. Complement Ther Clin Pract. 2019;35:201–7.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Sotirova MB, McCaughan EM, Ramsey L, Flannagan C, Kerr DP, OʼConnor SR, et al. Acceptability of online exercise-based interventions after breast cancer surgery: systematic review and narrative synthesis. J Cancer Survivorship : Res Pract. 2021;15(2):281–310.

    Article  Google Scholar 

  63. Simon N, McGillivray L, Roberts NP, Barawi K, Lewis CE, Bisson JI. Acceptability of internet-based cognitive behavioural therapy (i-CBT) for post-traumatic stress disorder (PTSD): a systematic review. Eur J Psychotraumatol. 2019;10(1):1646092.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Sprogis SK, Currey J, Considine J. Patient acceptability of wearable vital sign monitoring technologies in the acute care setting: a systematic review. J Clin Nurs. 2019;28(15-16):2732–44.

    Article  PubMed  Google Scholar 

  65. Griffin JB, Ridgeway K, Montgomery E, Torjesen K, Clark R, Peterson J, et al. Vaginal ring acceptability and related preferences among women in low- and middle-income countries: a systematic review and narrative synthesis. PLoS One. 2019;14(11):e0224898.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  66. Goldberg SB, Riordan KM, Sun S, Kearney DJ, Simpson TL. Efficacy and acceptability of mindfulness-based interventions for military veterans: a systematic review and meta-analysis. J Psychosom Res. 2020;138:110232.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res. 2017;17(1):88. https://doi.org/10.1186/s12913-017-2031-8.

  68. Cambridge University Press. (n.d.). Fidelity. In Cambridge dictionary. Retrieved March 2021, from https://dictionary.cambridge.org/dictionary/english/fidelity.

  69. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443–51.

    Article  PubMed  Google Scholar 

  70. Troy V, McPherson KE, Emslie C, Gilchrist E. The Feasibility, Appropriateness, meaningfulness, and effectiveness of parenting and family support programs delivered in the criminal justice system: a systematic review. J Child Fam Stud. 2018;27(6):1732–47.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Peltea A, Berghea F, Gudu T, Ionescu R. Knee ultrasound from research to real practice: a systematic literature review of adult knee ultrasound assessment feasibility studies. Med Ultrason. 2016;18(4):457–62.

    Article  PubMed  Google Scholar 

  72. Learmonth YC, Motl RW. Important considerations for feasibility studies in physical activity research involving persons with multiple sclerosis: a scoping systematic review and case study. Pilot Feasibility Stud. 2018;4:1.

    Article  PubMed  Google Scholar 

  73. Seitz DP, Brisbin S, Herrmann N, Rapoport MJ, Wilson K, Gill SS, et al. Efficacy and feasibility of nonpharmacological interventions for neuropsychiatric symptoms of dementia in long term care: a systematic review. J Am Med Dir Assoc. 2012;13(6):503–6.e2.

    Article  PubMed  Google Scholar 

  74. Chipps J, Brysiewicz P, Mars M. Effectiveness and feasibility of telepsychiatry in resource constrained environments? A systematic review of the evidence. Afr J Psychiatry. 2012;15(4):235–43.

    CAS  Google Scholar 

  75. Soneson E, Howarth E, Ford T, Humphrey A, Jones PB, Thompson Coon J, et al. Feasibility of school-based identification of children and adolescents experiencing, or at-risk of developing, mental health difficulties: a systematic review. Prev Sci. 2020;21(5):581–603.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, Bakken S, Kaplan CP, Squiers L, Fabrizio C, Fernandez M. How we design feasibility studies. Am J Prev Med. 2009;36(5):452-7. https://doi.org/10.1016/j.amepre.2009.02.002.

  77. Chapman DJ, Morel K, Anderson AK, Damio G, Perez-Escamilla R. Breastfeeding peer counseling: from efficacy through scale-up. J Hum Lact. 2010;26(3):314–26.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Troup J, Fuhr DC, Woodward A, Sondorp E, Roberts B. Barriers and facilitators for scaling up mental health and psychosocial support interventions in low- and middle-income countries for populations affected by humanitarian crises: a systematic review. Int J Mental Health Syst. 2021;15(1):5.

    Article  Google Scholar 

  79. Ben Charif A, Zomahoun HTV, LeBlanc A, Langlois L, Wolfenden L, Yoong SL, et al. Effective strategies for scaling up evidence-based practices in primary care: a systematic review. Implement Sci. 2017;12(1):139.

    Article  PubMed  PubMed Central  Google Scholar 

  80. McCrabb S, Lane C, Hall A, Milat A, Bauman A, Sutherland R, et al. Scaling-up evidence-based obesity interventions: a systematic review assessing intervention adaptations and effectiveness and quantifying the scale-up penalty. Obes Rev. 2019;20(7):964–82.

    Article  PubMed  Google Scholar 

  81. World Health Organization. Practical guidance for scaling up health service innovations. Geneva: World Health Organization; 2009. Available from: https://apps.who.int/iris/bitstream/handle/10665/44180/9789241598521_eng.pdf;jsessionid=FFD201E7790BD61165F70FF4F21A6AE3?sequence=1

    Google Scholar 

  82. Milat A, Lee K, Conte K, Grunseit A, Wolfenden L, van Nassau F, et al. Intervention scalability assessment tool: a decision support tool for health policy makers and implementers. Health Res Policy Syst. 2020;18(1):1.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Bradley EH, Curry LA, Taylor LA, Pallas SW, Talbert-Slagle K, Yuan C, Fox A, Minhas D, Ciccone DK, Berg D, Pérez-Escamilla R. A model for scale up of family health innovations in low-income and middle-income settings: a mixed methods study. BMJ Open. 2012;2(4):e000987. https://doi.org/10.1136/bmjopen-2012-000987.

  84. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367.

    Article  PubMed  PubMed Central  Google Scholar 

  85. Hailemariam M, Bustos T, Montgomery B, Barajas R, Evans LB, Drahota A. Evidence-based intervention sustainability strategies: a systematic review. Implement Sci. 2019;14(1):57.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Francis L, Dunt D, Cadilhac DA. How is the sustainability of chronic disease health programmes empirically measured in hospital and related healthcare services?-a scoping review. BMJ Open. 2016;6(5):e010944.

    Article  PubMed  PubMed Central  Google Scholar 

  87. Braithwaite J, Ludlow K, Testa L, Herkes J, Augustsson H, Lamprell G, et al. Built to last? The sustainability of healthcare system improvements, programmes and interventions: a systematic integrative review. BMJ Open. 2020;10(6):e036453.

    Article  PubMed  PubMed Central  Google Scholar 

  88. Ishola F, Cekan J. Evaluating the sustainability of health programmes: A literature review. African Evaluation Journal. 2019; 7(1): 1–7. https://doi.org/10.4102/aej.v7i1.369.

  89. Mok WKH, Sharif R, Poh BK, Wee LH, Reilly JJ, Ruzita AT. Sustainability of childhood obesity interventions: a systematic review. Pak J Nutr. 2019;18:603–14.

    Article  Google Scholar 

  90. Ament SM, de Groot JJ, Maessen JM, Dirksen CD, van der Weijden T, Kleijnen J. Sustainability of professionalsʼ adherence to clinical practice guidelines in medical care: a systematic review. BMJ Open. 2015;5(12):e008073.

    Article  PubMed  PubMed Central  Google Scholar 

  91. Lauckner C, Whitten P. The state and sustainability of telepsychiatry programs. J Behav Health Serv Res. 2016;43(2):305–18.

    Article  PubMed  PubMed Central  Google Scholar 

  92. Herlitz L, MacIntyre H, Osborn T, Bonell C. The sustainability of public health interventions in schools: a systematic review. Implement Sci. 2020;15(1):4.

    Article  PubMed  PubMed Central  Google Scholar 

  93. Flynn R, Newton AS, Rotter T, Hartfield D, Walton S, Fiander M, et al. The sustainability of Lean in pediatric healthcare: a realist review. Syst Rev. 2018;7(1):137.

    Article  PubMed  PubMed Central  Google Scholar 

  94. Crespo-Gonzalez C, Benrimoj SI, Scerri M, Garcia-Cardenas V. Sustainability of innovations in healthcare: A systematic review and conceptual framework for professional pharmacy services. Res Social Adm Pharm. 2020;16(10):1331–43.

    Article  PubMed  Google Scholar 

  95. Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12(1):110.

    Article  PubMed  PubMed Central  Google Scholar 

  96. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13(1):87-108. https://doi.org/10.1093/her/13.1.87.

  97. Banyai I. Zoom. New York: Viking; 1995.

    Google Scholar 

  98. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):117.

    Article  PubMed  PubMed Central  Google Scholar 

  99. Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health. 2019;7(64):64.

    Article  PubMed  PubMed Central  Google Scholar 

  100. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    Article  PubMed  Google Scholar 

  101. Sohn H, Tucker A, Ferguson O, Gomes I, Dowdy D. Costing the implementation of public health interventions in resource-limited settings: a conceptual framework. Implement Sci. 2020;15(1):86.

    Article  PubMed  PubMed Central  Google Scholar 

  102. Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019;22(4):785–801.

    Article  PubMed  PubMed Central  Google Scholar 

  103. Patton DE, Pearce CJ, Cartwright M, Smith F, Cadogan CA, Ryan C, et al. A non-randomised pilot study of the Solutions for Medication Adherence Problems (S-MAP) intervention in community pharmacies to support older adults adhere to multiple medications. Pilot Feasibility Stud. 2021;7(1):18.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  104. Bulthuis SE, Kok MC, Raven J, Dieleman MA. Factors influencing the scale-up of public health interventions in low- and middle-income countries: a qualitative systematic literature review. Health Policy Plan. 2020;35(2):219–34.

    Article  PubMed  Google Scholar 

  105. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26.

    Article  PubMed  PubMed Central  Google Scholar 

  106. Bernet AC, Willens DE, Bauer MS. Effectiveness-implementation hybrid designs: implications for quality improvement science. Implement Sci. 2013;8(Suppl 1):S2.

    Article  PubMed Central  Google Scholar 

  107. Landes SJ, McBain SA, Curran GM. An introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2019;280:112513.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. The University of Melbourne, School of Health Sciences, Melbourne, Australia

    Marlena Klaic, Peter Hudson, Linda Denehy & Jill J. Francis

  2. The Royal Melbourne Hospital, Allied Health Department, Melbourne, Australia

    Marlena Klaic

  3. The University of Melbourne, School of Health Sciences, Faculty of Medicine, Dentistry and Health Sciences, Department of Nursing, Melbourne, Australia

    Suzanne Kapp

  4. Centre for Palliative Care, St Vincent’s Hospital, Melbourne, Australia

    Peter Hudson

  5. End-of-life Care Research Department, Vrije Universiteit Brussel (VUB), Brussels, Belgium

    Peter Hudson

  6. Centre for Digital Transformation of Health, The University of Melbourne, Melbourne, Australia

    Wendy Chapman

  7. Department of Allied Health, Peter McCallum Cancer Centre, Melbourne, Australia

    Linda Denehy

  8. Department of Critical Care, The University of Melbourne, Melbourne, Australia

    David Story

  9. Department of Anaesthesia, Austin Health, Melbourne, Australia

    David Story

  10. Ottawa Hospital Research Institute, Clinical Epidemiology Program, Ottawa, Canada

    Jill J. Francis

  11. Department of Health Services Research, Peter McCallum Cancer Centre, Melbourne, Australia

    Jill J. Francis

Authors
  1. Marlena Klaic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suzanne Kapp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Hudson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wendy Chapman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Linda Denehy
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David Story
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jill J. Francis
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MK completed the systematic searches, synthesis of data, framework design and substantial writing in collaboration with JF. SK was the second reviewer for 10% of the literature included in the review and commented on drafts. PH commented on drafts and provided a clinical scenario. WC commented on drafts and provided a clinical scenario. LD commented on drafts and provided a clinical scenario. DS commented on drafts and provided a clinical scenario. JF contributed the original idea for the framework, coordinated the authorship team and substantial writing in collaboration with MK. All authors participated in the consensus process and approved the final submitted version of the manuscript.

Corresponding author

Correspondence to Marlena Klaic.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Search strategy.

Additional file 2.

Papers included in the overview.

Additional file 3.

Characteristics of studies included in the overview of reviews (n=252).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Klaic, M., Kapp, S., Hudson, P. et al. Implementability of healthcare interventions: an overview of reviews and development of a conceptual framework. Implementation Sci 17, 10 (2022). https://doi.org/10.1186/s13012-021-01171-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-021-01171-7

Keywords

Implementation Science

ISSN: 1748-5908