A guide to scaling up population health interventions January 2016, Volume 26, Issue 1

Introduction

Population-wide health improvements depend on wide-scale implementation of health interventions that have been found to be effective in a controlled research setting. This usually requires scaling up interventions to reach broader populations or multiple settings. Although the scale of an intervention may seem an obvious concept, the terms ‘scalability’ and ‘scaling up’ have been applied in different ways and contexts.¹ They have been variously applied in the literature to describe the dissemination of a new technique, prototype product or process innovation; epidemiological and economic forecasting; ‘growing’ an organisational or system capacity to implement to a new level; and translating a small-scale initiative into a government policy.^2,3

For clarity, specific definitions for ‘scalability’ and ‘scaling up’ are needed. In the health sector, scalability is “… the ability of a health intervention shown to be efficacious on a small scale and or under controlled conditions to be expanded under real-world conditions to reach a greater proportion of the eligible population, while retaining effectiveness”.⁴ Scaling up is “… deliberate efforts to increase the impact of successfully tested health interventions so as to benefit more people and to foster policy and program development on a lasting basis”.⁵ Scalability is the capacity of an individual intervention to be scaled up, while scaling up is a process of increasing the scale of a successfully tested health intervention. Of course, some health interventions that have been shown to be effective in research contexts will never be scalable because of a lack of strategic alignment, cost or incompatibility with existing infrastructure.

Guide to scaling up

The issue of how to scale up health interventions is receiving greater attention in the health literature.^1,4-7 Although a number of frameworks and guides exist^4,5,8 they have not been informed by empirical examination of how scaling up decision making and processes occur in high-income countries. To fill this gap, the New South Wales (NSW) Ministry of Health developed Increasing the scale of population health interventions: a guide.⁹ The guide was developed using a systematic review of scaling up models and methods³, and a two-round Delphi process with a sample of senior policy makers, practitioners and researchers actively involved in scaling up processes.^2,10 It was developed as a practical guide to assist health policy makers, health practitioners, and others with responsibility for scaling up effective population health interventions.⁹ The guide is divided into four steps: scalability assessment, developing a scaling up plan, preparing for scaling up, and scaling up the intervention. This article provides a concise synthesis of the guide’s key scale-up steps (Table 1).

Table 1.     Steps in the scaling up process

Source: Milat et al.⁹

Although the guide is written as if the user is starting from the point of assessing the scalability of an intervention, later steps can be used by those already involved in scaling up to review their implementation processes. The guide is not intended to be prescriptive. Its purpose is to help policy makers, practitioners, researchers and other decision makers make appropriate methodological and practical choices. It has grown out of population health experience and is therefore written from a population health perspective; however, its core concepts could be applied to other human service endeavours, particularly those that involve programmatic interventions.

The guide can also be used by researchers in the design of research studies that may be suitable for scaling up. For example, step 1 could be used to identify research gaps and to guide researchers towards seeking funding to address such gaps. Similarly, it may be used to assist researchers to present intervention research findings, thus providing the information necessary for health practitioners and policy makers to assess the scalability of an intervention. In addition, the later stages of the guide can be used by researchers to identify opportunities for partnering with evaluation and monitoring efforts when interventions are scaled up.

Evidence of effectiveness

A number of key concepts in the guide merit closer examination. The key prerequisite for scaling up a population health intervention is that it is effective.^4-6 For the purpose of this guide, an effective intervention is one that has achieved significant intervention effects either through a highly controlled efficacy study or an effectiveness study implemented in more diverse real-world contexts.

Ideally, evidence of effectiveness should be provided from randomised controlled research trials; however, it is increasingly acknowledged that evidence of effectiveness, especially for public health interventions, can be derived from a broader range of research designs, including pragmatic trials, quasi-experimental studies, cluster randomised trials, the relatively new ‘stepped wedge design’, and multiple baseline designs.^11,12 In addition, natural experimental studies may be the only option when it is impossible to manipulate exposure to the intervention.¹³ As with other quasi‐experimental designs, natural experiments cannot definitively determine causation and are subject to bias; however, they can provide a useful inferential tool that may not be obtainable in any other way.

It is also important to consider whether the size of the effect of the intervention is known and whether this may be of policy significance. Effects of interventions are likely to be smaller as they are scaled up; therefore, relatively large effect sizes should be demonstrated in the efficacy stage if an acceptable level of effect is to be maintained when interventions are scaled up.⁴ This reduction in effect is in part because of difficulties maintaining the dose and fidelity of the original intervention in real-world settings. It is rare for interventions to remain unchanged as they are scaled up, because of the need to adapt them to suit the local context and the organisational, financial and human resources available for scaling up.^4,6,10 These adaptations may reduce effectiveness, but they can improve acceptability and efficiency, highlighting the importance of measuring intervention effectiveness throughout the scaling up process.

Reach and adoption are at the heart of scalability.⁴ Reach refers to the level of individual participation of an intended target population in an intervention.¹⁴ Adoption is the proportion of settings, practices or organisations that adopt an intervention.¹⁴ In either case, it is important that interventions reach as large a proportion as possible of those eligible to receive them once they are scaled up, and that interventions are adopted by as large a proportion of eligible settings as possible. Whether an intervention has differential rates of reach and adoption should also be determined.

It is important to acknowledge that all the information that would ideally be available to assess the scalability of an intervention may not be available at the time the assessment is completed. For example, it might not be possible to accurately determine how large the effect size at a population level needs to be to achieve a population health gain, or how much the program can be changed (to reduce cost or suit different contexts) while still retaining fidelity and outcomes. Where there are gaps in the available research evidence, decision makers may need to consider information from other sources such as expert advice, practice-based knowledge or parallel evidence from similar programmatic interventions. Where no information is available, a judgement is required about how important the missing information is, whether any gaps can be addressed during implementation, or whether further research is required before scaling up can be recommended.

A central part of any scaling up process is selecting the approach. Step 2 outlines the two main approaches: vertical and horizontal.⁵ Scaling up using a vertical approach involves the introduction of an intervention simultaneously across a whole system and results in institutionalisation of a change through policy, regulation, financing or health systems. Examples of successful vertically scaled up interventions include the introduction of mandatory seatbelt legislation¹⁵, smoking bans in outdoor eating areas¹⁶ and the introduction of new health system financing models.¹⁷ Scaling up using a horizontal approach involves the introduction of an intervention across different sites or groups in a phased manner, often beginning with a pilot program, followed by stepwise expansion, learning lessons along the way to help refine further expansion. Examples of successful horizontally scaled up interventions include the stepwise expansion of the effective falls prevention in older people intervention, Stepping On¹⁸, in Local Health Districts across NSW, and the expansion of lifestyle-based diabetes prevention programs in community settings in the US using a YMCA implementation model.¹⁹

Evaluation and monitoring

It is also important that appropriate evaluation frameworks are built into intervention delivery from the outset⁴, particularly for steps 3 and 4. Formative evaluation before scale-up will be required to test the appropriateness and acceptability of the scaled up intervention with the target audience and other stakeholders. Subsequent evaluation and monitoring efforts during scale-up should focus on measuring effectiveness over time, rates of reach and adoption, acceptability, compatibility with existing interventions and costs. The emphasis placed on measuring each of these aspects during scale-up will depend on what is already known about the intervention (gaps in evidence identified in step 1) and the approach to scaling up the intervention. A scarcity of resources often means that key program outcomes, which require specific data collection efforts, are either not measured or are measured with data that may or may not be intended for that purpose.² In light of this, questions of the validity of performance measures are important, as is an understanding of the limitations of using performance data to inform decision making.

Conclusion

Although the issues outlined in the guide are important, many issues surrounding the scale-up of interventions are still being debated, and methods will continue to develop. However, it is clear from the literature and our research that, despite the unpredictable outcomes of scaling up attempts, a good plan can guide the scaling up process in the right direction and make success more likely.^2,3 Moreover, successful scale-up requires systematic use of evidence², and it is particularly important that data from monitoring is linked to decision making throughout the scaling up process. This guide argues that plans for scaling up need to consider a broad range of factors and balance what is desirable with what is feasible.

The publication, Increasing the scale of population health interventions: a guide, can be downloaded from the NSW Ministry of Health’s website

Copyright:

© 2016 Milat et al. This article is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence, which allows others to redistribute, adapt and share this work non-commercially provided they attribute the work and any adapted version of it is distributed under the same Creative Commons licence terms.