State Adoption of Incentives to Promote Evidence-Based Practices in Behavioral Health Systems
Abstract
Objective:
Despite the critical role behavioral health care payers can play in creating an incentive to use evidence-based practices (EBPs), little research has examined which incentives are used in public mental health systems, the largest providers of mental health care in the United States.
Methods:
The authors surveyed state mental health directors from 44 states about whether they used any of seven strategies to increase the use of EBPs. Participants also ranked attributes of each incentive on the basis of key characteristics of diffusion of innovation theory (perceived advantage, simplicity, compatibility, observability, and gradually implementable) and perceived effectiveness.
Results:
Almost three-quarters of state directors endorsed using at least one financial incentive; most paid for training and technical assistance. Few used other incentives. Strategies perceived as simple and compatible were more readily adopted. Enhanced rates and paying for better outcomes were perceived as the most effective but were the least deployed, suggesting that simplicity and organizational compatibility may be the most decisive factors when choosing incentives.
Conclusions:
Payers are not using the incentives they perceive as most effective, and they are mostly using only one strategy for reasons of simplicity and compatibility. Future work should focus on barriers to measurement that likely hinder the adoption and implementation of paying for better outcomes and enhanced reimbursement rates, with the ultimate goal of measuring the effectiveness of incentives on EBP implementation efforts.
Many public behavioral health systems are actively trying to increase the availability of evidence-based practices (EBPs) for the people they serve. A rapidly expanding list of state and county systems encourage agencies to use EBPs by offering them incentives or instituting mandates. Free training and technical assistance are often provided as well (1,2). Despite this momentum and the growing consensus that EBPs are critical to providing quality care, they are still used infrequently (3,4). One explanation for this enduring research–practice gap is that the policies used to encourage the implementation of EBPs are themselves not effective (5).
Financing and establishing regulatory policy are important drivers of EBP use in almost every published conceptual framework of factors affecting implementation (6). Raghavan and colleagues (5) have suggested that payers have at their disposal the following policy mechanisms to increase the use of EBPs within agencies: implementing enhanced reimbursement strategies for the use of EBPs, training providers and consumers in EBPs, publicly recognizing high-performing agencies, reducing regulatory burden, giving agencies that use EBPs advantages in competitive bidding processes (fast-tracking), paying for better outcomes, and directing patient traffic toward agencies using EBPs. In 2003, the President’s New Freedom Commission on Mental Health urged the public sector to provide financial incentives to increase the use of EBPs (7). Few of the other incentives described by Raghavan and colleagues have been widely promoted, and there has been almost no measurement of whether and how incentives are used in practice (8–10).
State behavioral health systems are in a unique and powerful position to promote the use of EBPs. Although ultimately we are interested in how financial incentives may promote the use of EBPs, we must first establish the extent to which different incentive strategies are used. In this study, we gathered information about states’ use of each incentive and state officials’ perceptions of the characteristics of these incentives. We characterized these incentives as innovations that states choose to adopt or not adopt and therefore used diffusion of innovation (DOI) theory (11) as a framework. The DOI framework has been used across varied disciplines (for example, agriculture, medicine, and information technology) to understand the adoption of new innovations or practices. It posits five key characteristics of an innovation that influence the decision to adopt: perceived advantage, compatibility with existing procedures, simplicity, gradually implementable, and observable. Perceived advantage is the benefit that the innovation appears to provide over current procedures. Compatibility reflects the congruence between the innovation and existing procedures. Simplicity describes how readily alterations in extant procedures could be accommodated.
DOI theory posits that an innovation is more likely to be accepted if it can be gradually implemented in small steps on a trial basis (trialability) and if the results are observable, or visible to others. Moreover, these key attributes are critical factors that encourage the adoption of any innovation—in this case, innovations to promote EBP. In his synthesis of diffusion studies, Rogers (11) suggested that perceived advantage, simplicity, and compatibility are key factors that explain adoption decisions, whereas observability and trialability are not as consistently important. We therefore hypothesized that incentives that hold relative advantage and are simple and compatible would be more likely to be implemented.
Methods
Participants and Procedures
We partnered with the largest organization of state mental health directors, the National Association of State Mental Health Program Directors, to distribute a brief survey. We sent an e-mail inviting directors (or their designated proxies) to complete a survey on incentives and EBPs. Several reminders were sent. Data were collected from representatives of 44 states (response rate=88%), representing 90% of the U.S. population (12). Data were collected between October 2016 and April 2017. All procedures were approved by the University of Pennsylvania Institutional Review Board.
Measures
Our survey instrument consisted of 21 items. Efforts were made to minimize the number of items to enhance the response rate. Respondents were queried on their use of seven different innovations described by Raghavan and colleagues (5) and the probability that they would implement each incentive in the future to promote the use of EBPs. Respondents were asked to endorse whether each incentive met the criteria for each of the five DOI characteristics (perceived advantage, compatibility with existing procedures, simplicity, gradually implementable, observable). Finally, respondents were asked to endorse (on a five-point scale) how effective each incentive would be in promoting the adoption of EBPs if the barriers to these incentives were removed. The survey was developed and field tested in consultation with an advisory board of stakeholders who had administrative and research experience in public mental health systems.
Statistical Analysis
For each incentive, we calculated the percentages for states’ use, endorsement of each of the five DOI attributes, and perception of perceived effectiveness (N=44; see Table 1). We then examined the association between these seven summary statistics with Pearson product–moment correlation coefficients to assess the relationships between endorsement of each DOI key attribute and use of incentives.
DOI characteristicb | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Implementation | Better than what we are doing now to promote EBPs | Compatible with existing procedures and infrastructure | Simple to implement | Gradually implemented in small steps or on a trial basis | Effects immediately observable | Effectivenessc | ||||||||
Incentive type | N | % | N | % | N | % | N | % | N | % | N | % | M | SD |
Paid for training and technical assistance | 27 | 61 | 6 | 14 | 29 | 66 | 42 | 95 | 8 | 18 | 7 | 27 | 3.79 | .94 |
Publicly recognized high-performing EBP providers | 10 | 23 | 12 | 27 | 22 | 50 | 34 | 77 | 12 | 27 | 5 | 10 | 3.29 | .77 |
Directed patients to providers who use EBPs | 8 | 18 | 7 | 16 | 13 | 30 | 30 | 68 | 9 | 20 | 4 | 8 | 3.64 | .96 |
Enhanced reimbursement rates | 5 | 11 | 18 | 41 | 14 | 32 | 31 | 70 | 22 | 50 | 4 | 5 | 4.40 | .80 |
Reduced regulatory burden for providers who provide EBP | 4 | 9 | 8 | 18 | 7 | 16 | 16 | 36 | 13 | 30 | 7 | 4 | 3.55 | .94 |
Paid agencies for better clinical outcomes | 4 | 9 | 15 | 34 | 10 | 23 | 23 | 52 | 23 | 52 | 9 | 4 | 4.24 | .98 |
Fast-tracked providers using EBPs in the competitive bidding process | 1 | 2 | 2 | 5 | 7 | 16 | 18 | 41 | 9 | 20 | 1 | 1 | 3.24 | 1.23 |
Results
Overall, 71% of states endorsed using at least one incentive to promote the use of EBPs. Table 1 shows that 61% paid for training and technical assistance. Fewer than one-quarter reported using each of the other strategies: publicly recognizing high-performing EBP providers (23%), directing patients to providers who use EBPs (18%), enhanced reimbursement rates (11%), paying agencies for better clinical outcomes (9%), and fast-tracking providers using EBPs in the competitive bidding process (2%). The two incentives rated as most likely to be effective if implemented were enhanced reimbursement rates and paying for better outcomes.
Figure 1 displays the correlations between the use of incentives and the DOI constructs. Perception of the innovation’s compatibility with existing procedures and its use were positively correlated (r=.92, p=.003). There was also a positive correlation between simplicity and use (r=.85, p<.017).
Discussion
The majority of expenditures in the behavioral health marketplace are publicly funded (13). State governments, the largest payers in behavioral health, therefore have much at stake in designing and implementing effective incentives to improve care. We found that the most commonly used incentive to increase the use of EBPs providing training and technical assistance. Few states do anything else, despite the growing evidence and consensus that, as the primary vehicle of EBP implementation efforts in the public sector, training is necessary but not sufficient (14,15). Training alone does not lead to change in practice (16,17)
On average, state directors rated training as both compatible and simple to use. However, it was rated as much less effective and much lower on all other DOI attributes than enhanced reimbursement rates and paying for better outcomes, which were among the least used strategies. This finding suggests that, of the DOI constructs, simplicity and compatibility may be particularly important in driving the adoption of financial incentives to promote EBPs in public behavioral health systems.
We have two hypotheses about why enhanced reimbursement rates and paying for better outcomes are not used, despite their being rated as effective and advantageous. First, of the seven incentives, these two require by far the most measurement. The adoption of enhanced reimbursement rates and paying for better outcomes to promote EBPs, although simple in concept, are extraordinarily complex in execution. They require sophisticated data collection and analysis, and payers must have the financial capacity to pay for incentives (18). Although there are many validated measures of symptoms and functioning, there are no definitive and few widely accepted behavioral health outcome measures, such as the presence or absence of disease or mortality. Some have argued that incentives, metrics, and populations have not been adequately designed or identified to ensure (and measure) quality improvement through incentives (19–21). Helping states implement enhanced rates and paying for outcomes will first require more research and support to help these systems implement comprehensive and easy-to-use approaches to measuring outcomes.
Second, enhanced reimbursement rates and paying for better outcomes have the potential to create inequity in the public behavioral health care system, with some providers benefiting more than others, which can in turn affect providers’ bottom line, fracture relationships between payers and providers, or even reduce access to care if providers leave the network. Any implementation in a behavioral health care system requires a delicate dance of collaboration, coordination, and negotiation among all stakeholders, particularly given rapidly contracting state behavioral health budgets (22,23). Collaborative stakeholder approaches to identifying fair and transparent incentives would be particularly beneficial to this end (24). Future research that includes qualitative study might allow for a more detailed assessment of and insight into these challenges while identifying specific barriers that may be common across settings.
It is important to acknowledge some limitations of this research. First, state mental health directors and their designated proxies may not be fully informed about all state behavioral health initiatives, localized efforts, or pilot projects. The turnover in state leadership amplifies this potential missed information (personal communication, Hepburn B, 2017). We also recognize that in some states, the county may be a more meaningful unit of analysis in terms of the role of payment, policy, and management, and county-level representation was not included in this study. In addition, state policies and financing change frequently, particularly since passage of the Affordable Care Act. The findings presented in this article are a snapshot of an evolving process as it appeared in 2016.
Conclusions
Despite these limitations, this study contributes new information about the financial incentives used by the biggest players in the behavioral health marketplace to promote EBPs. To our knowledge, this is the first national survey of state-level payers regarding the financial incentives they use to increase the use of EBPs. Payers are not using the incentives they perceive to be most effective, and they are largely only using one strategy for reasons of simplicity and compatibility. Future work should focus on barriers to measurement that likely hinder the adoption and implementation of enhanced reimbursement rates and paying for better outcomes, with the ultimate goal of measuring the effectiveness of incentives in EBP implementation efforts.
1 : Scaling up evidence-based practices for children and families in New York State: toward evidence-based policies on implementation for state mental health systems. Journal of Clinical Child and Adolescent Psychology 43:145–157, 2014Crossref, Medline, Google Scholar
2 : A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research 43:893–908, 2016Crossref, Medline, Google Scholar
3 : The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. American Psychologist 65:73–84, 2010Crossref, Medline, Google Scholar
4 : Research, data, and evidence-based treatment use in state behavioral health systems, 2001–2012. Psychiatric Services 67:496–503, 2016Link, Google Scholar
5 : Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implementation Science 3:26, 2008Crossref, Medline, Google Scholar
6 : Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science 4:50, 2009Crossref, Medline, Google Scholar
7 Achieving the Promise: Transforming Mental Health Care in America. Pub no SMA-03-3832. Rockville, MD: New Freedom Commission on Mental Health, 2003Google Scholar
8 : State policy influence on the early diffusion of buprenorphine in community treatment programs. Substance Abuse Treatment, Prevention, and Policy 3:17, 2008Crossref, Medline, Google Scholar
9 : Employing policy and purchasing levers to increase the use of evidence-based practices in community-based substance abuse treatment settings: reports from single state authorities. Evaluation and Program Planning 34:366–374, 2011Crossref, Medline, Google Scholar
10 : Implementation of evidence-based practices for treatment of alcohol and drug disorders: the role of the state authority. Journal of Behavioral Health Services and Research 36:407–419, 2009Crossref, Medline, Google Scholar
11 : Diffusion of Innovations, 5th ed. New York, Simon & Schuster, 2003Google Scholar
12 State Population Totals Tables: 2010–2016. Washington, DC, US Bureau of the Census, 2017. https://www.census.gov/data/datasets/2017/demo/popest/state-total.htmlGoogle Scholar
13 : Mental Health Financing in the United States: A Primer. Washington, DC, Kaiser Commission on Medicaid and the Uninsured, 2011 https://kaiserfamilyfoundation.files.wordpress.com/2013/01/8182.pdfGoogle Scholar
14 : The role of state mental health authorities in promoting improved client outcomes through evidence-based practice. Community Mental Health Journal 41:347–363, 2005Crossref, Medline, Google Scholar
15 : The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clinical Psychology Review 30:448–466, 2010Crossref, Medline, Google Scholar
16 : Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice 17:1–30, 2010Crossref, Medline, Google Scholar
17 : Is supervision necessary? Examining the effects of Internet-based CBT training with and without supervision. Journal of Consulting and Clinical Psychology 84:191–199, 2016Crossref, Medline, Google Scholar
18 : Can we pay for performance in behavioral health care? Psychiatric Services 68:109–111, 2017Link, Google Scholar
19 : Time to get serious about pay for performance. JAMA 309:347–348, 2013Crossref, Medline, Google Scholar
20 : Designing smarter pay-for-performance programs. JAMA 312:2617–2618, 2014Crossref, Medline, Google Scholar
21 : Influencing quality of outpatient SUD care: implementation of alerts and incentives in Washington State. Journal of Substance Abuse Treatment 82:93–101, 2017Crossref, Medline, Google Scholar
22 : State Mental Health Cuts: The Continuing Crisis. Arlington, VA, National Alliance on Mental Illness, 2011Google Scholar
23 : The perfect storm: collision of the business of mental health and the implementation of evidence-based practices. Psychiatric Services 67:159–161, 2016Link, Google Scholar
24 : A collaborative approach to identifying effective incentives for mental health clinicians to improve depression care in a large managed behavioral healthcare organization. Administration and Policy in Mental Health and Mental Health Services Research 38:193–202, 2011Crossref, Medline, Google Scholar