INTRODUCTION

Although complex, high-need patients account for the majority of health care spending,1 the use of predictive analytics for pro-active patient management of high-risk populations has been limited. The Veterans Health Administration (VHA) developed the Care Assessment Needs (CAN) score2 to help primary care teams identify high-risk patients. The CAN score reflects clinical and demographic characteristics that predicted future hospitalization and mortality for 4,598,408 VHA primary care patients2 with robust areas under the curve (AUCs) for predicting hospitalization (0.84), death (0.86), and hospitalization and/or deaths (0.82). The original CAN score algorithm had 90 input variables; the current version has 36 variables and has similar predictive accuracy. All VHA primary care providers and teams have access to a dashboard of CAN scores for their patient panels calculated weekly. The CAN score is expressed as a percentile of probabilities ranging from 0 percentile (lowest risk) to 99th percentile (highest risk). In this paper, we describe the population identified by the CAN score report and assess primary care team experience using the CAN score.

METHODS

Demographic and clinical characteristics and VHA utilization were assessed for VHA-enrolled primary care patients with CAN scores in the top quartile (n = 1,718,558) during 2016. We assessed CAN score use per month during a 3-month observation period (2/2016–4/2016), which was tracked electronically. We assessed user acceptability of CAN scores through an online survey of primary care providers and nurses who logged into the CAN score report during the observation period. The 5-item 5-point Likert scales in our survey were based on the 10-item Systems Usability Survey.3 Responses ranged from strongly agree to strongly disagree with items assessing CAN score accuracy, frequency, and usability (Table 2). We calculated item frequencies and conducted a retrospective content analysis of the survey’s open-text comment field.

RESULTS

The CAN algorithm identifies a high-risk population with multiple comorbidities at high risk for poor outcomes (Table 1). We found that there was wide variability in use of CAN scores among 8650 primary care providers and 6433 nurse care managers. During the three survey months, there were an average of 6450 uses of the CAN report among an average of 1850 individual users. Because surveys were anonymous, we were unable to count the number of unique respondents from the n = 400 responses we received. A majority of respondents reported that they use the score regularly (68%), that the score accurately represented their patient population (69%), and that they were confident in their ability to use the CAN score for clinical practice (72%) (Table 2). Qualitative responses indicated CAN scores were often used for care coordination or palliative care services. Respondents also indicated a need for additional help using the score, and for the CAN system to be more interactive and more fully integrated in the electronic health record.

Table 1 Patient demographics, comorbidities (identified by ICD-9 codes), and VHA utilization among Veterans enrolled in primary care in 2016, by CAN score percentile
Full size table
Table 2 Responses from n = 400 primary care personnel regarding CAN report
Full size table

DISCUSSION

Since 2012, VHA has used a predictive model to identify patients at highest risk for hospitalization and death for primary care teams. Among clinicians who used the CAN report, most believed that the CAN score accurately identified their high-risk patients, and that they used the score regularly. Comments indicated, however, similarly to other studies of predictive analytics,4 that CAN score results were not always clinically actionable. The limited number of report users compared with potential users illustrates the challenge of integrating a risk score into clinical practice workflow.

The results we report are preliminary, and limited by our low survey response rate. Our responses may be biased in favor of clinicians with greater CAN score familiarity. Nonetheless, our results provide a starting point for understanding and maximizing the clinical utility of predictive indexes in routine practice. The CAN score is being applied in a number of quality improvement projects and research studies with the goal of learning not only the effectiveness of intensive management for these patients but also the clinical processes needed to make use of CAN score information.5, 6 The CAN score illustrates how a health care system can harness electronic health record data to identify high-risk patients who may benefit from clinical interventions. Additional work is needed to fully characterize the heterogeneity of these patients’ needs, and to determine the best methods for using the score to target patients in need of specific services.