BACKGROUND

The Centers for Disease Control and Prevention estimates that more than 1 in 3 American adults have prediabetes and are at risk of progression to type 2 diabetes and it is deleterious long-term complications.1 The Diabetes Prevention Program (DPP) randomized controlled trial showed that the intensive lifestyle change reduced the incidence of progression to type 2 diabetes by 58% and metformin decreased the risk by 31%.2 Moreover, the follow-up DPP Outcomes Study demonstrated continuing positive health outcomes at 10- and 15-year follow-ups.3, 4 Despite this strong evidence supporting the importance of diabetes prevention, widespread translation of the DPP intervention into real-world settings continues to present challenges.

Behavioral interventions such as the DPP require engaged and activated patients to achieve optimal outcomes. Patient activation is patient self-efficacy for chronic care self-management and is an indicator of readiness for engagement in care.5 Patients who are younger,6, 7 more educated,6, 8 have higher income,6, 8 and better self-rated health tend to be more activated.6,7,8 Increases in patient activation and engagement have been associated with better self-care,7 disease-specific self-management,9,10,11,12 and health outcomes.13, 14

The Altarum Consumer Engagement (ACE) Measure™ is a patient engagement tool that includes questions about confidence in maintaining a role in one’s health, patterns of seeking information about health and healthcare, and asking about and participating in treatment decisions.15 The ACE was developed to identify factors that encourage adoption of patient involvement in their own healthcare and to evaluate programs designed to facilitate patient decision-making. This measure of patient engagement is unique in that it addresses dimensions of engagement that patients need to utilize information sources and tools. The ACE was administered in an online survey of 2079 adults in the general population. Based on principal components and Rasch analyses, the developers identified a 6-item Commitment scale, 5-item Informed Choice scale, 5-item Navigation scale, a 5-item Ownership scale, and a total activation scale.15 Spearman correlations of these scales with the 13-item Patient Activation Measure® ranged from 0.29 to 0.65.

The ACE was administered to a sample of adults with type II diabetes,16 but its psychometric properties in patients with prediabetes have not been previously published. We used data from the Prediabetes Informed Decisions and Education (PRIDE) study to evaluate the psychometric properties of the current version of the ACE that consists of 12 of the 21 original survey items (ACE-12, see Table 1) at baseline (https://altarum.org/sites/default/files/uploaded-related-files/ACE%20ProductSheet_091615f.pdf).

Table 1 ACE-12 items
Full size table

METHODS

Study Design

We analyzed data from the intervention arm of the PRIDE study, a cluster-randomized trial of a shared decision-making (SDM) intervention to help patients select an evidence-based diabetes prevention strategy (i.e., intense lifestyle change, metformin, or both). Patients in the intervention arm also had the option to choose no prevention strategy. The study was approved by the Institutional Review Board at the University of California, Los Angeles (IRB#15-000310).

Setting and Participants

The PRIDE study was conducted from 2015 to 2018 at UCLA Health.17 Twenty primary care clinics in the greater Los Angeles area were randomized to SDM intervention (10 clinics) or usual care (10 clinics). We used electronic medical record data to identify eligible patients. Inclusion criteria were age 18–74, overweight (body mass index [BMI] ≥ 24 kg/m2 or ≥ 22 kg/m2 if Asian) and a diagnosis of prediabetes (HbA1c 5.7–6.4% within the prior 3-months). Patients with diabetes (i.e., any HbA1c > 6.4%, ICD 250.xx billing code, oral antiglycemic medications and/or insulin), chronic kidney disease (i.e., estimated glomerular filtration rate ≤ 45 ml/min/1.73m2), active eating disorder(s), and women who had polycystic ovary syndrome or were pregnant or planning to get pregnant in the next year were excluded.

After eligible patients were identified, the primary care providers (PCPs) were notified and a formal standardized invitation letter signed by the PCP was sent to the patient via mail. A research assistant then invited patients in the intervention arm to an SDM visit with a clinical pharmacist. PCPs were also invited to refer patients with prediabetes to the study.

Intervention

The SDM intervention was delivered by clinical pharmacists embedded in each intervention clinic, who work closely with the clinic PCPs. The pharmacists used an online decision aid called “Prediabetes: Which Treatment Should I Use to Prevent Type 2 Diabetes?” to facilitate the SDM sessions. The decision aid was created by Healthwise, a national provider of health information and patient decision support tools for over 35 years. The decision aid provides information about prediabetes, compares intensive lifestyle changes and metformin as two evidence-based options for diabetes prevention, and summarizes the relative risk reduction as well as potential side effects of each option.17

Measures

We administered the ACE-12 items at baseline to participants prior to the SDM session and again at 4- and 12-month follow-up interviews. The ACE-12 uses a 5-point response scale: strongly disagree, somewhat disagree, neither agree nor disagree, somewhat agree, and strongly agree, and higher scores represent greater patient engagement. Participants also completed the Patient Health Questionnaire (PHQ-9) at baseline and 4- and 12-month follow-up intervals. Other patient characteristics measured at baseline were age, race, ethnicity, income, education, body mass index (BMI), hemoglobin A1c, and comorbidities. Patient outcomes for the PRIDE study are reported in a separate manuscript.17, 18

Analysis

We conducted an exploratory factor analysis of the ACE-12 to explore whether any items loaded on different factors than those identified in the original developmental work. We identified the number of factors using the Tucker-Lewis reliability coefficient and the scree plot of eigenvalues with squared multiple correlations as communality estimates. Good model fit was evaluated by the comparative fit index (CFI) ≥ 0.95 and root mean square error of approximation (RMSEA) ≤ 0.06.19 We then fit confirmatory factor analytic models for the original ACE-12 scales and the variants on these scales suggested by the exploratory factor analysis. To minimize problems with overfitting in the sample, we also fit the confirmatory factor analytic models on a national representative mail and web survey of over 4000 consumers: https://altarum.org/publications/right-place-right-time. Good model fit was evaluated by the comparative fit index (CFI) ≥ 0.95 and root mean square error of approximation (RMSEA) ≤ 0.06.19

Each response category for items in a survey should have the highest likelihood of being selected at some area of the distribution and the categories should be ordered monotonically with respect to the underlying concept being measured. We evaluated the ACE-12 response options using item characteristic curves (ICCs) based on the graded response model.20

We then evaluated construct validity by estimating associations of the ACE-12 scales with education level, depression symptoms, self-reported health, hemoglobin A1c, BMI at baseline, and weight loss over 12 months. We hypothesized that those who chose an intervention versus those who chose not to adopt an evidence-based prevention strategy after SDM, those who have more education (greater than a high school education) and better self-reported health (‘very good” or “excellent”) would have higher activation scores. We also hypothesized that more activated patients would be more likely to complete the DPP and lose 5–7% of their baseline weight. In contrast, based on evidence from prior literature we hypothesized that patients with more depressive symptoms (PHQ-9 score ≥ 5),21 those with higher A1c16, and those who are obese (BMI ≥ 30.0) would be less activated.15, 16, 22 We report product-moment correlations and associated effect sizes (Cohen’s d, where d = (2*r)/SQRT(1 − r2). Cohen’s conventions for magnitude of correlations were used: > 0.100 to < 0.243 = small; 0.243 to < 0.371 = medium; and ≥ 0.371 = large.23

Confirmatory factor analysis models were estimated using Mplus version 7.24 All other analyses were conducted with SAS 9.3 (SAS Institute, Inc., Cary, NC).

RESULTS

Participant Characteristics

Sample characteristics are given in Table 2. The sample (n = 515) had a mean age of 56 (range = 45–67); 56% were female; 17% were Hispanic, 83% were non-Hispanic. The majority of participants were White (46%), followed by Asian/Pacific Islander (19%), African American (15%), Native American (1%), and Other (19%). The mean A1c was 5.95 (SD = 0.2) and mean BMI was 30.31 (SD = 5.16) at baseline.

Table 2 Sample Characteristics
Full size table

Factor Analyses

The scree plot (Fig. 1) and Tucker and Lewis reliability coefficient (0.95) suggested three underlying factors. The rotated three-factor solution provided confirmation of the ACE-12 scales. However, we found that item #1 (I spend a lot of time learning about health) loaded on the navigation factor rather than on the informed choice factor.

Fig. 1
figure 1

Scree plot.

Full size image

Fit of the confirmatory factor analysis models are summarized in Table 3. Single-factor models for the original scales and the variants of the informed choice and navigation scales fit the data well according to the comparative fit index (i.e., ≥ 0.95) but RMSEA values for the commitment and informed choice scales exceeded the threshold (≤ 0.06) for good fit. The three-factor model for the revised version of the scales fit the data better than did the original scales. Ordinal alpha25 was 0.75 for the commitment scale and alphas were similar for the original and the variants, respectively, of informed choice (0.71 and 0.78), and navigation (0.54 and 0.62).

Table 3 Confirmatory Factor Analytic Models
Full size table

Item Characteristic Curves

ICCs are shown in Figure 2 and grouped by ACE-12 scales. Seven of the items had response categories performing as expected (i.e., each category had the highest likelihood of being selected at some area of the distribution and was ordered monotonically with respect to the underlying concept). There were issues with the other five items. The neither agree nor disagree option was never most likely to be chosen for Q2 (commitment, Fig. 2a) and Q6 (informed choice, Fig. 2b). The strongly disagree option was never most likely to be selected for Q1 (informed choice, Fig. 2b) and Q5 (navigation, Fig. 2c). The somewhat disagree and strongly agree options were never most likely to be chosen for Q3 (navigation, Fig. 2c).

Fig. 2
figure 2

Item characteristic curves for ACE-12 scales. a Commitment: 0 = strongly disagree; 1 = somewhat disagree; 2 = neither agree nor disagree; 3 = somewhat agree; and 4 = strongly agree. b Informed choice: 0 = strongly disagree, 1 = somewhat disagree, 2 = neither agree nor disagree, 3 = somewhat agree, and 4 = strongly agree. c Navigation 0 = strongly disagree, 1 = somewhat disagree, 2 = neither agree nor disagree, 3 = somewhat agree, and 4 = strongly agree.

Full size image

Construct Validity

Correlations of the ACE-12 scales are presented in Table 4. The pattern of correlations was similar for the original scales and the variants of the informed choice and navigation scales. Hence, we summarize only the correlations for the original scales below. Patients with a high school education or lower at baseline scored as less activated on the commitment scale (r = − 0.124, d = − 0.250; p = 0.004), choice scale (r = − 0.085, d = − 0.171, p = 0.009), and total activation score (r = − 0.042, d = − 0.084, p = 0.011) at baseline. Education level at baseline was not associated significantly with the learning about health scale. Whether an individual selected an intervention and A1c at baseline were not significantly correlated with activation scores. Depressive symptoms correlated significantly with the commitment scale (r = − 0.313, d = − 0.659, p < 0.001) and total activation score (r = − 0.172, d = − 0.349, p = 0.012). Specifically, those who scored 5 or higher on the PHQ-5 reported scored less on the commitment scale and the overall ACE-12 score. Depressive symptoms had the largest magnitude of association with the commitment scale (r = − 0.313, d = − 0.659, p ≤ 0.001). Lastly, patients who self-reported their health as less than “very good” scored lower on the commitment scale (r = − 0.308, d = − 0.647, p < 0.001), navigation scale (r = − 0.137, d = − 0.277, p ≤ 0.001), and total activation score (r = − 0.279, d = − 0.581, p ≤ 0.001). Except for the choice scale, all the original ACE scales were significantly correlated with self-reported health. Those who were obese (BMI ≥ 30.0) scored less on the commitment scale (r = − 0.177, d = − 0.360, p = 0.001), and total activation scale (r = − 0.136, d = − 0.275, p = 0.016). Weight change over 12 months was not associated with greater activation scores at baseline.

Table 4 Correlations of ACE-12 Scales with Other Variables
Full size table

DISCUSSION

The results of this study provide some support for the original ACE-12 scales derived by the Duke et al.15 This is important given the need to better measure and understand patient activation among patients with chronic conditions for which behavior change and/or chronic medication use are recommended. Because of the challenges in recruiting and retaining patients into behavior change programs26 and supporting patients in adherence to metformin for diabetes prevention, measuring activation at baseline can be useful in identifying patients who may benefit from extra support.

In terms of hypothesized correlations between patient activation and other variables, ACE-12 scores in our study were negatively associated with education, depressive symptoms, and obesity at baseline. Similarly, our analysis showed that higher ACE scores correlated with better self-reported health at baseline. Less patient activation was observed for those with more depressive symptoms. These findings are generally consistent with a large body of literature showing that patients with diabetes and depression often have difficulty engaging in lifestyle modification,27 adherence to medications,28, 29 and report worse quality of life30 and health outcomes.31 Likewise, consistent with existing literature,15, 22 obesity was associated with less patient activation in our study.

Interestingly, patients with less education reported higher activation at baseline. This finding is contrary to prior research showing less activation among those who are less educated.32 One possible explanation is that study participants with less education may have had less baseline understanding of prediabetes and were highly motivated to enroll in the study and learn about their condition and prevention strategies to avoid diabetes. Similarly, patients with more education who were highly activated may have already raised this issue with their PCP and/or sought information from other sources and may have been less interested in participating in SDM for diabetes prevention. Lastly, we found that better self-reported health at baseline had the largest association with patient activation.

Our study has several limitations. First, the sample was comprised of overweight/obese patients with prediabetes from one large medical system, limiting the generalizability of the findings. In addition, construct validity analyses relied primarily on data gathered at baseline. Future studies may want to analyze construct validity by exploring longitudinal relationships among the variables. Hence, the direction of effects cannot be known from the associations reported.

Given the relationship between patient activation and engagement and improved health outcomes, our study provides support for the ACE-12, a brief patient activation tool that can be easily used and is accessible for research, quality improvement, or clinical use without licensing fees. Future studies evaluating this tool among patients with multiple chronic conditions will be needed to determine whether Q1 (I spend a lot of time learning about health) should remain in the informed choice or be included in the navigation scale. In addition, the reliability of navigation scale was suboptimal (0.54) so it may be important to include additional items to achieve adequate reliability for this construct.