US population norms for the EQ-5D-5L and comparison of norms from face-to-face and online samples

The EQ-5D is the most widely used generic multi-attribute utility instrument in the world, and it has numerous applications in health care [1]. It is applied in a variety of research and clinical practice settings, such as clinical trials, cost-utility analysis (CUA), patient surveillance, and population health measurement [2‐8]. The EQ-5D was developed as a brief, generic measure of health which includes a health state classifier that has five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. The first version of the EQ-5D had 3 levels of health problems (EQ-5D-3L), and more recently in 2011, a more descriptively rich 5-level version (EQ-5D-5L) was introduced which describes 3125 health states (D^L = 5⁵ = 3125) [9, 10]. An index-based utility score can be generated from self-assessments using the descriptive system, by applying a value set based on societal preferences for EQ-5D health states. The value set is based on preference choice tasks elicited from the general population. In addition, the health state classifier is also accompanied by the visual analogue scale (EQ VAS), anchored at 0 for “the worst health you can imagine” and 100 for “the best health you can imagine” [9]. Thus, the EQ-5D-5L can provide two summary scores of health that can inform decision-making: the patient’s self-rating of health on the EQ VAS, and an index-based utility score. The latter is used to facilitate the generation of a common metric of health in the form of quality-adjusted life-years (QALYs) in CUA that can inform resource allocation across health care sectors.

Among the numerous applications of the EQ-5D-5L, a set of utility index and EQ VAS benchmark values for the general population, i.e., population reference data or population norms, are useful for comparing burden of disease and as normative reference values [11]. Patient or sample values can be compared to these benchmark values to determine how individuals or groups measure against the general population in decision models, evaluation of clinical programs, assessment of public health in large-scale applications, etc. [6, 11‐23]. Although the use of the EQ-5D-5L continues to expand, the US does not yet have population norms for comparison. In 2019, an EQ-5D-5L value set for the US was developed [24]. It was developed utilizing the standardized international protocol recommended by the EuroQol Group and used preferences from an adult sample representative of the US general population. Apart from the valuation of EQ-5D-5L health states, these respondents also self-reported their own health using the EQ-5D-5L and the EQ VAS, allowing for estimation of US EQ-5D-5L population norms [24].

In the past, most studies reporting population norms have been based on data collected in-person as part of population health studies or using mail surveys [11, 12]. The standardized international protocol for EQ-5D-5L valuation studies was developed for a face-to-face, interviewer-assisted setting to ensure the respondent task comprehension and preserve higher data quality for the challenging preference choice tasks [25, 26]. The US EQ-5D-5L study used quota-based sampling based on age, gender, race, and ethnicity to ensure representativeness of the sample for the population and examined non-quota-based respondents characteristics like education to support comparability to the general population. A substudy was also conducted that sought to replicate the face-to-face protocol using online panels, applying the same quota-based sampling criteria [27, 28]. The online and face-to-face samples were dissimilar in terms of health, and the online sample was also less comparable to the general population more than the face-to-face sample in terms of non-quota-based characteristics. Thus, the face-to-face valuation study was selected as the primary source of population norms for the present study. However, as online data collection continues to gain relevance, there is value in understanding the differences between the two sources of respondents.

The primary aim of this study was to estimate EQ-5D-5L EQ VAS and index-based norms for the US general population using the data from the US valuation face-to-face study. A secondary aim was to compare face-to-face and online samples based on non-quota-based characteristics to understand the issues of generalizability with respect to mode of data collection.

In this study, we reported age and gender-based norms for the EQ-5D-5L in the US general population. The reference norms were based on data collected from implementation of the EuroQol international standardized protocol using a study design that used quota-based sampling and involved face-to-face, interviewer-assisted administration of the survey and valuation tasks. Data based on the same quotas for sampling collected via online panels provided systematically different respondents and norms, so these were reported separately.

Index-based utility and EQ VAS mean scores did not consistently decrease with increasing age. Instead, the norms were characterized by plateaus or minor increases in older age bands. This pattern may be explained by the lower reported prevalence of anxiety/depression with increasing age as well as different psychometric properties of the EQ-5D-5L (e.g., differential item functioning) and varying priorities for the dimensions by age [35‐41]. In the face-to-face sample, approximately 24% of respondents 65 and older reported any problems with anxiety/depression whereas 42.1% of respondents 18 to 24 years old reported the same.(Supplementary material Appendix A) Although issues with mental health may be more often associated with social stigma in older adults and cause fewer older adults to indicate issues with mental health [42], differing levels of social desirability bias due to interviewer presence by age band was unlikely the only contributor to this unexpected distribution by age; the same differential between age bands was observed in the online sample: 24–30% of online respondents 65 and older and 57.1% of respondents 18–24 reported any problems with anxiety/depression (Supplementary material Appendix F).

Similar to the norms reported here, other research corroborates higher prevalence of mental health (e.g., anxiety and depression) issues in US young adults [35‐38]. In 2017, the National Institute of Mental Health (NIMH) and the Substance Abuse and Mental Health Services Administration (SAMHSA) found that 4.7% of US adults 50 years and older had a major depressive episode in the past year compared to 13.1% in respondents 18 to 25 years old [35]. Older data from 2001 to 2003 showed that 9.0% of respondents 60 years and older had anxiety disorder in the past year whereas 22.3% of respondents 18 to 29 years old reported the same [36]. Further, mental health reasons for leaving jobs were highest in youngest members of the workforce, and anxiety has also been increasing among young adults since 2008 [37, 38]. In developed, Western countries such as Germany and the UK, some younger segments of the population were also more likely to report mental health issues compared to older segments [43, 44].

Response shift may also have occurred in the older respondents, potentially leading them to interpret or conceptualize the meaning of “no problems” differently from younger respondents who may impose a more ideal expectation when self-reporting their health [39‐41, 45, 46]. These interpretation differences could contribute to distinctive psychometric properties of the EQ-5D-5L in different age segments of the population. A combination of increased mental health problems in younger adults and differing interpretation of the problem severity labels could have contributed to the decreased prevalence of anxiety/depression in older respondents and the observed patterns of norms across age groups.

The patterns seen in the US EQ-5D-5L norms are also present in other international norms. Similar to this study, a greater proportion of younger respondents in Chinese urban and Indonesia general populations reported problems with anxiety/depression on the EQ-5D-5L [17, 19]. In China, 34.5% of men 20 to 29 reported any problems with anxiety/depression whereas only 11.5% of men over 70 did the same; the trend was similar in Chinese women [19]. Approximately 40.1% of Indonesian respondents aged 17 to 30 indicated they had any issues with anxiety/depression compared to 32% of respondents older than 50 [17]. Similar to the present study, Canadian, specifically Alberta and Quebec, and Indonesian EQ-5D-5L norms also plateaued in adjacent, older age groups [17, 18, 47].

The systematically lower mean utility and VAS values noted in online respondents relative to face-to-face respondents were likely affected by a combination of factors. Social desirability bias may have contributed to the observed disparities between samples. A greater proportion of online respondents reported issues across all EQ-5D-5L dimensions compared to face-to-face respondents (Supplementary material Appendices D and E). Further, a smaller portion of online respondents indicated that their general health was excellent compared to the face-to-face respondents (12.1% versus 20%; Table 1). If the populations had similar health, face-to-face respondents may have been unwilling to admit health issues in front of an interviewer due to social desirability bias. Past research also found that self-reported health and norms differed when surveys were self-administered versus interviewer-administered via telephone [48].

However, respondents could also have truly differed between modes of data collection due to the varying selection pressures of recruitment and survey needs for each mode of data collection [49]. US online panels tend to be disproportionately White and unrepresentative of minorities [50, 51]. The company which administered the online surveys used in this study noted comparable patterns of minority under-representation in the online panels they employ, which contributed to the low prevalence of non-Black minorities recruited into the online sample [52]. In addition, online respondents must have reliable access to internet and a computer with which to access it and belong to a survey panel to be selected for the study. Face-to-face respondents needed to participate in the study at centralized locations, potentially requiring transportation to attend interviews outside of their homes. Respondent characteristics which determine agreement to survey participation may additionally differ by mode. Distinctions between samples were most noticeable when respondent characteristics unlikely to be susceptible to social desirability bias were examined. For example, online respondents were more likely to have children under 18 than face-to-face respondents (Table 1). Finally, face-to-face respondents aged 65 and older were more comparable to the US general population than these respondents in the online sample, particularly in terms of age and gender distribution. (Supplementary material Appendix G).

As the online population appeared to be less representative of the US general population, the two samples were kept distinct. Based on the aforementioned differences between online and face-to-face respondents as well as the improved representativeness in older respondents, the face-to-face norms should be the primary set of normative values for the US population, particularly for data collected in-person. The online norms may be most useful for comparison of unsupervised data elicited from online panels.

The study had several limitations. Quota-sampling was employed for practical time and cost considerations, and random sampling may have allowed for a more representative sample of the US population, particularly in the face-to-face sample. This shortcoming can be observed in the education attainment and general health of respondents who were at least 65 years old (Supplementary material Appendix G). The sample sizes included in the present study were relatively small in comparison to the US population size, and each face-to-face respondent represented approximately 290,000 US inhabitants. As the EQ-5D-5L is not included in any large-scale US general population surveys, these data were the best available sources to estimate US general population norms. Finally, the mean index and VAS values estimated for each diagnosis in both online and face-to-face samples should only be used as a general guidance for disease burden as these subgroups are likely heterogenous in disease stage, acuity, and patient experience.

With the present study, US EQ-5D-5L utility and EQ VAS norms are now available as general population benchmarks to support health services research across research, clinical, and policy settings for two major methods of data collection. End-users of the EQ-5D-5L may choose the set most appropriate for the application.