Comparative evaluation of the health utilities index mark 3 and the short form 6D: evidence from an individual participant data meta-analysis of very preterm and very low birthweight adults

The following criteria have been utilized to identify relevant prospective cohorts: (1) have used two distinctive preference-based measures to assess HRQoL in adulthood (defined as ≥ 18 years [45]) amongst individuals born VP/VLBW, (2) included a comparison control group of term-born and/or normal birthweight individuals, and (3) contributed data to the RECAP consortium (www.​recappreterm.​eu), a database of cohorts of individuals born VP/VLBW. Two different and recent systematic reviews of preference-based HRQoL outcomes following preterm birth or low birthweight had identified eligible cohorts [46, 47]. The following two prospective cohort studies met the study inclusion criteria: The Bavarian Longitudinal Study (BLS) [48] and The Victorian Infant Collaborative Study (VICS) [49]. These two studies were designed to assess the associations of VP/VLBW status with various health outcomes [50] as well as received country-specific ethical approvals, including participants’ written informed consent in adulthood.

Table 1 described the background eligibility criteria, age(s) at assessment, and the control groups for the BLS and VICS cohorts. Detailed descriptions of each participating cohort (the study’s population, methodology, types of data and variables) have been previously published [48, 49]. All variables of interest across BLS and VICS were harmonized, meaning that an identical set of definitions, scaling methods, and classification were applied to all variables across BLS and VICS cohorts.

Participants’ perceptions of their HRQoL were assessed using both the HUI3 and SF-12 [48, 49]. Study participants completed the unedited Health Utilities Index 15-item questionnaire for usual health status assessment, which was obtained from the Health Utilities Index developers and covers the HUI3 health status classification system. The HUI3 was developed to describe HRQoL in general population and clinical contexts and consists of eight attributes: ambulation, dexterity, cognition, vision, hearing, speech, emotion, and pain [51‐53]. Within each attribute, the levels of function were scored on a 5- or 6-point scale ranging from optimal function to severe impairment. Responses within each of the eight attributes can be valued as single attribute utility (SAU) scores on a scale ranging from 0 and 1 [51]. Responses within each of the eight attributes can also be mapped onto an eight-attribute health status vector. Algorithms reflecting the preferences of the general public for the HUI3 health states can be used to convert responses to the measure’s eight attributes into multiplicative multi-attribute utility scores. The Canadian algorithms [51‐54] were applied in both cohorts, reflecting the preferences of 504 adults in the general population who were living in the city of Hamilton, Ontario, and who had previously been asked to value selected HUI3 health states using both visual analogue scaling and standard gamble techniques. HUI3 multi-attribute utility scores are valued on a cardinal scale ranging between -0.36 and 1.0, with -0.36 representing the worst possible HUI3 health state, 0.0 representing dead, and 1.0 representing full health [53, 54].

The SF-12 includes 12 of the 36 items contained within the SF-36. These have an identical dimension structure [55], and for each dimension, item responses are mapped onto a 0 to 100 scale. Responses to the SF-12 items were converted [56] into SF-6D multi-attribute utility scores using the UK SF-6D utility algorithms [55]. The SF-6D algorithms reduce the eight dimensions of the SF-36/12 to six by merging role limitations due to emotional and physical problems and eliminating general health perceptions. SF-6D multi-attribute utility scores are valued on a cardinal scale ranging between 0 and 1.0, with 0 representing dead and 1.0 representing full health [55]. For the SF-6D, only two out of six dimensions (physical functioning, role limitations) reflect physical aspects of health, while other dimensions (social functioning, pain, mental health, vitality, and emotional) relates to non-physical aspects of health. By contrast, most HUI3 attributes reflect the physical health of the individual (vision, hearing, speech, ambulation, and dexterity).

We used the following outcome variables of interest: HUI3 and SF-6D multi-attribute utility scores and the difference between HUI3 and SF-6D utility scores. The minimum clinically important difference in multi-attribute utility score is considered to be 0.03 for the HUI3 [57] and 0.04 for SF-6D [58, 59].

We combined IPD across the BLS and VICS cohorts. To identify whether our assessments of agreement between the HUI3 and SF-6D measures should be disentangled by birth status, we initially estimated the association between VP/VLBW status and HRQoL in adulthood using one-stage IPD analysis, which could be implemented either using fixed or random effects [60]. Fixed effects models were used because individuals born VP/VLBW and controls were enrolled across distinct geographical regions and time frames. This implies the presence of systematic differences across the BLS and VICS cohorts. However, we also utilized random effects as a robustness check. Models were adjusted for age and sex of the participants, mode of delivery (cesarean section vs vaginal delivery), and number of days in hospital after birth, as well as for the harmonized socio-demographic/socio-economic variables: maternal education level at birth or during childhood and maternal ethnicity.

We computed means, standard deviations, and t tests for unequal variances, medians, and Kruskal–Wallis tests to assess differences in agreement between HUI3 and SF-6D multi-attribute utility scores within VP/VLBW individuals, controls, and the combined sample. To identify statistically significant predictors that explain observed differences between HUI3 and SF-6D multi-attribute scores on covariates, we used generalized mixed models in a one-step approach. Models were estimated using multivariate linear fixed effects.

Furthermore, agreement between the HUI3 and SF-6D multi-attribute utility scores was investigated using the intra-class correlation measures and Bland–Altman plots. The analysis was performed for VP/VLBW individuals and controls separately as well as for the combined sample. An intra-class correlation coefficient less than 0.75 is indicative of moderate agreement, while an intra-class correlation coefficient greater than 0.75 indicates good agreement [60, 61]. Bland–Altman plots display the mean \(\left( {\frac{HUI3 + SF - 6D}{2}} \right)\) overall scores and the difference (HUI3-SF-6D) against each other. A line of mean difference estimates systematic difference between the two instruments, with limits of agreement estimated as the mean difference plus/minus 1.96 standard deviation of the mean difference. Limits of agreement (LoA) reflect the expected range in which 95% of observed differences would lie, with wider limits of agreement indicating poorer agreement [62]. Good concordance between the HUI3 and SF-6D would show a mean difference close to zero with ≤ 5% of scatter points lying outside the limits of agreement.

Analyses were performed using STATA version 17 and p-values of 0.05 or less were considered statistically significant.

Table 1 displays baseline characteristics of the participants of the BLS and VICS cohorts. Years of birth ranged from 1985 to 1986 for BLS and 1991–1992 for VICS. Pooled data consisted of 778 HUI3 assessments (417 VP/VLBW individuals and 361 controls) and 780 SF-6D assessments (411 VP/VLBW and 369 controls). The mean age at assessment was 18 years for VICS participants and 26.3 years for BLS participants. Table 2 shows the characteristics of VP/VLBW individuals and controls with non-missing HUI3 and SF-6D multi-attribute utility scores. Within the meta-cohort no statistically significant differences were found by birth status across the following characteristics: age, sex of the participants, maternal education level at birth or during childhood, and maternal ethnicity.

Using a one-stage IPD meta-analysis, to identify whether our assessments of agreement between the HUI3 and SF-6D measures should be disentangled by VP/VLBW status, we initially estimated the association between VP/VLBW status and HRQoL in adulthood. The adjusted impact of VP/VLBW status on the HUI3 multi-attribute utility score was -0.04 (95% CI − 0.06, − 0.01) with no significant impact on the SF-6D multi-attribute utility score (Table 3). To understand the sources of identified differences we present the additional evidence in Online Appendix A (Tables A.1, A.2). We utilized random effects models and reported results in Online Appendix B. Further evidence on the association between VP/VLBW status and HRQoL in adulthood using HUI3 and SF-6D can be found in a recent study [63].

Table 4 displays descriptive and inferential statistics for HUI3 and SF-6D multi-attribute utility scores for each group considered. Mean and median estimates for HUI3 multi-attribute utility scores were consistently higher compared with their respective SF-6D values. All differences were clinically [57‐59] and statistically significant within the meta-cohort across all groups considered (p < 0.01). Table 5 shows the estimates from regressing differences between HUI3 and SF-6D multi-attribute scores on covariates. The evidence suggests that none of the variables considered was a statistically significant predictor of observed differences between multi-attribute scores.

The correlation coefficient (ρ) between HUI3 and SF-6D multi-attribute utility scores for the BLS and VICS cohorts was computed. The evidence showed that ρ between the two multi-attribute utility scores within VICS was 0.45 (ρ = 0.51 for VP/VLBW individuals and ρ = 0.31 for controls), which was higher compared with ρ = 0.35 within the BLS cohort (ρ = 0.37 for VP/VLBW individuals and ρ = 0.33 for controls). Within the meta-cohort, the ICC was 0.40 for the VP/VLBW sample, 0.29 for the controls, and 0.36 for the combined sample. Overall, the evidence suggests that the HUI3 and SF-6D multi-attribute scores had moderate or low correlation.

The Bland–Altman plots were constructed by birth status (see Fig. 1) and showed a mean difference of 0.06 (95% CI 0.04, 0.07), i.e., HUI3 multi-attribute utility scores for controls were higher than the SF-6D multi-attribute utility scores for controls. The mean difference for VP/VLBW individuals was 0.03 (95% CI 0.01, 0.05), meaning that HUI3 multi-attribute utility scores were higher than SF-6D multi-attribute utility scores in this group. In the Bland–Altman plot (Fig. 1), the data points deviate widely from the agreement line at low levels of mean utility and the relationship between the difference in HUI3 and SF-6D utilities shifts in magnitude but not in direction. The same pattern is observed by combining the VP/VLBW sample with controls (Fig. 2), generating a mean difference between the paired observations of 0.04 (95% CI 0.03, 0.06). Notably, in all groups considered, the Bland–Altman plots showed a funneling effect with stronger agreement as the mean overall utility score approached 1.0. However, in the Bland–Altman plots, the 95% LoA ranged from − 0.30 to 0.37 within the VP/VLBW sample, − 0.22 to 0.34 within controls, and − 0.27 to 0.36 within the combined sample. Most importantly, in all three groups considered (VP/VLBW, controls, and the combined sample), the 95% agreement differences were far wider than the clinically meaningful differences postulated for the HUI3 and SF-6D.

The data structure made it possible to examine the agreement between measures within VP/VLBW individuals, normal birthweight or term born controls, and within the combined sample. We were able to assess the validity of the results by replicating the main finding across different populations, which strengthens the study’s conclusions and which had not been studied previously as far as we are aware. This is the major strength of this study. Furthermore, our study ascertained agreement between the outputs of the HUI3 and SF-6D measures using controls selected from the general populations in Germany and Australia. This implies that results of this study may be generalizable to populations from Germany and Australia.

Another strength of this study is that we were able to confirm VP/VLBW status in each participant due to the rigorous recruitment, data collection, and follow-up methods utilized by the participating cohorts, which also harmonized relevant socio-demographic factors. Furthermore, our study employed socioeconomically diverse samples of VP/VLBW individuals and controls. Finally, results of this study are not affected by biases associated with proxy parental reporting [70] because participating cohorts used self-reported HRQoL data.

It is important to note that the scoring algorithms for the HUI3 and SF-6D differ in certain respects. Thus, while our study shows that the utility differences we found are driven by the underlying concepts of health being measured, the methods employed are not able to measure the contributory effects of valuation protocols, i.e., differences in scoring algorithms. A further limitation is that our study included cohorts from only two countries. Thus, replication of this study with data from other countries, particularly low- or middle-income countries, would be a valuable contribution to the literature.

Our report did not include the EQ-5D in this comparative evaluation because no individual study that contributed to the RECAP platform assessed HRQoL using the EQ-5D. This is a limitation because a recent review identified that the EQ-5D is the most frequently recommended multi-attribute utility instrument in HTA guidelines [17]. Thus, our study is not able to provide comprehensive evidence regarding the most appropriate preference-based HRQoL measure to ascertain utility scores in adulthood for VP/VLBW individuals or for normal birthweight or term born controls. Comparing agreement of the EQ-5D, HUI3, and SF-6D for VP/VLBW individuals and normal birthweight or term born controls offers a fruitful direction for further investigation.