Valuation of the EQ-5D-3L in Russia

The standardized valuation protocol developed by EuroQol for EQ-5D-5L valuation studies [20, 21] was used with EQ-Portable Valuation Technology (EQ-PVT) to assist the interviews. Since the EQ-PVT was developed specifically for EQ-5D-5L valuation studies, its design was adapted to the EQ-5D-3L valuation study by the EuroQoL: 27 health states and the worst health state (“33333”) were selected to be directly valued using composite TTO (cTTO) tasks, and 60 pairs of health states were selected to be directly valued using discrete choice experiment (DCE) tasks. For the cTTO task, 18 states were selected from the orthogonal design by Yang et al. [22]. These were supplemented with 5 “mild” health states with only one deviation from full health (e.g. “12111”), the state “33333”, with severe problems on all dimensions, and 4 other intermediate states, while assuming level balance. The states for cTTO tasks were divided into three blocks of ten states each, and each block contained at least one mild state and the state 33333. The states for DCE tasks were selected from Stolk et al., where they were generated using a Bayesian efficient design approach, and divided into six blocks each of ten pairs of states, labelled “A” and “B” [23]. When conducting an interview, each respondent completed ten cTTO tasks and ten DCE tasks. The standardized valuation protocol developed by EuroQol with EQ-PVT has been pre-tested in Russia in a pilot study. The convenient sample of young Russians (n = 81) was used for the pilot study. The aim of the pilot study was to train the interviewers prior to the main study. The results from the pilot study and the main valuation study followed the same pattern, e.g. mobility dimension received the highest weight.

A target sample size of 300 participants was recruited from the general Russian population. Using three blocks of ten states in the cTTO task, each health state is valued by 100 respondents, except for state 33333, which is valued by all respondents. For the DCE task, 50 responses were collected per choice pair, which meets the standards for the rule of thumb by Orme [24]. The sample size in our study is similar to the power calculation results used for EQ-5D-5L valuation studies based on the updated methodology recommended by the EuroQoL Group [25]. Respondents were recruited using quota-based sampling with quotas for age and gender to represent the general Russian population from six regions (Moscow, Moscow region, the Republic of Tatarstan, Volgograd region, Murmansk region, Smolensk region). Recruitment was conducted by the Russian Public Opinion Research Center VCIOM. During the recruitment process, the participants aged 18+ years were asked whether they are willing to discuss different health problems, and before the interview, verbal informed consent was obtained from all respondents. All the participants received an incentive of 1,000 Russian rubles.

The data were collected between August and November 2019 using face-to-face computer-assisted interviews appointed at selected sites by TB, VF, NM, SR. All interviews had the same structure. First, participants were given background information, informed about anonymity and confidentiality and their ability to stop the survey at any time. Second, participants assessed their health by the EQ-5D-3L and EQ-VAS. Third, respondents were introduced to the cTTO task, completed three practice, and then ten real cTTO tasks. Participants were instructed to read descriptions of states out loud to be actively involved in the study. Participants who did not wish to trade-off in the cTTO task were considered as non-traders. Next, participants completed ten DCE tasks. Finally, sociodemographic characteristics were collected.

Before the actual data collection, the interviewers received training and performed up to 25 test interviews each as a part of the study preparation process. Interviewers collected data in rounds of 10–20 interviews per interviewer per week. After each round of data collection, the EuroQoL Research Foundation (FDP and BR) reviewed data quality and provided feedback. An interview was considered to be of poor quality if one of the following criteria were met: the time spent on explaining the cTTO task in the wheelchair example is less than 3 min; the explanation of the “worse than death” task in the wheelchair example is omitted; there are inconsistencies in the cTTO ratings (if the value for state “33333” is not the lowest and is at least 0.5 higher than that of the state with the lowest value); the time spent on ten cTTO tasks is less than 5 min. If any of these criteria are met, the interview was considered to be of suspicious quality [26]. The quality control process implies that if four or more out of ten completed interviews are considered to be of suspicious quality, these ten completed interviews would be dropped. No interviews were excluded due to on-going low-quality interviewer performance.

Mean, standard deviation, median, interquartile range (IQR), percentages were used to describe the sample and data characteristics. The models were estimated using TTO-only data, DCE-only data, DCE and TTO data in combination (i.e. hybrid models) [27, 28].

cTTO panel data were modelled with random-effects GLS regression, random-effects Tobit regression (accounting for left-censored at −1 data), and interval regression (accounting for heteroskedasticity) based on the observed values for the 28 states in cTTO tasks. A conditional logit model was estimated on the basis of comparison of the 60 pairs of states in DCE tasks. Since the conditional logit model generates coefficients on a latent arbitrary utility scale, the coefficients were rescaled by rescaling parameter to represent the health-utility scale. This rescaling parameter θ, was derived from the hybrid models, where the DCE and cTTO data are modelled together directly, and allows us to present DCE values on a health-utility scale.¹

The agreement between the cTTO and DCE data was investigated by inspecting the predicted values for the 243 EQ-5D-3L health states for all estimated models. Furthermore, the relative agreement of the dimensions and relative ratios between the estimated coefficients were also considered. Different hybrid models (standard hybrid model, hybrid model with censoring at −1, hybrid model corrected for heteroscedasticity, hybrid model corrected for heteroskedasticity and censoring at −1) were used to generate values for all 243 health states defined by the EQ-5D-3L. All data were analyzed as disutilities (1 – utility score), so coefficients represent the utility decrement of moving from base level to level two and from base level to level three.

The most appropriate model was selected based on the criteria: the significance of the coefficients; logical consistency; goodness of fit and predictive performance. The model was considered to be logically consistent in case worse health states had estimated values lower than better health states. Goodness of fit was assessed using the Akaike (AIC) and the Bayesian information criteria (BIC). Predictive performance was analyzed by comparing predicted and observed values of cTTO using Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). To promote the estimation of HRQoL based on the EQ-5D-5L questionnaire, along with this the EQ-5D-3L study, as an interim solution, the mapped EQ-5D-5L value set was produced [29]. The van Hout et al. algorithm uses patient responses to the EQ-5D-3L and EQ-5D-5L questionnaires to calculate conditional probabilities of reporting certain problems in the EQ-5D-5L, given the patients’ EQ-5D-3L response [29]. These conditional probabilities can then be used to determine values for EQ-5D-5L health states, based on the EQ-5D-3L predicted in this study. All analyses were performed using STATA and R statistical packages.

A sensitivity analysis was performed on the cTTO data to assess the impact of the inclusion of non-traders. GLS random intercept models were estimated for the final dataset without the non-traders and the final dataset supplemented with the data from the non-traders. The mean predicted utility for state “33333” were then compared to test whether there was a meaningful difference between the two groups.

In total, 313 respondents were recruited to participate in the study, and 300 (95.9%) participants successfully completed the interview. The response rate in this study was roughly 90%. Thirteen respondents were considered as non-traders to be excluded from the primary analysis. Among 300 interviews, 10 (3.3%) interviews did not meet the quality criteria as described in the methods section, but no interviews were excluded, as the interviewer`s performed well in general. Mean interview time in cTTO part of the interview was 23.7 ± 7.7 min with 9.7 ± 3.9 min to complete ten cTTO tasks. To reach the point of indifference in 10 cTTO tasks, it took an average of 6.4 ± 1.5 interactions per task.

Sociodemographic characteristics of the included respondents are presented in Table 1. The sample was representative of the Russian population in terms of age and gender but not for the residential area according to the Russian Federal State Statistics Service. 71.7% of respondents had higher education, 68.9% were employed, and the median per-person income was 35,000 Russian rubles with IQR 22,000–50,000 Russian rubles. 120 (40.0%) respondents reported no health problems of any dimension, and 56 (18.7%) reported moderate health problems in one dimension of the EQ‐5D‐3L. Median self-rated health using EQ‐VAS was 80 with IQR 70–90.

The final cTTO data set includes 3,000 cTTO responses. The distribution of all 3,000 observed cTTO values, included in the primary analysis, are presented in Fig. 1. Among 3,000 cTTO values, 600 (20.0%) and 2,360 (78.7%) cTTO values were considered “worse than death” and “better than death”, respectively. The number of values clustered at −1, 0, and 1 was 211 (7.0%), 41 (1.3%), and 426 (14.2%), respectively.

The mean cTTO values ranged from -0.424 for the health state “33333” to 0.944 for the health state “11211”. To test the face validity of the observed cTTO data, median and IQR of cTTO values were plotted against a misery index (Fig. 2). The misery index as a proxy for severity was calculated by summing all the integers of five dimensions ranging from 5 for the full health state (“11111”) to 15 for the state “33333”. The observed variation of cTTO values between more severe and less severe states, as indicated by misery indexes, shows the presence of heteroskedasticity.

The final DCE data set includes 3,000 responses. Figure 3 shows the proportion of “A” responses, plotted against the differences in the misery indexes between a health profile “A” and a health profile “B”. The proportion of those choosing a better health state was strongly correlated to the difference in the misery indexes between compared states.

All models based on cTTO-only data produced some insignificant coefficients. The results of the models were logically consistent. The state “33333” value was −0.46 for the GLS regression, −0.518 for the Tobit regression, and −0.56 for the interval regression. The AIC and BIC were lowest for the random intercept GLS regression model (Online Resource 1). The lowest predicted error as indicated by MAE and RMSE was for the interval regression. The conditional logit model for DCE-only data resulted in one insignificant coefficient with the results also being logically consistent (Online Resource 1).

Comparing cTTO and DCE predicted values for 243 health states resulted in a similar pattern, as being shown in a kernel density function (Fig. 4).

cTTO and DCE models sets of coefficients were in relative agreement; that is, the most important dimension was mobility, and the least important was pain/discomfort (Online Resource 1). Furthermore, the relation between the coefficients of the DCE and the cTTO models seems to be linear (Fig. 5), which supports the use of hybrid models.

Several types of hybrid models produced statistically significant coefficients, but slightly different results, with the worst state “33333” value of −0.455 for the standard hybrid model, −0.504 for the hybrid model with censoring at −1, −0.503 for the hybrid model with heteroskedasticity correction, and −0.574 for the hybrid model with censoring and heteroskedasticity correction. All hybrid models were logically consistent (Table 2). For all hybrid models, mobility dimension had the most significant impact on the utility decrement and the lowest decrement was due to anxiety/depression. The final value set has been based on Model 3c as this model had the best model fit (the lowest AIC, BIC, MAE, RMSE). To obtain utility for an EQ-5D-3L health state, for instance “12233”, the following calculation based on the hybrid model 3c is needed: Utility weight (“12233”) = 1–0 (no problems in Mobility) – 0. 075 (some problems in Self-Care) – 0.073 (some problems in Usual Activities) – 0.377 (severe problems in Pain/Discomfort) – 0.179 (severe problems in Anxiety/Depression) = 0.296.

The crosswalk value set for EQ-5D-5L is presented in the Online Resource 2.

The sensitivity analysis suggested that the inclusion of non-traders had no significant effect on the coefficients. For the GLS regression model, the value for state “33333” increased by 0.06 utilities. Since there were 13 non-traders out of 313 respondents, this does not seem to be a meaningful difference.