Assessing measurement invariance in the EORTC QLQ-C30

An important measurement property of a questionnaire is measurement invariance (MI), which states that the relationship between the items of a questionnaire and the latent construct that is measured is stable and independent of group membership or the measurement occasion [1]. If the assumption of MI is violated, observed differences between groups are not true differences in the construct of interest but may reflect systematic error. In other words, when using a questionnaire for group comparison, members of different groups must assign the same meaning to the items and scale that indicate the construct. Only when a questionnaire is MI, a valid comparison can be made between groups [2].

One of the most widely used quality of life (QoL) questionnaires in cancer research is the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30) [3]. Only a few studies have examined the MI of the EORTC QLQ-C30 concerning change over time, and clinical and patient characteristics. A small qualitative study in a heterogeneous cancer sample showed different cognitive processes underlying QoL appraisal before and after radiotherapy in patients with cancer [4]. In prostate cancer patients, the physical functioning and role functioning subscales of the QLQ-C30 gained importance over time for representing the QoL construct. The same study showed that a change in internal standards (a form of response shift) made patients perceive their emotional and cognitive functioning more positive at follow-up [5]. In a larger study (n > 30000) with a multicultural heterogeneous cancer sample, however, researchers found little measurement bias in the QLQ-C30 across time points of assessment (baseline, on-treatment, and off-treatment), regardless of treatment status [6].

Concerning other clinical and patient characteristics, one study showed measurement bias for age and previous treatment in the QLQ-C30, but not for sex and treatment preference [7]. However, the sample size of this study was rather small, considering the statistical approach used. In contrast, the QLQ-C30 was found to be MI for age, sex, and type of surgery (i.e., robot- or not robot-assisted) in lung cancer patients [8], and for primary cancer sites in a large heterogeneous cancer sample (n = 1906) [9]. Two other studies found that the QLQ-C30 was mostly MI across ethnic groups [10], languages [11], countries [12].

The QLQ-C30 is designed to measure QoL in heterogeneous cancer populations, which could lead to measurement non-invariance and biased group comparisons. Because the literature on MI of this questionnaire is limited, we investigate the MI of the QLQ-C30 in a large Dutch patient sample with different primary cancer sites. Additionally, potential measurement bias concerning age, sex, time since diagnosis, and life stage (i.e., cancer survivors versus patients in their last year of life) is evaluated.

In total, 7007 patients with cancer were included in the analysis, of which 266 patients were in their last year of life (Table 1). Patients in the total sample were often male (58%) and were on average 66 years old (SD = 12). The majority of the patients (69%) were diagnosed with a primary tumor less than five years ago and 68% had physical comorbidities. Patients in their last year of life were on average older (71, SD = 9) and more often had physical comorbidities (75%) compared to patients in the total sample. Ovarian cancer and multiple myeloma was relatively more prevalent in patients in their last year of life (11 and 12% respectively) compared to the total sample (5 and 4% respectively).

As MI is necessary for valid evaluation of inter-individual differences in QoL, we tested several levels of MI of the QLQ-C30 Global QoL, functional, and multi-item symptom scales for different grouping variables using state-of-the-art multiple-group Structural Equation Modeling techniques that explicitly takes into account the ordinal measurement level of the QoL indicators. We found that the model structure fits the data well across groups, and we found empirical evidence for valid between-group comparison of QLQ-C30 latent means in subpopulations based on time since diagnosis and life stage as operationalized in our study. However, when imposing equality constraints on thresholds, loadings, and intercepts, across groups based on age, sex, and primary cancer sites results showed significantly worse fit for the model with equal loadings and the model with equal intercepts. Because there is some doubt in the appropriateness of comparing models based on goodness-of-fit indices when using DWLS for ordinal indicators [29], we also analyzed the impact of releasing thresholds and loadings for one item at a time. We found that only a few item’s associated factor means were influenced and that effect sizes were relatively small and in most cases canceled each other out.

This is in line with Costa et al. [9] who found little bias in the comparison of patients with various primary cancers. While a previous study by Marzorati et al. [8] found MI with regard to sex, we only found marginal differences on item level. Also, findings from a study by King-Kallimanis et al. [7] and Marzorati et al. [8] indicated some measurement bias based on age; however, in those studies and in our study effect sizes of the changes in the item’s associated factor means were small to medium. Other studies found measurement bias with regard to change over time (patients pre- and post-cancer treatment) [4, 5], while we, with regard to time since diagnosis, and Scott et al. 2009 [6] with regard to disease trajectory, did not. This discrepancy could be attributed to the heterogeneous cancer sample in the latter two studies, and to the operationalization of measurement occasion (i.e., within-group comparisons [7] versus between-group comparisons at various time intervals (the current study and Scott et al. 2009 [6]).

Considering the relevance of determining MI in order to legitimately compare subgroups, research into MI of questionnaires is important because both clinical and scientific decisions are based on between-group comparisons of QoL scores. QoL and other types of patient-reported outcomes are now increasingly recognized as important outcomes in cancer research, where they complement the more traditional outcomes such as overall survival [30]. Standardized questionnaires with adequate psychometric properties are also vital for daily clinical practice, as patient-reported outcomes are increasingly used to anticipate more adequately the changing problems and needs of patients [31, 32], which in turn has the potential to improve clinical outcomes (e.g., fewer emergency-room visits, fewer hospitalizations, a longer duration of palliative chemotherapy, and superior quality-adjusted survival) [31].

There are some limitations of our analyses that deserve attention. Firstly, to our knowledge, MI in the QLQ-C30 between the stages of life has not been examined previously. Our results indicate that there is no measurement bias between patients in their last year of life and cancer survivors. However, our group of patients in their last year of life was relatively small for the analysis conducted. Further analysis on this specific patient population is therefore warranted. Secondly, information about cause of death in the subgroup of patients at the end of life and to what extent these patients anticipated their death was not available in our study. It can be hypothesized that knowing that one will die in the near future may have an effect on self-reported QoL. Thirdly, our sample sizes for some of the primary cancer sites were also small. Although there is no clear guidance on sample size requirements for MGCFA with ordinal items, one of the primary cancer site groups included in the analysis barely exceeded 200. Fourth, we were not able to evaluate MI across all primary cancer types, and due to negative error variances, we had to exclude the scales Nausea and/or Pain in some of our group comparisons. We think this is because of the acuteness of symptoms like nausea/vomiting or pain, which, compared to cancer patients on active treatment, are hardly experienced in our sample that largely consisted of cancer survivors. Latest development within the EORTC Quality of Life Group has therefore been to develop a cancer survivorship assessment strategy [33]. Fifth, we were not able to control for cluster effects in our analysis because the software we used currently does not provide cluster-robust SEs/tests for ordinal data. While the intraclass correlations due to study differences between indicators of latent variables were small (< 4%), the design effect of clustering is presumably larger and leading to biased standard errors and Chi-squared tests, so the results of our study should be interpreted with care. Treating the data as continuous and using the MLR estimator while controlling for cluster effects is an alternative approach but unfortunately, we were not able to reach valid model solutions when using this approach. We note that in general, arriving at valid model estimates was challenging for the data at hand, which is presumably the result of seeking to estimate complex models with many constructs, which are each based on a limited number and highly skewed ordinal indicators and with considerably varying cluster sizes. Lastly, concerning the treatment of missing data, we have followed current EORTC guidelines. This means that our missing data approach is not a state-of-the-art method to deal with missing data. Other approaches such as Full Information Maximum Likelihood (FIML) or Multiple Imputation methods may be more appropriate and the comparison of results based on the EORTC guidelines to those acquired by different state-of-the-art imputation methods should definitely receive attention in future studies.

The strength of our study is that it is one of the largest studies into MI of the QLQ-C30, and we used a relatively large population-based sample of patients with cancer. This allowed for an extensive analysis of essential MI levels for group comparison on various patient characteristics. Our findings contribute to the methodological quality of research practices in general which have the potential to improve clinical and scientific decisions making. Our study also raises awareness about measurement bias, as this is often overlooked in the validation phase of questionnaire development [34].

In conclusion, our results show empirical evidence for the valid between-group comparison of QLQ-C30 latent means across groups of time since diagnosis and life stage. We could not confirm the highest level of MI across groups based on age, sex, and primary cancer sites. But given the few instances of non-invariance between these grouping variables, there is reason to be confident that valid conclusions can be drawn from between-group comparisons of QLQ-C30 latent means based on these characteristics. Nonetheless, future research should evaluate the potential confounding effect of variables such as treatment, age, and sex. We stress the importance of including MI evaluation in the development and validation of instruments measuring QoL in heterogeneous populations.