Development of a prediction model for clinically-relevant fatigue: a multi-cancer approach

Globally, the cancer survival rate has increased over time as standards for treatment and care continually improve [1]. However, patients with cancer continue to experience long-term consequences [2], with fatigue being the most prevalent across cancer types, cancer treatment regimens, and throughout different stages of the disease trajectory [3]. Furthermore, across a wide range of cancer types, patients often experience persistent fatigue after treatment [2, 4, 5], which can have a long-term negative effect on a patient’s health-related quality of life (HRQoL) [2, 4, 6]. Hence, patients might benefit from early fatigue-related supportive care [4].

A common method to measure a patient’s fatigue is by using patient reported outcome measures (PROMs). The European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Core-30 (EORTC QLQ-C30) [7], which assesses patients functioning and symptoms, has been used and validated in various cancer populations [8, 9]. Clinical relevance thresholds have been defined for the EORTC QLQ-C30 to improve the interpretation of the scores in daily clinical practice [10]. Using this threshold, predicting future clinically relevant fatigue (CRF) would support clinicians in selecting patients who need long-term fatigue-related supportive care.

To better predict CRF in patients with cancer, prediction models can be defined and used to anticipate a patients’ future level of fatigue to inform patients and their health care team of their potential need for supportive care [11]. However, prediction models for cancer are more commonly used to predict cancer survival [12‐14], support diagnostics, and, to a lesser extent, provide information on HRQoL [15, 16]. Within predictive modelling, machine learning (ML) is increasingly utilized [17‐19]. To date, few ML-based models have been developed specifically to predict cancer-related fatigue. These models were predominantly cancer-specific, such as for breast cancer [6, 18]. Such specification overlooks a wider adoption of CRF-related prediction across cancer types [6]. Further, the performance of previously reported models predicting CRF varied across studies due to missing relevant predictors, such as functioning and symptoms [19], or having homogeneous data [18]. Therefore, developing a ML-based prediction model validated for patients with multiple cancer types using PROMs as predictors would address the research gaps currently present within the cancer-related fatigue domain.

The aim of this study is to develop and compare prediction models to predict CRF in multiple, heterogeneous cohorts of patients with non-metastatic cancer between two and three years after diagnosis [20], using ML-based models. Predictors include clinical and sociodemographic factors, as well as HRQoL domains (functioning and symptoms) reported within three months after diagnosis. In addition, the study aims to assess the validity of such prediction models across diverse cancer populations.

Alongside logistic regression, eXtreme Gradient Boosting (XGBoost), support vector machines (SVM), and artificial neural networks (ANN) were employed in this study. Each algorithm had demonstrated good results as a tool for developing prediction models in previous studies and reviews [12, 13, 20, 33‐35]. All model development steps (Appendix 3) were conducted using various packages under R version 4.2.1 [27].

First, a randomly selected 80% of the data was used for model development and the remaining 20% was used for model testing. Categorical predictor variables were modelled using one-hot encoding [36]. Hyperparameter tuning was performed using a random search, validated through ten-fold cross-validation, and used balanced accuracy as the defining criterion. Class imbalance, where the ratio between the number of patients with CRF (minority class) and the number of patients without (majority class) was too high to reliably develop a prediction model, was addressed using the Synthetic Minority Oversampling Technique (SMOTE). SMOTE utilizes k-Nearest Neighbours to create synthetic examples of the minority class in the outcome measure, thereby oversampling the minority class [37]. This study used the default k = 5 neighbours, and the default ratio between the minority and majority class was 1:3, oversampling the minority class such that 50% of the model development data consisted of synthetic examples.

This study aimed to develop and compare prediction models to predict CRF in different, cancer patient cohorts. The goal was set to address a research gap and support clinicians in selecting patients who need long-term fatigue-related supportive care. This study found that the logistic regression model with LASSO regularisation performed better than the ML-based models, indicating that a simpler model is preferred for creating prediction models for future CRF. The combination of multiple, heterogenous cohorts with different cancer types in the same model was feasible, as the logistic regression model remained stable across the various cohort studies. However, for all the models, calibration was poor, which suggested a high risk of underdiagnosing CRF. This poor calibration may be explained by class imbalance, i.e., the lack of patients with CRF as an outcome.

Unfortunately, SMOTE only partially improved the class imbalance in our study. The influence of class imbalance was reflected in an imbalance between sensitivity and specificity, with sensitivity being near zero. In a clinical application, it is preferable to have both high sensitivity and high specificity, as this ensures that type I (false positives) and type II errors (false negatives) are minimised [51]. Although we based our dichotomisation on the threshold for clinical importance defined by Giesinger et al. [10], a lower threshold for fatigue did not significantly reduce class imbalance in our sensitivity analysis. Therefore, using a continuous representation of the fatigue scales as an outcome measure may potentially alleviate class imbalance.

Following internal–external cross-validation, each cohort performed consistently with regard to each other as well as to the overall model with all cohorts included. This was evident in most of the performance measures. Regarding balanced accuracy, the average value was consistent throughout all cohorts, albeit still as low, as the overall model. This meant that, when tested, each cohort performed similarly with respect to class imbalance [48]. Aside from balanced accuracy, this consistency was also observed in each cohort’s sensitivity and specificity.

A strength of this study is the combination of multiple, heterogenous cohorts with different cancer types in the same model. Further, this study demonstrates that predictors missed in previous studies [18, 19], such as functioning and symptoms, can be included into a prediction model for CRF. Alternatively, other questionnaires measuring these constructs could be used in future research. Limitations include the initial data collection and processing, which revealed a considerable class imbalance between patients with and without CRF at both T_baseline and T_endpoint. Previous studies have highlighted the issue of non-respondents and attrition bias within the PROFILES registry [53, 54], indicating that patients with lower quality of life and more symptoms are underrepresented in the current study. Hence, a selection bias favouring less-fatigued patients in our study is inevitable. This bias was reflected in the relatively low proportion of patients reporting CRF (11.9%), which is much lower than other studies studying CRF (e.g., 38% [18]). Therefore, while SMOTE could not completely correct the class imbalance in our study, and sensitivity analysis using different thresholds did not suggest an improvement, future studies should focus on reducing selection and attrition bias. Moreover, when dealing with class imbalance, future research could also investigate whether using other methods, e.g., simple random oversampling, provides comparable performance to SMOTE within this context.

Due to the current unavailability of new, unseen data, external validation was not possible for this study. We believe that the internal–external validation procedure was a robust method for assessing the model’s performance across different cohorts, offering preliminary evidence for external validity. Future research should utilize new datasets from the PROFILES registry, including additional cancer types, as they become available. Furthermore, collaborations with external and international institutions will be essential for conducting comprehensive external validation across diverse cancer populations. Subgroup analyses should also be conducted to evaluate the model’s performance across various demographic and clinical subgroups. Additionally, the exploration of federated learning techniques is recommended to enable model training across multiple centers while ensuring the protection of patient data privacy.

Further, there is value in investigating the extent to which a patient’s fatigue changes over the shorter term (e.g., at 12 months) or within intervals (e.g., every three months). Output from such models could help to adapt care more quickly [55], also when predicting other symptoms. Moreover, the timepoints defined in the current study provided nuanced differences for prediction and subsequent interpretation. Future research should, therefore, incorporate timeline anchors (e.g., date of diagnosis) into prediction models to provide a well-defined point of reference is available for clinical interpretation.

In addition, given that CRF can potentially emerge at any timepoint after baseline measurements, a longitudinal time-series model should be explored as a modelling strategy to capture CRF. In our analyses, we developed statistical models for predicting the (non-)presence of CRF 12 months after diagnosis. Due to insufficient time-series data, we could not adopt a longitudinal modeling approach. The main benefit of a longitudinal time-series is that it can capture CRF at any time point since diagnosis and is not restricted to one year after diagnosis. However, the main disadvantage is that longitudinal models are more complex and, therefore, more challenging to interpret. We expect that time-series models would better capture future fatigue because they utilize not only baseline fatigue as input but the whole time-series, which can include autocorrelation and may be informative for future predictions (i.e., if a person has elevated fatigue in month X, they are likely to experience fatigue in month X + 1 as well). A potential challenge is the consistent availability of the data at evenly spaced time points. This condition is not always met for different reasons and can create heteroscedasticity in the data (i.e., unevenly distributed observations across time), which would need to be addressed by a more complex statistical model, that accounts for the (un)conditional variance, such as weighted least squares, which uses the inverse of the (estimated) variance of each observation as a weight.

Finally, data on recurrence and its subsequent treatment were not incorporated in the current study, which could have had a substantial effect on the occurrence of CRF at T_endpoint. Recurrence and its related treatment regimen could potentially be incorporated into the model in the form of time-varying parameters. However, this would require including “future” data not observed at T_baseline, which was not the purpose of the current model. Future research could use a counterfactual interpretation of the predictions of such a time-varying parameter model by, for example, predicting fatigue at T_endpoint conditional on the presence (or absence) of recurrence at a future time point.

In this study, we did not discuss ML explainability measures that have received prominence in recent years, such as Shapley values, because our best model (i.e., logistic regression) is readily explainable: a linear relationship is established between CRF and the predictors in the logit space. The coefficients indicate the change of CRF per unit change of the predictors, and, therefore, clinicians can directly see the effect of each predictor on CRF. Had any of the more complex ML models shown better performance, ML explainability would be paramount to investigate how the models arrive at their predictions.

To conclude, this study demonstrated that CRF prediction modelling using logistic regression and ML is feasible, when combining multiple, heterogenous cohorts. Although the model’s discrimination was good, the low sensitivity and poor calibration indicated a high likelihood of underdiagnosis of future CRF. Yet, compared to different ML algorithms, logistic regression performed better and was robust across cohorts, which suggests the advantage of having a simpler model to predict CRF. Despite demonstrating how CRF can be predicted, the clinical applicability of the model based on this study remains inconclusive. Therefore, to support clinicians in selecting patients who need long-term fatigue-related supportive care, future research should consider reducing selection/attrition bias and include recurrence data when developing CRF-related prediction models.