A systematic review of health state utility values and psychometric performance of generic preference-based instruments for children and adolescents with mental health problems

Mental health problems (MHPs) denote a worldwide public health challenge and one of the leading causes of disability in children and youth population [1, 2]. With the prevalence being around 14% worldwide, MHPs often account for a high global burden of disease (13% of disability-adjusted life-years) [3]. MHPs often persist to adulthood, impacting not only the health and well-being of children and youth but also their socioeconomic trajectories and family life [4]. Depression, anxiety and behavioural disorders are among the leading causes of disability while suicide is one of the leading causes of death among youths [3].

Economic evaluation is increasingly used to inform decision-making in priority setting worldwide, and is often recommended by government agencies [5]. Economic evaluations using cost-utility analyses (CUAs) commonly require the estimation of quality-adjusted life-years (QALYs) to capture the benefits resulting from health interventions. QALYs are calculated by combining length of life and health-related quality of life (HRQoL), where the time spent in a particular health state is weighted by corresponding health state utility values (HSUVs)—used to denote the “quality” in QALYs [6]. HSUVs represent the strength of preference for a particular health state and are anchored on a cardinal scale between 0 (dead) and 1 (full health), where full health is usually defined by the domains of HRQoL measured on each instrument [7].

HSUVs can be derived using either direct or indirect valuation methods [6]. Direct methods include choice-based valuation methods, such as standard gamble (SG), time trade-off (TTO), discrete choice experiments, or best–worst scaling to value health state description (vignettes) or individual’s health state. Indirect valuation method involves the use of generic multi-attribute utility instruments (MAUIs) which usually comprise a descriptive system (largely health related quality of life questionnaires) describing various dimensions of HRQoL, and a preference-based scoring algorithm (tariff) used to convert responses from the descriptive system to a single, numeric HSUV [7]. MAUIs are available as self-report (i.e. one assesses one’s own health state) or proxy-report (i.e. one assesses the health state experienced by someone else) [7].

In modelling-based economic evaluations, analysts may not have the resources and time to acquire original HSUVs for health states of interest, thus often relying on systematic reviews on HSUVs as a robust and transparent source of evidence for various health conditions. Furthermore, HSUVs can be used to explore the burden of disease, especially when compared to population norms or people without the disorder in question. To the best of our knowledge, there has been no previous systematic review on HSUVs for children and youth with MHPs. Furthermore, with the reorientation of mental health services aimed at the young population, which is widely defined as those aged between 15 and 25 years [8], an age group that often experiences a high rate of MHPs [9], exploring HSUVs for the age range up to 25 years is helpful to support studies aimed at this age group which would otherwise be unintentionally excluded by the artificial cut-off point of being 18 years old between adolescents and adults [10].

Over the last decade, while there has been an increased focus on the valuation methods using MAUIs to produce HSUVs for children and youth, the guidelines from international agencies for assessing health outcomes in this population remain unclear [10, 11]. This is concerning as an adequate measure to capture healthcare interventions’ benefits plays a crucial role in the robustness, transparency and rigour of economic evaluations. In adult population, it is well reported that some commonly used MAUIs lack sensitivity to measure the impact of MHPs on quality of life [12]. Specifically, there is evidence that the most commonly used MAUIs, such as the EQ-5D and SF-36 are not sufficiently sensitive to reflect the impact of severe MHPs such as schizophrenia [13, 14], or bipolar disorder [15]. Meanwhile, the psychometric performance of MAUIs used in children and youth with MHPs is not well explored.

The aims of this paper are three-fold: (i) to identify reported utility values associated with MPHs in children and youth aged less than 25 years and conduct a meta-analysis of reported HSUVs if appropriate; (ii) to summarise the elicitation techniques used to derive utility values among children and youth with MPHs; and (iii) to provide a summary of the evidence on the psychometric performance of the identified MAUIs used in this space.

This paper, for the first time, provided a systematic review of evidence on (1) the HSUVs of various MHPs, (2) the current practice to generate HSUVs, and (3) the psychometric performance of MAUIs used in children and youths with MHPs. The review found 38 studies reporting HSUVs for 12 types of MHPs in children and youths aged between 4 and 25 years across 12 countries between 2005 and October 2021. Among 12 types of MHPs reported in the review, depression and ADHD are the most explored MHPs, with 10 studies each. Overall, the review found that Disruptive Behaviour disorder has the lowest HSUV of 0.06 from children in primary education valuing vignettes in a cross-sectional study [43] while cannabis use disorder was associated with the highest HSUVs of 0.88 from an intervention study [44]. Moreover, the indirect valuation method using MAUIs (used in 95% of included studies) is the most frequently used approach to generate HSUVs for all 12 MHPs while only two studies using the direct valuation method was focused on ADHD. Of the ten MAUIs found in this review, the most frequently used was EQ-5D-3L, used in 15 out of 38 studies across eight out of 12 types of MHPs. CHU9D, the only MAUIs developed for use in childhood specific populations was used in 13 out of 38 studies across seven out of 12 MHPs. However, the review found no measure that has sufficient evidence of psychometric performance in children and youths with MHPs.

Heterogeneity was noticeable across the included studies. The variations, mainly caused by different elicitation methods and MAUIs used, scoring algorithms, age range, country of research, and study designs, have made it difficult to derive an estimate of the overall effect. For example, using samples of 11–17 years old adolescents at risk of self-harm, one study reported an HSUV of 0.68 using EQ-5D-3L with the UK adult tariff [28] while another study reported an HSUV of 0.57 using CHU9D with the Australian adolescent tariff [24]. Even using the same clinical sample, different MAUIs produced different HSUVs, for example, an HSUV of 0.56 derived from HUI3 while an HSUVs of 0.81 from EQ-5D-3L based on the same sample of adolescents with depression [57]. These findings are consistent with results from adult literature where different MAUIs produced different scores even in the same person [67]. Furthermore, only one study reports the populations norms for several MHPs [24], none of studies compared HSUVs with their corresponding populations norms. Thus, future research may benefit from such comparison.

We found some MAUIs were used for samples with the age range different from their recommended age range. Specifically, EQ-5D-3L, an adult-specific MAUI, was used in children aged as young as 4 years through proxy-report [31] or as young as 8 years old using self-report [41]. While the recommended ages for 16D are 12–15 years [10], the review found that the 16D was used in adolescents as young as 10 years old [60] or as old as 22 years [48]. We noted that studies may use measures for sample outside of the recommended age range as a way to save time and resources, or to facilitate comparisons across different age groups. However, transitioning between child and adolescent/youth populations can be challenging as there are significant developmental differences between these age groups [68]. Therefore, we urge future research to consider the potential limitations and accuracies that may result from this approach.

This review found limited evidence of the psychometric performance of MAUIs used in children and youths with MHPs with only nine studies covering eight MAUIs in five types of MHPs. Of these MAUIs, more studies assessed the psychometric performance of EQ-5D-3L than other measures. Only CHU9D was assessed using a sample with general MHPs while other MAUIs were assessed using a specific MHP samples. Specifically, EQ-5D-3L was assessed in ADHD and depression, EQ-5D-5L was assessed in PTSD, EQ-5D-Y-5L was assessed in psychiatric disorders, and QWB, SF-6D, HUI2&HUI3 were assessed in depression. Furthermore, we found that MAUIs were used in MHPs where evidence on their psychometric performance is lacking, such as the use of EQ-5D-3L in personality disorder [38, 63], behaviour disorders [43], substance used [44, 45, 65] and psychiatric disorders [39].

Evidence on the psychometric performance of MAUIs used in children and youths with MHPs is not conclusive, which makes it difficult to recommend one MAUI over another. Although the best psychometric evidence exists for EQ-5D-3L and CHU9D in terms of discriminant validity, convergent validity and responsiveness, other psychometric properties remain unclear. For example, both CHU9D and EQ-5D-3L have not been assessed regarding their feasibility, test–retest reliability, and interrater reliability. The evidence of other MAUIs was found either mixed or limited in number of studies and type of MHPs explored. Specifically, EQ-5D-5L was found to lack responsiveness while EQ-5D-Y-5L showed inconsistencies in its ability to converge with other measures. Although the psychometric performance of EQ-5D-5L and EQ-5D-Y-5L may look alarming, it is worth noting that the evidence was derived from one study per measure with small samples less than 100 which may limit the statistical power of the results. The findings on the convergent and discriminant validity of EQ-5D-3L, SF-6D and HUI3 in adolescents with depression align with findings in the adult depression literature [14, 15]. The limited evidence in terms of type of MHPs explored, extensive psychometric properties, number of studies with sufficient samples urges further research to provide comprehensive picture of the psychometric properties of all measures used in children and youths with MHPs.

Indeed, various aspects of MAUI psychometric assessments need further attention. Specifically, we found limited or no evidence in test–retest reliability, internal consistency, and interrater reliability of MAUIs. Various validity tests such as face validity and content validity would also contribute to more conclusive evidence of the psychometric performance of MAUIs in children and youths with MHPs. Lastly, although MAUIs were used across different countries using different languages, the review noted that the cultural validity of MAUIs has not been investigated.

Limitations of our study include the inability to conduct a meta-analysis of reported HSUVs due to the heterogeneity of the included studies and the small number of studies per MHP, which would otherwise provide an overview picture of the impact of MHPs on HSUVs and the associated disease burden [22]. Non-English studies and grey literature were excluded, which may limit the comprehensiveness of our review. Furthermore, following previous systematic reviews [10, 11, 19, 23], we did not conduct quality assessment of the included studies as well as the appropriateness of the statistical analyses undertaken in included studies, however results of included studies were discussed in consideration of sample sizes.

This review provides an overview of HSUVs of various MHPs, the current practice to generate HSUV, and the psychometric performance of MAUIs used in children and youths with MHPs. We report limited evidence on the psychometric performance of MAUIs in terms of psychometric properties assessed, type of MHPs explored, number of studies per measure and sufficient sample sizes. This highlights the need for more rigorous and extensive psychometric assessments to produce conclusive evidence on the suitability of MAUIs used in this area.