"not one size fits all” The challenges of measuring paediatric health-related quality of life and the potential role of digital ecological momentary assessment: a qualitative study

This was a qualitative study involving one-to-one semi-structured interviews. We recruited clinicians and researchers with experience using HRQoL measures with children (5–11 years) via professional networks and snowballing [15]. We sent invitations via the National Institute for Health and Care Research (NIHR) paediatric methodology incubator, the Centre for Development, Evaluation, Complexity and Implementation in Public Health Improvement (DECIPHer), and the Centre for Academic Child Health (CACH, University of Bristol). The recruitment channels were UK-based, and so we anticipated only recruiting experts from within the UK. We also used snowballing, and because of this, we ended up recruiting three participants from outside of the UK (see Table 2). Study invitations included a participant information sheet and an online expression of interest (EOI) form (captured via REDCap [16]). The EOI form captured contact details, sampling and eligibility data (demographic and professional characteristics). We purposively sampled, aiming for diversity in the professional background, geography, gender, and ethnicity of participants. We contacted sampled participants and obtain consent via REDCap. Participants were offered a £20 voucher for participation. We undertook iterative sampling, analysis and sampling until reaching data saturation [17]. We ended recruitment when we achieved a diverse sample (particularly considering professional backgrounds), when we were no longer creating additional codes, and when we determined additional interviews were no longer adding additional information (saturation).

We conducted one-off interviews designed to last 60 min, conducted face-to-face or online via Microsoft Teams. Only the interviewer and participant were present. The semi-structured topic mapped onto our research questions: the challenges of measuring HRQoL in children (section one), views on using Digital EMA and passive/sensing data to capture HRQoL (section two), see Table 1. Interviews were audio recorded, transcribed verbatim using a professional transcription service, and pseudonymised.

We analysed data thematically [19] (AB, HF, LT), with each author analysing the transcript from their completed interview. Following initial familiarisation with the data, we completed independent coding of the of the first set of transcripts undertaking line-by-line coding. We then meet to review the initial codes and collate these into a coding framework, which encapsulated all the initial codes. We set up the coding framework in NVivo [20] and applied this to transcripts. The coding framework was refined as additional transcripts were analysed. Through repeated discussion, we refined codes and grouped these into themes and sub-themes. Analysis was inductive and driven by the data.

Interviews and analysis were conducted by LT (a female PhD student, using qualitative methods in her PhD and with a Health Psychology MSc), HF (a female PhD student with a background in interdisciplinary psychology and neuroscience, and with previous experience of qualitative methods), AB (a female Lecturer, who used qualitative methods during her MSc and PhD). The authors (AB, MR, EC) had experience in paediatric clinical trials using HRQoL measures, and this experience stimulated their interest in improving paediatric HRQoL measures. The researchers were interested in EMA applied to paediatric HRQoL, but had no conflicts of interest in this line of research. For most interviews, there was no existing relationship between the participant and interviewer. In two cases, the interviewer had previously worked in the same University Faculty but had a limited prior relationship.

We received 26 EOI forms. We interviewed 18 participants, see Tables 2 and 3.

We developed four themes. See Fig. 1 for an overview of themes; Table 3 for themes, subthemes and illustrative quotes; and the supplementary file for additional supporting quotes.

Participants questioned whether the domains included in measures were “applicable” for all children. Some participants questioned the extent to which children can “unpick” (Participant 5, research project manager) different domains of HRQoL- for example, questioning children’s ability to distinguish mood from physical discomfort. Some participants suggested dropping discrete domains favouring a single, global assessment. Whilst others felt it was important to keep multi-dimensional domains to maintain sensitivity and to “capture the heterogeneity in kids’ quality of life” (Participant 6, Epidemiologist).

Participants questioned the extent to which children have the capacity to engage in preference tasks to determine utility weighting from domains. This value preference work has yet to be done in children under eight years, and participants said it was unclear whether the older children (around eight years) did engage appropriately with these judgement tasks.

Participants talked about children reflecting on their HRQoL in a more immediate way, being less likely to reflect back over long time periods and being more influenced by the immediate context. Clinicians described the challenge of obtaining a complete, accurate clinical picture when the child’s account is restricted to recent events and the impact of this on determining appropriate treatment.

Participants also talked about children being more “sensitive” (Participant 3, paediatrician/researcher) in how they experience or report their HRQoL, resulting in higher variability. This can create problems when measuring HRQoL infrequently, with infrequent measurements failing to capture fluctuation and creating “guesswork” between the measured time points.

Because of these differences in how children report HRQoL, some participants said it might not be valid to ask younger children about their HRQoL (“[HRQoL measures] can’t be used in under-eights”, participant 6, epidemiologist). Others said children should not be underestimated, saying it is important to give children the “benefit of the doubt” (Participant 11, nurse/research fellow).

Themes 1.1 and 1.2 highlight the difference in how children conceptualise, reflect on and report their HRQoL. These differences are tied to development, and as children mature, so too will their conceptualisation, reflection and reporting of HRQoL. Participants highlighted that HRQoL measures typically deal with these developmental changes by using fixed age bounds for measures. Participants described this fixed approach as incongruous with the highly individual nature of child development. As an alternative, they suggested selecting measures based on the child’s individual ability rather than age.

To respond to individual differences, participants suggested tailoring domains and tailoring weightings to the individual.

Participants also talked about individualising the formats of measures, offering formats tailored to the child's preferences (e.g. text versus visual). Further, participants described a need to tailor measures to different clinical contexts, for example, acute versus chronic conditions.

Participants flagged potential "discrepancy” between parent and child reports, driven by different conceptualisations, priorities, and perspectives; child reports and proxy reports are “not really the same data point” (Participant 5, research project manager). For example, parents may take a longer-term view of the child’s HRQoL compared to the child’s immediate perspective (as described in theme 1.2). Participants had different views on whether the child or proxy report is more accurate, and one participant questioned the notion of one view being more “accurate” than the other. In general participants said that as children get older, there is a reduced use of parent-proxy and an increased privileging of the child’s perspective.

Theme one described the challenges of infrequent data collection and variability of HRQoL in children. Many participants talked about the value of EMA repeated measurement in this context. Researchers spoke about the “really rich data set” (Participant 18, academic) produced by repeated measures, the increased ability to capture variability, the “peaks and troughs” (Participant 10, health economist) and “get a better picture of what’s going on” (Participant 10, health economist). Clinicians discussed the value of more data points in between appointments, with in-the-context data enabling an understanding of “triggers” (Participant 9, nurse consultant) and the impact of “setting” on the child’s experiences (Participant 9, nurse consultant). One participant described repeated measures between appointments as a mechanism for remote monitoring and reducing the need for clinic visits.

Participants also reflected on the burden placed on clinicians and researchers using and analysing the rich, complex datasets produced by repeated EMA and passive data. Researchers would need to consider resourcing specialist statistical training or data analysis support, and there would need to be consideration of how the complex data could be analysed in line with established Health Economic analysis guidelines (e.g. how to develop quality adjusted life years from HRQoL measures). Participants flagged the significant groundwork needed to make sense of the passive/sensing data; understanding norms, confounders, and highly variable data. Clinicians recognised the lack of time/capacity to work with a volume of raw longitudinal EMA and passive data. They expressed a need for succinct insights and overviews, through effective data visualisation.

Reconciling richness and burden Participants discussed methodological solutions to address the tension of richness versus burden arising from EMA. Item reduction could mitigate the impact of the burden of repeated measurement. However, there were divergent views about item reduction. Some stated briefer measures (even “one question”) would be “more likely to gain acceptance”, "more valuable in clinical practice" (Participant 13, general practitioner) and are typically used for health economic evaluation. Others were concerned about losing valuable, “multifaceted” (Participant 10, health economist) information. Participants did raise the need for short measures to be “validated” and “reliable” (Participant 15, researcher). Another methodological solution was the use of “short, discrete” (Participant 2, paediatrician/academic) periods of EMA measurement to offset the burden (similar to the concept of measurement bursts [21]).

When making decisions around balancing richness and burden of EMA, weighing up methodological choices like frequency, item reduction and measurement bursts, participants said that the decision-making needed to be tailored. Tailoring to the specific health context of use (e.g. acute conditions needing more frequent measurement to understand changes than chronic), the individual needs of the child, considering factors such as age, and whether it was being used in a clinical versus research context. In some contexts, the priority would be capturing multifaceted data (longer measures over fewer time points); in other contexts, the priority would be temporal richness (repeated measures with briefer measures).

Individualisation Participants described digital EMA as having the potential to offer a variety of formats for viewing and responding to measures, e.g. audio, images, text and even features such as “gamification” (Participant 4, health economist). The ability to offer different formats, adapted/tailored to the individual’s needs and preferences, could increase accessibility and make the measures more meaningful to the child. For example, it could facilitate offering “simple imagery” for younger children.

Participants raised the point that "primary school children aren’t likely to have phones" (Participant 14, health economist). Some talked about wearable devices (e.g. smartwatches) as a practical and appropriate technologies for this age group, though some still felt this was inappropriate for younger children (under eights). Participants raised concerns about the practicality of administering HRQoL measures via wearable devices, emphasising that solutions needed to be “workable and simple enough” (Participant 7, general practitioner/academic). There were concerns about the child’s ability to navigate the technology independently, particularly for younger children (those under eight years). There were also the risks of devices being lost, forgotten or left uncharged; the inability to wear the devices in certain contexts (e.g. whilst at school, during physical exercise lessons, or whilst asleep); and discomfort or aggravation caused by devices. These were also ethical concerns about inequality and inclusion for groups of children without access to new technologies or where their health condition would be a barrier to use (e.g. children with eczema having skin aggravation), as well as concerns about the social impact of the child wearing a visible device (e.g. stigma and bullying) or EMA prompts being too “intrusive”.

In general, a digital approach was seen as “preferable to a paper-based" (Participant 10, health economist), likely to “reduce the barriers” (Participant 5, research project manager), and improve completion rates. Participants described the benefits of wearable technology; the measure being physically present (“on your arm”), the ability to use digital “reminders” (Participant 10, health economist) to prompt EMA completion and the reduced likelihood of losing questionnaires. In terms of smartphone administration, participants commented on the ubiquity of smartphones and preferences for phone-based rather than paper-based measures.

This is the first study to explore clinicians' and researchers' views about the role of Digital EMA and passive data in addressing the challenges of measuring paediatric HRQoL. Key challenges included: uncertainty around conceptualising paediatric HRQoL, with conceptualisation likely to change with maturity; the distinct way children report HRQoL (more immediate, sensitive, and variable); recall issues, compounded by infrequent measurement; and significant individual differences across childhood, incongruous with the fixed content and formats of HRQoL measures. Digital EMA may address these challenges. In-the-moment and in-the-context measurements may address recall issues and ecological validity. Repeated measurement may help with issues of variability, whilst also providing richer datasets. Technology could enable personalisation of content (domains/items) and formats. A key consideration of using Digital EMA is the trade-off between richness and burden, with this trade-off being different for different contexts. There were concerns over the methodological, practical, and ethical challenges of Digital EMA and passive data collection. There were also challenges identified about paediatric HRQoL measurement, which Digital EMA would not clearly address. For example, the challenge of interpreting changes in HRQoL over time (does a change in HRQoL over time reflect a real change in health status, or is it due to developmental changes).

Individualisation/tailoring Our findings highlight the challenge of using fixed/static HRQoL measures with a diverse population, in terms of the child’s development and health context. Technology could be used to tailor HRQoL measures to the context and individual, with the tailoring of protocols (e.g. frequency, item reduction, measurement bursts [21]), content (domains and items), and formats (e.g. text vs visual). Existing technologies enable the individualisation of outcome measurement. Mesmerise software [22] enables researchers to set-up tailored EMA protocols (e.g. tailoring content, frequency and timings of prompts), as well as enabling respondents to set-up individualised parameters for their EMA schedule (e.g. specifying time limits for receiving prompts). There are examples of health-tracking Smartphone Applications designed to administer a core of fixed domains, with users able to select additional tailored domains from a pre-set list of options [23]. Computerised Adaptive Testing [24] uses responses to generate the next question, reducing the number of questions needed to provide an accurate T-score. Technology has also been used to offer a range of formats of outcome measures, including interactive visualise designs [25], video [26] and voice assistants. We are not aware of the use of these novel technologies for paediatric HRQoL, and this warrants further research.

Objective markers Our findings highlight that the subjective nature of HRQoL is one of the fundamental challenges of measuring HRQoL in children: can children reliably report HRQoL? Can proxies meaningfully report on the subjective experiences of children? These findings are consistent with existing literature, for example, findings of discrepancies between child and proxy reports of the child’s HRQoL [27, 28]. Participants described using objective passive/sensing data from wearables as objective markers to complement data on the quality of experience. Using objective markers aligns with Lin et al.’s conceptualisation that “HRQoL includes both subjective and objective components… objective assessment focuses on what the individual can do, such as walking” [2]. Our participants described the additional benefit of objective data placing minimal burden on respondents, consistent with findings that wearable activity monitors foster greater tracking adherence than manual tracking [29]. Future research could explore digital objective markers as a compliment to HRQoL measurement, with attention to issues around accuracy of sensing data, the complexity of managing the outputs (clinically and in research), and practical and ethical concerns about data collection via wearables.

Continuity of measurement, in the context of development changes Participants hypothesised that reflections/reporting of HRQoL changes with development. Younger children may adapt to illness quicker and be more immediate and sensitive in their reflections/reporting of HRQoL (leading to higher variability). This hypothesis, that the mean and variability of HRQoL change with development, is a hypothesis that HRQoL measures are statistically non-stationary [30]. This poses the question- are changes in HRQoL over time due to a change in health status or due to developmental changes in how children report HRQoL? This is worthy of further exploration.

We recruited participants from a range of professional backgrounds, from across England. As such, it is likely that we have captured a range of practices and perspectives on HRQoL measurement. Our robust analysis procedures ensured we represented the range of views, including divergent/conflicting opinions. We reached data saturation [17] for our main aims.

Participants typically provided rich reflections on using HRQoL measures, drawing on their experiences. In contrast, some participants were unfamiliar with EMA, so although all participants did offer their thoughts on EMA, for many these were reflections on a hypothetical concept. Reflections based on real-life technology usage can be more beneficial than hypotheticals [31], and piloting with user feedback would be a beneficial next step for this research. Further limitations include an all-female group of researchers undertaking interviews and analysis, as well as a lack of ethnic diversity in the participant group. However, we think it is unlikely that gender or ethnicity would significantly influence research findings around paediatric HRQoL measurement in the UK. Finally, transcripts were professionally transcribed to ensure data quality. However, transcripts were not returned to participants for comment, which could have enhanced data quality. Further, participants were not asked to provide feedback on the findings.