“If we ask, we must act”: co-designing the implementation of the EQ-5D-Y-5L as a Paediatric Patient Reported Outcome Measure in Routine hospital Outpatient Care for Kids to meaningfully impact clinical visits (P-PROM ROCK Phase 2)

When children receive care from doctors, nurses and other healthcare providers, it can be helpful for them or their parents to tell their clinician about how their/their child’s overall health and quality of life is going. This can be done using child friendly health questionnaires, these are also known as paediatric patient reported outcome measures (P-PROMs) or health related quality of life questionnaires. However, we don’t know how we should integrate these questionnaires into care in a way that will meaningfully improve healthcare for children. The point of this study is to work together with young patients, their caregivers, and their healthcare providers to understand the best way to integrate these questionnaires into the care of children in Australia. This study found that simply asking a child or their caregiver to complete the health questionnaire is unlikely to improve healthcare for children. To meaningfully improve care, pathways and supports are needed to help families easily understand and complete the questionnaires, and for clinicians to ensure these questionnaires are looked at and responded to appropriately in appointments. This study designed a program, known as the P-PROM ROCK Program, that outlines how these questionnaires should be integrated into care alongside resources and supports.

Patient Reported Outcome Measures (PROMs) are validated tools used to understand a patient’s health, functioning, or quality of life from their perspective [1]. Paediatric PROMs (P-PROMs) are specifically designed for use with children and young people [2]. Where possible, P-PROMs are reported by the child themselves (child self-report) however, where the child is too young or not able to self-report, P-PROMs may be reported by the caregiver or parent (caregiver/parent proxy report) [3]. PROMs can be used in a range of contexts, including routine clinical care, population health research, clinical trials and registries, and health technology assessment [1, 4‐6]. While the use of PROMs in research and health technology assessment is well established, their use in routine clinical care is more recent [4, 7]. PROMs can be a way to support patient-centred clinical care by formally bringing the patient’s perspective of their health to the clinical visit [4, 7]. When PROMs are implemented with the purpose of improving the patient-clinician visit, PROM data can improve communication, decision-making, patient engagement, and satisfaction [4, 8]. PROM data can also be used at the health system level to inform quality improvement and policy decisions [4]. Despite some uptake and evaluation of PROMs in adult clinical care, there has been limited uptake in paediatrics [9, 10].

Evidence suggests that P-PROMs with fewer items and good measurement properties could lead to improved response rates and lower patient burden [1, 11]. The EQ-5D-Y-5L is a short generic P-PROM with five items that has been shown to be easy and quick to complete [12‐14]. Recent evidence comparing the performance of common generic P-PROMs in Australian children identified the EQ-5D-Y-5L as having one of the strongest psychometric performances [15]. Although the EQ-5D-Y-5L has not been trialled in routine clinical care, the adult version of the instrument, the EQ-5D-5L, has been found feasible for use in adult routine clinical care [16]. Given these potential advantages combined with the availability of funding support from the EuroQol Research Foundation, the EQ-5D-Y-5L was selected as the tool to explore in P-PROM ROCK study.

Phase 1 of the P-PROM ROCK study qualitatively explored patient, caregiver and service provider perspectives regarding the potential use of EQ-5D-Y-5L in routine clinical paediatric care [17]. Results from Phase 1 identified that although stakeholders are supportive of using a generic P-PROM such as the EQ-5D-Y-5L, simply collecting EQ-5D-Y-5L data was considered unlikely to have a meaningful impact on the individual patient visit [17]. Stakeholders highlighted that careful consideration is needed to ensure families are supported in completing EQ-5D-Y-5L prior to their clinical visit and emphasised that service providers need to be equipped to use and act on the data captured [17]. Phase 1 also identified that the use of a single score or preference weighted scoring to summarise the data, as is common to the use of such instruments in economic evaluation [18‐20], would likely not be the most useful in individual clinical care: participants preferred their responses to the EQ-5D-Y-5L to be displayed by item [17]. Hence, more research is needed to understand exactly how the EQ-5D-Y-5L should be implemented, including how data from it should be scored and displayed in routine clinical care to ensure it is both feasible and useful.

The International Society for Quality of Life Research (ISOQOL) has developed a user guide for implementing PROMs in clinical practice [21]. The guide and accompanying manuscript note that a PROM program is more likely to be successful if there is meaningful and substantial engagement with stakeholders [21, 22]. This is consistent with the wider research literature that emphasises the importance of meaningfully engaging stakeholders to enable successful implementation of health initiatives following a historical paucity of such involvement, resulting in implementation failure and research waste [23‐25]. A recent systematic review by Bele et al. identified seven studies that had investigated the impact of implementing a P-PROM in routine paediatric clinical care [10]. Of these seven studies [10], only three reported engaging clinician stakeholders in the design of the P-PROM intervention or implementation [26‐28]. No studies reported engaging parents or children. Since that systematic review, only one study could be identified that reported the inclusion of children and caregivers in the co-design of a P-PROM program [29]. More research is needed that includes the perspectives of parent and child consumers, as well as service providers, in the design process. Such learnings are likely valuable to others seeking to implement P-PROMs in clinical contexts.

Building on Phase 1 [17], this paper reports Phase 2 of the P-PROM ROCK study. This study aims to co-design use of the EQ-5D-Y-5L, a generic P-PROM, in Routine Outpatient Care for Kids (P-PROM ROCK), to maximise impact on patient-clinician visits. A further aim was to reflect and report on the co-design process. The P-PROM ROCK Program developed from these efforts will be piloted and evaluated in Phase 3.

Five co-design workshops, nine feedback sessions and two optimisation sessions were conducted.

In this study, adolescent patients, caregivers of patients, and service providers co-designed use of the EQ-5D-Y-5L in routine paediatric outpatient care, developing the P-PROM ROCK Program. This co-designed P-PROM ROCK Program acknowledges that simply collecting a generic P-PROM in a clinical context, without additional resources to support its implementation, completion, or use, is unlikely to have a meaningful impact on patient-clinician visits. Consequently, the P-PROM ROCK Program is multi-dimensional, wrapping around the patient, their caregiver, and service providers, delivering support at different points along the journey of EQ-5D-Y-5L implementation, completion and use in routine clinical care. The P-PROM ROCK Program includes clinician and patient education, a clinically meaningful display of EQ-5D-Y-5L responses, a patient-centred approach to flagging a concerning response, integration into local workflows and systems, and resources to support patients, caregivers, and service providers to respond to EQ-5D-Y-5L information. Engaging patients, caregivers of patients, and service providers in the co-design process was feasible, with all participants able to meaningfully contribute to the design. Furthermore, this engagement was considered essential to generating a user-centric design.

This is the first published study to report on the co-design process of a P-PROM program that includes patients, their caregivers, and service providers. Previous studies had either not involved any end users in their P-PROM program design, only included service providers [26‐28], or had not reported the details of the co-design process [29]. This is also the first study to report use of EQ-5D-Y-5L in clinical settings [10]. The inclusion of adolescents and their caregivers in this co-design process is a strength of this study and was pivotal to ensuring the design was acceptable and applicable to patients and their families. It was important to include both adolescents and caregivers, as their voices differed. Adolescent participants brought joy and creativity to the co-design team. A service provider participant reflected that the adolescents’ creativity served as a reminder they were designing something for a children’s hospital. Bringing providers together with patients and caregivers kept a focus on patient relevance and guided decisions back toward patient benefit. After the workshops, several participants reflected on the benefits of this joint involvement, noting this broadened their thinking beyond their individual experience. Involvement of service providers in the co-design was critical to gain their buy-in. Many co-design processes keep service providers and patients separate, whereas our experience suggested significant benefits of bringing these groups together. Another strength of this study was the use of two methodological frameworks, [31, 32]. These two frameworks were complementary and without both it is likely important steps and approaches would have been missed. For example, without the seven-step co-design framework, key steps in the co-design process may have been missed, and without the Double Diamond Model, certain discussions or ideas may have been missed [31, 32]. Additionally, as part of Phase 1 of this study, the recruitment of participants for this co-design study was given careful consideration as part of the ‘recruitment’ step of seven-step co-design framework. As a result, participants in this co-design study reflected a range of different medical areas (chronic, episodic, acute, surgical) and roles (service providers, patients, caregivers). Moreover, within these groups, careful consideration was given to ensure representation from different key sub-populations. For example, some patients have more complexity and hence have contact with a wide range of different health services, whilst others only have contact with a single service, and participants with these different experiences were included in the study.

Including patients, caregivers, and service providers together in workshops was not without its challenges. Firstly, the sample size had to be kept to a size that would enable all participants to share their thoughts in a whole group context. Secondly, scheduling a time for workshops that all participant groups could attend was difficult. Consequently, only two online co-design workshops were able to be conducted with all participant groups present and feedback was largely obtained on an individual basis. This may have resulted in different feedback than a group context. A limitation to hosting the workshops online was that quite a few participants were not familiar with the technology which may have impacted their ability to fully engage, especially with creative tasks. Future studies may consider meeting with participants one-on-one beforehand to ensure they are confident with the technology. Potential power imbalance between participants is another limitation of this study; however, efforts were made to minimise this, including ground rules, having designated times where services providers and patients/caregivers were separated, and giving participants different ways to engage in discussion, including anonymously. One adolescent participant reflected that the creative activities were a great way to reduce power imbalances. Additionally, one caregiver participant reflected on how respectful all participants were, even when disagreeing on something. Several participants reflected on factors that contributed to this respectful environment where they felt comfortable sharing, including: (1) consistent relationship with facilitator throughout, (2) same participants in each workshop, and (3) trust in facilitator. Service providers who took part in this study may also reflect those who are early adopters and more open to the use of P-PROMs in clinical care, hence, future research should explore if these findings hold true with service providers who are more hesitant about the use of P-PROMs. Non-English-speaking participants were not represented in this study, which is a limitation and should be a focus of future research. Additionally, it is unknown how elements of this design may be generalised to other age groups, clinical settings, and geographic contexts. This should also be the focus of future research.

When the P-PROM ROCK Program is compared with other existing PROM programs, there are important differences and similarities. In terms of displaying PROMs for use in routine clinical care, different programs have taken different approaches. For example, an adult kidney care PROM program in Canada, known as the EMPATHY trial, displays EQ-5D-5L responses in a report card style, with different response levels categorised as a tick, a caution sign or a stop sign [16]. This differs to the P-PROM ROCK Program display, as participants wanted to avoid displays that looked like school reports or framing responses in a way that could be perceived as failure (colour red in a traffic light system). They also wished to avoid applying external thresholds or value judgements to responses, preferring a system that empowers patients to decide what is flagged for discussion. A P-PROM program in the Netherlands, known as KLIK, displays PedsQL domain scores using a traffic light colour system for clinicians but not for families or children, due to similar concerns identified in this study regarding negatively framed displays [39]. When flagging results for clinicians, the KLIK system uses external thresholds to determine what is flagged as red or orange to clinicians. As discussed above, participants in this co-design study wanted patients to have control over what is flagged as an issue based on what is a concern to them, hence they designed a flagging system where the patient or caregiver is in control of what is flagged for discussion. Both programs displayed PROMs by item or domain, similar to the P-PROM ROCK Program, which may suggest this approach could be applied more widely in routine clinical care. This differs to how the EQ-5D-Y-5L is typically scored when used in clinical trials, where the aim is to facilitate use in economic evaluation, and is an important finding of this study [18]. In terms of resources and supports, service provider participants in the P-PROM ROCK study appreciated the simplicity of the decision support tool used in the KLIK P-PROM program [39]. Consequently, a very similar approach was taken in this study. However, additional supports were also generated as part of the P-PROM ROCK Program that focus on how to action concerns identified on the EQ-5D-Y-5L. These supports were categorised by level of concern: urgent (supports for clinician to action in appointment) and mild/moderate (community supports for families to action after appointment). The differences and similarities between the P-PROM ROCK Program and other programs may be due to the local context or the addition of patients in the design process.

This Phase 2 study builds on our previously published qualitative Phase 1 [17], and provides a co-designed program for implementing, collecting and using a generic P-PROM, the EQ-5D-Y-5L, in clinical outpatient care (hospital-based visit), in an Australian tertiary paediatric hospital. A key finding from this study, based on the perspectives of the service providers, caregivers and adolescent patients who took part, is that if patients are asked to complete P-PROMs, there is a duty of care for this information to be reviewed and acted upon appropriately by service providers. Involving children, caregivers and healthcare service providers in the co-design of P-PROM collection processes, displays, training, and supports to respond, provides essential insights and gives P-PROMs the best possible chance of successful implementation and impact on patient care. The ultimate test of whether routine use of P-PROMs is impactful is whether the data are actively used in healthcare decisions, and whether healthcare is delivered in a way that better meets patients’ needs. The planned evaluation of the P-PROM ROCK Program, Phase 3, will provide evidence on these outcomes.