Introduction

Understanding and capturing patients’ perspectives of their health and impact of healthcare is now recognised as a key component of effective, patient-centred services.1, 2, 3 Patient-reported outcome measures (PROMs) are increasingly advocated and used to achieve this.4, 5, 6 PROMs are questionnaire instruments measuring any outcome related to health, illness or treatment that are directly reported by patients themselves. Different PROMs assess different health constructs. These include health-related quality of life (HRQoL), wellbeing, health status, functional status, participation, and symptoms (eg, pain severity). They are seen as having a potential to improve services and healthcare, by providing validated and standardised patient-assessed evidence of effectiveness and quality at the same time as facilitating interactions between professionals and patients and supporting shared decision-making. Increasingly, they are used to evaluate outcomes of new interventions in the context of trials or in studies of natural history. Importantly, PROMs are to be distinguished from patient-reported experience measures (PREMs), which are used to capture the process of and specific experiences during healthcare (eg, whether the patient was seen on time), as opposed to outcomes of healthcare (eg, change in functional ability or symptoms).7

The need for and value of PROMs is well established in paediatric and child health.8, 9 A plethora of influential and widely used PROMs for children now exists that capture a variety of health outcomes, ranging from HRQoL to symptom severity. These include generic instruments that allow comparisons between different patient populations, as well as disease-specific measures targeted to those with specific conditions.8, 10, 11, 12 It is widely held that children can report on their health validly and reliably using standardised PROM questionnaires from the age of 7 years13, 14 (and possibly as early as 5 years10, 15), given the opportunity and the child-friendly means to do so. Nevertheless, important practical and methodological challenges exist in developing and applying self-report questionnaires for children.10, 13, 14, 16, 17, 18

Working with specialist clinical populations such as those with visual impairment (VI) potentially exacerbates these challenges. There has been limited investigation of these challenges and of approaches to addressing them. Thus, we undertook a review of the literature in this area, complemented by drawing on experimental data and experience from our own programme of research developing a suite of age-appropriate vision-specific PROMs of two types, one assessing vision-related quality of life (VQoL)19 and the other functional vision (FV)20 of children and young people with VI. Although the need for such measures is widely accepted, the recent rush to develop vision-specific, child-centred PROMs21 has not been fully informed by understanding of the characteristics and needs of visually impaired children as questionnaire respondents. We present here a synthesis of the literature with lessons learned from our research, so as to spark debate about the direction of travel for PROMs for childhood visual disability.

Need for vision-specific proms in paediatric ophthalmology

Childhood onset VI has significant impact on the developing person, with cumulative consequences for their social-emotional functioning, cognitive development, education, and future prospects.22, 23, 24, 25, 26 Most visually impairing disorders affecting children in developed countries are not currently treatable or preventable, so a substantial focus of paediatric ophthalmology is on support, visual habilitation and maintenance of vision of affected children, rather than restorative treatments.27, 28 A critical part of the ongoing support provided by paediatric ophthalmology services is understanding children’s own assessment of the impact of their visual impairment on their daily lives, measured routinely and over time, to complement objective clinical assessments (such as acuity). Using PROMs to do this child-led assessment would allow for detection of changes in quality of life, participation or functional status in individual children and variation across a population of children both as a function of personal circumstances (eg, educational transitions, adverse life events), as well as clinical care and interventions.

Until recently, there was a paucity of valid and reliable vision-specific PROMs to capture children and young people’s perspectives about their VI, and there were concerns that existing measures lacked in quality, as assessed by the ‘gold standard’ PROMs guidelines.29, 30 In 2013 we completed a systematic review of available child-appropriate PROMs for use in paediatric ophthalmology.21 In total, we identified 17 measures, 6 of which were suitable specifically for children and young people with visually impairing disorders as listed in Table 1 (for quality assessment refer to the existing review21). The rest are targeted to specific ophthalmic conditions. At the time of that review, based on a detailed quality assessment, we found that only a small number of instruments were in a sufficiently advanced stage psychometrically to be recommended for use in clinical care illustrating significant challenges of developing robust child-appropriate PROMs. These challenges, which likely explain the lack of appropriate measures, are discussed in more detail below.

Table 1 Child-appropriate PROMs for children and young people with visual impairment (VI) (regardless of the VI cause)a
Full size table

Challenges and opportunities

As highlighted above, the paucity of high quality PROMs for visually impaired children is not surprising. PROM development is a time-consuming, labour-intensive, iterative, and reflective process. It involves a set of methodologically rigorous and complex stages, each dependent on capturing information from the target patient population to ensure its relevance, reliability and validity. Conventionally, this starts with interviewing the target patient groups to determine the content, as well as comprehensibility and practical aspects of the questionnaire, followed by piloting and formal evaluation with representative samples. Adherence to these methodological principles and robust methodology is harder to achieve in development and application of paediatric PROMs because of the additional considerations and challenges relating to children.18, 31 In Table 2 we summarise the key recommendations and good practices extracted from the literature relating to developing and applying PROMs for children in general. Crucially, there are a number of specific additional considerations for children with visually impairing disorders, which we also present in Table 2 and further discuss below.

Table 2 A summary of recommendations for developing and applying patient-reported outcome measures (PROMs) for children, including specific recommendations for children with visual impairment
Full size table

Theoretical underpinning of PROM constructs

PROMs are used to assess a variety of different constructs (eg, HRQoL, functional status). A firm grasp of the theoretical underpinning of the construct to be measured is critical in order to make accurate inferences about outcomes.8 However, in the ophthalmic literature there is frequent conflation of the related, but distinct constructs of VQoL, FV and visual functions (typically acuity), which together describe the impact of impaired vision on an individual.21, 32 Importantly, in keeping with the established phenomenon of the ‘disability paradox’, which describes persons with severe illnesses or disabilities nevertheless experiencing and reporting good quality of life,33 poor eyesight does not necessarily mean extreme functional limitations or reduced emotional and social fulfilment, as assessed by the affected individual. This has important implications for how ‘the impact of visual disability’ should be viewed and measured by professionals. By conflating these constructs, for instance by using FV measures to assess VQoL in children with VI and vice versa, or by assuming poor QoL based on reduced visual functions in a child, clinicians risk inaccurate inferences about the effectiveness of treatment and interventions. A truly individualised and comprehensive assessment of the impact of VI, through complementary but not interchangeable PROMs, would capture self-reported FV and as VQoL as adjuncts to objective clinical assessments.

Developmental or age-appropriate PROMs?

Developmental issues must be considered so as to ensure PROMs appropriate for children of different ages as well as with different cognitive and communication abilities.18 Cognitive development determines children’s ability to engage in questionnaire development activities (eg, interviews and focus groups), as well as in the question-answer process involved in questionnaire completion.13, 14 For instance, the understanding of and vocabulary required for the concepts being assessed, the issues that children perceive as important (and which should inform the instrument content) and the format of the instrument (including the number and type of response options or time-frame used) all vary by age.14 Engagement with a PROM relies on language, reading skills and memory, all of which are still developing in early school years.13 Reliability and validity of children’s responses also improve with age.15, 18 Thus, understanding of typical developmental stages should inform age thresholds for age-appropriate questionnaire versions; however this is not straight forward because of the individual variations within age groups.18 In terms of visually impaired children, superimposed on this is the challenge that VI from birth or infancy results in delay in key developmental milestones23 and applying age-specific criteria to development of PROMs becomes even more difficult. Thus, we suggest that for children with VI developmentally-appropriate as opposed to age-appropriate questionnaires may be more apposite. Further research is required to delineate thresholds, comprising a combination of qualitative and quantitative techniques to capture relevant content with reliability and adequate sample size at the upper and lower bounds of the target age range to test developmentally appropriate cut offs.18

Flexible versus standard PROM formats

We propose that the notion of a ‘standard’ questionnaire format needs to be revisited. Questionnaires are by nature visual tools (even when presented electronically or large print), posing substantial challenges for usability by children with a range of levels of VI. For instance, recommendations concerning response choices are based on research with sighted children.14 Many questionnaires developed for sighted children also include pictures (eg, smiley faces), based on the assumption that pictures help children maintain interest and attention and clarify the response process,10, 18 but this would be difficult to implement for children with varying degrees of VI. Even with normally sighted children it is assumed that information presented visually will be integrated and facilitate the question-answer process. But visually impaired children have to work hard to manipulate questionnaire information presented either visually (for those without useful residual vision) or verbally before they can engage with the issues targeted by any given questionnaire, requiring additional memory and attention workload irrespective of cognitive ability. Thus, their true ability to report on complex issues, such as those relating to QoL, could be buried by a standardised questionnaire design and its associated administrative burden. Significant expertise is required to develop innovative and flexible child-centred approaches, with questionnaires individually adapted in size, format (eg, audio-assisted for more severe VI) or colour as required to facilitate self-reporting and ensure data quality. While this challenges the notion of ‘standard’ questionnaire methodology, we suggest this flexibility is the reality of developing and applying questionnaires that are suitable for the unique paediatric population of children with VI and align with the principles of 'personalised medicine'.

Self-reporting

The third issue we would highlight is the feasibility of self-reporting by children. The default position in the paediatric PROM literature is that even children between ages 5–7 years can reliably self-report without parents as proxies, but evidence about the nature and intricacies of self-reporting by children remains limited. Our experience in a study that involved a postal survey with around 100 children with VI aged 10–15 years is that almost half needed some parental help with questionnaire completion, including reading and scribing the answers as well as clarifying some questions; this was not confined only to younger participants or those with more severe impairment.19 We found that even some older visually impaired teenagers who are developmentally and cognitively well placed to self-report may rely on basic help with reading and scribing. However, the presence of an adult as ‘scribe’ or ‘interpreter’ in the process may be sufficient to influence responses (or lead to non-response13), especially where there is disclosure on sensitive topics relating to privacy and social life. Conversely, PROM completion in healthcare settings, where questionnaire administration is facilitated by a professional rather than parent, is not always feasible nor, where achieved, necessarily satisfactory as full privacy and sufficient time for reflection cannot be guaranteed. Researchers and clinicians working with visually impaired children, as well as other paediatric populations with complex needs, may need to move away from the narrow definition of self-reporting as applied to adults. To capture children’s views it may be necessary to embrace the reality of varying levels of self-report ability and respond flexibly to the need for help by a parent or professional, combined with capturing the information on whether and what help was needed and assessing its impact on the child’s responses. Our approach, for example, included providing appropriate instructions for the parents and supplying a complementary parent-specific instrument version to capture parents’ own perspectives of the same health outcome for their child, thereby positively harnessing their gate-keeping role and influence.

Epidemiological, statistical, and ethical considerations

Finally, the related issues of statistical challenges, ethical constraints and implications of unrepresentative samples for policy need to be considered. PROM development, especially psychometric validation, depends on large and representative sample sizes. However, researchers rarely report participation rates or address low or biased participation. Visually impaired children are a complex, heterogeneous and numerically small population who are also hard to reach.28, 34 For example, our participation rates in studies of VQoL were on average 30%, comparable (where reported) to those in other similar research,34, 35 but the potential impact in terms of both power and bias is disproportionately greater when the total population is smaller. We suggest that in studies of children with VI (and uncommon disorders in general) it may be necessary to rethink the balance between statistical significance levels and clinically significant findings. Related to this is the issue of biased participation relating to under-representation of certain groups such as ethnic minorities.34 Importantly, the cause of low participation rates may be children experiencing greater difficulties and thus being unwilling themselves (or their parents) to participate in studies that require disclosure of issues they find upsetting. This potentially impacts on how we capture and conceptualise ‘low’ HRQoL in children. Indeed, such bias in participation may also to an extent explain the trend for general skeweness towards better HRQoL in studies using generic measures.36, 37 We need to understand better what contributes to low and/or biased participation and to develop strategies to support families of affected children, especially from hard to reach groups, to take part. But there is a balance to be found and it is clearly unethical to persuade families to participate to ensure representative samples because there may be good reasons for declining. There is a need to explicitly recognise that bias exists in most studies and this can impact on equity if policy decisions are based on biased research.

Conclusions

There is a need for greater conceptual clarity and reflection, as well as increased pragmatism in development and application of PROMs intended for visually impaired children. Significant practical and methodological challenges in this field are widely recognised, but a ‘one size fits all’ methodological model currently prevails, driven by psychometric analytical trends. This does not align well with the complex paradigm of childhood VI and the values of ‘personalised medicine’. Since the gold standards of PROM development30, 38 were not developed with numerically small and heterogeneous populations of children with complex needs in mind, an imaginative reprofiling is required. Approaches would include formulating developmentally appropriate versus age-appropriate instruments with flexible formats and administration methods to facilitate self-reporting by children, as well as to ensure data quality. Equally, investment of time and resources would allow development of sensitive family-centred instructions and information sheets that capitalise on parents’ positive role in the research process and facilitate their understanding of the need to capture their child’s as well as their own unique perspectives. Finally, acceptance, awareness and explicit reporting of the existence of participation biases would help improve the relevance and scope of use of PROMs for policy.

Clinicians and academics need to engage in dialogue about these issues. As well as developing and applying robust age-appropriate vision-specific PROMs for children with VI, more research that focuses on visually impaired children as questionnaire respondents is also critical, but will require appropriate financial and infrastructure resources and multidisciplinary expertise. Understanding the unique nature and characteristics of their ability and needs as respondents will contribute to development of quality PROMS with meaning and traction in ‘real life’ clinical practice as well as research.

Members of the Child Vision PROM group

Contribution of members of the Child Vision PROM group (Phillippa Cumberland, Naomi Dale, Peng Tee Khaw, Gillian Lewando Hundt, Alki Liasis, Anthony Moore, Alison Salt, David Taylor) and the study research and advisory group (Corie Brown, Marianne Craig, Christine Ennals, Sarah Keeley, Jackie Osborne, Alexandra Robertson, Nidhi Sobti, Paula Thomas, Lucy Thompson and Jude Thompson).