Developing a short-form of the Genetic Counselling Outcome Scale: The Genomics Outcome Scale

https://doi.org/10.1016/j.ejmg.2018.11.015Get rights and content

Abstract

The Genetic Counselling Outcome Scale (GCOS-24) is a 24-item patient reported outcome measure for use in evaluations of genetic counselling and testing services. The aim of this study was to develop a short form of GCOS-24. The study comprised three phases. Phase I: Cognitive interviews were used to explore interpretability of GCOS-24 items and which GCOS-24 items were most valued by the target population. Phase II: The Graded Response Model was used to analyse an existing set of GCOS-24 responses (n = 395) to examine item discrimination. Phase III: Item Selection. Three principles guided the approach to item selection (i) Items with poor discriminative properties were not selected; (ii) To avoid redundancy, items capturing a similar outcome were not selected together; item information curves and cognitive interview findings were used to establish superior items. (iii) Rasch analysis was then used to determine the optimal scale. In Phase I, ten cognitive interviews were conducted with individuals affected by or at risk for a genetic condition, recruited from patient support groups. Analysis of interview transcripts identified twelve GCOS-24 items which were highly valued by participants. In Phase II, Graded Response Model item characteristic curves and item information curves were produced. In Phase III, findings from Phases I and II were used to select ten highly-valued items that perform well. Finally, items were iteratively removed and permutated to establish optimal fit statistics under the Rasch model. A six-item questionnaire with a five-point Likert Scale was produced (The Genomics Outcome Scale (GOS)). Correlation between GCOS-24 and GOS scores is high (r = 0.838 at 99% confidence), suggesting that GOS maintains the ability of GCOS-24 to capture empowerment, whilst providing a less burdensome scale for respondents. This study represents the first step in developing a preference-based measure which could be used in the evaluation of technologies and services used in genomic medicine.

Introduction

Genetic counselling and associated genomic testing services (hereafter shortened to ‘clinical genetics services’ (CGS)) have the potential to offer a number of benefits to individuals and families affected by conditions that may have a genetic aetiology. Recent studies have provided evidence that patients are seeking information and a supportive relationship, and that the benefits of genetic counselling include relief of uncertainty and feelings of vulnerability, as well as adaptation to the genetic condition in the family (Bernhardt et al., 2000; MacLeod et al., 2002; McAllister et al., 2008; Payne et al., 2007; Skirton, 2001). Robust and validated measures of these benefits are needed to provide evidence to service commissioners about the outcomes of investing in existing CGS or future service developments.

Evaluations of CGS have examined outcome variables such as knowledge, information recall, reproductive intentions, decisions made, anxiety or distress, patient satisfaction, perceived risk, perceived personal control, health behaviours, and decisional conflict (Payne et al., 2008; Madlensky et al., 2017). There is some evidence that genetic counselling can result in increased knowledge, perceived personal control, positive health behaviours, and accuracy of perceived risk amongst patients, and decreased anxiety, worry, and decisional conflict. There is also evidence that patients are typically very satisfied with genetic counselling (Madlensky et al., 2017).

Measures of process such as waiting times and numbers of patients seen have also been used, as well as the performance characteristics of genetic tests (e.g. sensitivity, specificity and predictive values) (Clarke et al., 1996; Payne et al., 2008). Little attention has been paid to exploring outcomes relevant to the population of individuals who use CGS (McAllister et al., 2008), and there have been calls for research to identify outcomes that are most important to patients (Madlensky et al., 2017).

Moreover, many of the measures which have been used to evaluate CGS have not undergone rigorous psychometric validation, with many having been assessed for internal consistency only, and few measures assessed for important characteristics such as reliability and responsiveness to change (Payne et al., 2008; McAllister and Dearing, 2015).

In 2011, the Genetic Counselling Outcome Scale (GCOS-24) (Fig. 1) was developed to provide an English language patient-reported outcome measure (PROM), specific to clinical genetics services (CGS) (McAllister, 2011b). GCOS-24 items are grounded in extensive qualitative research with CGS patients and providers, capturing an emergent theoretical construct labelled ‘empowerment’, comprising five sub-dimensions that summarise the outcomes valued by those stakeholders: cognitive, decisional and behavioural control, emotional regulation and hope (McAllister et al., 2008; McAllister, 2011a). ‘Empowerment’ was chosen as the construct name because it appeared to capture the ‘meaning’ across the five sub-dimensions. Despite ‘patient empowerment’ having gained considerable importance in healthcare policy globally, there is no universally accepted definition of the term. Whilst most definitions are consistent with the approaches and principles of patient-centred care, patient empowerment has been conceptualised in many different ways, including as an underpinning ethos (e.g. that patients have rights relating to autonomy, self-determination and power within their healthcare relationships), as empowering interventions (e.g. shared decision-making) and as an indicator (e.g. a patient ‘state’ ranging from low to high levels of the variable ‘empowerment’) (Bravo et al., 2015). A range of patient empowerment constructs have been operationalised in published measures of empowerment, including constructs that reflect patient states, patient experiences and capacities, patient actions and behaviours, patient self-determination and patient skills development (Barr et al., 2015). The ‘empowerment’ construct operationalised in the GCOS-24 is most consistent with a patient ‘state’.

GCOS-24 has been demonstrated to be valid, reliable and responsive, with no floor or ceiling effects observed (McAllister et al., 2011b), and has been used for service evaluation (Inglis et al., 2015; McAllister et al., 2016) and quality improvement (Costal-Tirado et al., 2017) in genetic counselling services. It has also received international attention, having been translated into Danish (Diness et al., 2017) and Spanish (Munoz-Caballo et al., 2018).

GCOS-24 has 24 items each with 7 response options (Fig. 1). GCOS-24 generates an overall ‘empowerment’ score, however it is not clear what interpretation can be attached to differences in score and between items. Further work is needed to attach ‘preference weights’ to the measure, reflecting the value or priority which is placed on each item by the target population (Sinnott et al., 2007). This will make it clear what interpretation can be attached to changes in score. In its current form, however, GCOS-24 produces a substantial number of possible response permutations (1.92 × 1020). A shorter version of the scale would make it possible to design a study to elicit such preference weights, thereby facilitating future use of the shorter scale in economic evaluations of genetic and genomic testing with and without genetic counselling.

Additionally, the wording of GCOS-24 items 1, 14 and 23, which refer specifically to CGS, means that the measure is unsuitable for use outside of CGS. Genetic testing is increasingly being performed outside the traditional models of service provision within CGS and is now moving into other specialities. This process is referred to as ‘mainstreaming genetic testing’ and is occurring, for example, in the context of cancer predisposition genes (Rahman, 2014), paediatrics (Valente et al., 2008), and neurogenetic testing (Lo et al., 2014). It is therefore becoming ever more important to have a valid and reliable PROM which can be used to evaluate genetic and genomic counselling and testing both within and outside of CGS. A further added benefit of a shorter measure would be to reduce completion time, which may also facilitate integration into clinical care.

In summary, a shorter version of GCOS-24 would be useful because (1) GCOS-24 is a thoroughly validated PROM for genetic counselling and testing services, since most other available CGS-specific PROMs have not been assessed for both reliability and responsiveness to change (2) genetic testing is increasingly being done outside the context of clinical genetics services, with no thoroughly validated PROM available (because GCOS-24 has items that refer specifically to clinical genetics services) and (3) most available CGS-specific measures have been developed for use in cancer genetics only, and are not suitable for general genetics services (4) there is no available PROM with attached preference weights that could be developed for use in economic evaluations of genetic counselling and testing services and (5) a shorter scale would reduce respondent burden and facilitate integration into clinical care.

Over recent years, the growing emphasis on patient-centred care has accelerated the demand for high-quality PROM data, leading to a rise in popularity for modern psychometric methods such as item response theory (IRT) (Alonso et al., 2013; Nguyen, 2014). IRT methods enable the creation of item banks for measuring specified health status domains, which in turn allows for item comparison and computerised adaptive testing (CAT) tools for tailored assessments without loss of scale precision or content validity (Bjorner et al., 2003; Cella et al., 2007; Haley et al., 2004; Harniss et al., 2007). The recognised value of IRT methods is demonstrated by the Patient-Reported Outcome Measurement Information System (PROMIS) initiative in the US, which aims to catalogue validated PROMs and build accessible item banks for measuring key health concepts applicable to a range of conditions.

This study aims to take the first step towards establishing a PROM which would be appropriate for routine use in audit and clinical evaluations of genetic services. The specific aim is to develop a short form of the GCOS-24 (using both qualitative and IRT methods), suitable for use both within and outside the context of CGS and in research, which still appropriately captures the empowerment construct.

For Phase I, participants were identified and recruited by Genetic Alliance UK (GAUK: https://www.geneticalliance.org.uk/), a national charity comprising over 180 support groups for genetic conditions. Phase II and Phase III used an existing dataset, comprising a set of responses to GCOS-24 (n = 395), collected in 2010 for the original psychometric validation (McAllister, 2011b). Specific details (e.g. gender, ethnicity, condition type, reason for referral) can be found in McAllister et al. (2011b).

Section snippets

Methods

There were three phases to this study. Phase I used qualitative cognitive interviews (Ericsson and Simon, 1980) to explore the relevance of the existing GCOS-24 items from the perspective of the target population. Phase II involved analysis of an existing data set of GCOS-24 responses (n = 395) using Samejima's Graded Response Model (GRM) (Samejima, 1969) to examine item discrimination. Phase III combined the results from Phases I & II to inform item selection, and employed the Rasch model to

Phase I: cognitive interviews

Of the 35 individuals contacted, ten (28.6%) replied and were successfully recruited to participate in think-aloud cognitive interviews. Participant characteristics are summarised in Table 1. For anonymity, participants are identified with the letter P followed by a number. Evidence confirming a diagnosis of a genetic condition was not sought, but all participants believed that their condition was genetic.

Table 2 summarises the items which were most valued by participants. For simplicity, items

Discussion

This study has developed a short-form (6-item) version of the Genetic Counselling Outcome Scale, potentially suitable for use in clinical audit and clinical evaluations of genetic counselling and testing services. The new scale, ‘The Genomics Outcome Scale’ or GOS, maintains the ability of GCOS-24 to capture the theoretical construct of empowerment (McAllister et al., 2011a), with the two scales showing a correlation of r = 0.838 at 99% confidence. Whilst the breadth of the latent trait

Acknowledgements

We extend our sincere thanks to all the patient support group members who contributed their time to participate in interviews for this study. We would also like to gratefully acknowledge the significant contribution of the Genetic Alliance UK, who recruited participants for the interview study. Their time, energy and commitment made this work possible. The copyright for GCOS-24 is owned by the Wiley publishing company, and we are grateful for their permission to reproduce GCOS-24 in this

References (40)

  • P. Bravo et al.

    Conceptualising patient empowerment: a mixed methods study

    BMC Health Serv. Res.

    (2015)
  • D. Cella et al.

    The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years

    Med. Care

    (2007 May)
  • A. Clarke et al.

    Outcomes and process in genetic counselling

    Clin. Genet.

    (1996)
  • A. Costal-Tirado et al.

    Using patient-reported outcome measures for quality improvement in clinical genetics: an exploratory study

    J. Genet. Counsel.

    (2017)
  • B. Diness et al.

    Translation and adaptation of the genetic counselling outcome scale (GCOS-24) for use in Denmark

    J. Genet. Counsel.

    (2017)
  • S. Embretson

    The new rules of measurement

    Psychol. Assess.

    (1996)
  • K. Ericsson et al.

    Verbal reports as data

    Psychol. Rev.

    (1980)
  • R. Hambleton et al.

    Comparison of classical test theory and item response theory and their applications to test development

    Instructional Topics in Educational Measurement

    (1993)
  • M. Harniss et al.

    Considerations for developing interfaces for collecting patient-reported outcomes that allow the inclusion of individuals with disabilities

    Med. Care

    (2007)
  • A. Inglis et al.

    Evaluating a unique, specialist psychiatric genetic counselling clinic: uptake and impact

    Clin. Genet.

    (2015)
  • Cited by (34)

    • The opinions and experiences of people with intellectual disability regarding genetic testing and genetic medicine: A systematic review

      2022, Genetics in Medicine
      Citation Excerpt :

      The positive effects reported ranged from access to improved medical care; anticipatory surveillance; educational support, which leads to improved sense of identity; better understanding of their own condition; connection with others; and removal of guilt.28 However, the potential impact genetic counselling may have on individuals with intellectual disability in terms of promoting informed choice regarding genetic tests; providing information and psychosocial support; and facilitating patient empowerment has not been systematically explored with a representative cohort of individuals with intellectual disability as they have been in the general population receiving genetic counselling.32 Indeed, although some participants reported that they positively identified with their diagnoses,24,33 this was not universal, and many commented on the discrimination and stigma they had been exposed to on account of their intellectual disability.24,26

    • The Clinician-reported Genetic testing Utility InDEx (C-GUIDE): Preliminary evidence of validity and reliability

      2022, Genetics in Medicine
      Citation Excerpt :

      Although a range of empiric strategies for measuring specific elements of clinical utility have emerged in the literature,6,10 including clinician-facing questionnaires,29-31 C-GUIDE represents a novel and standardized approach to quantifying this particular construct. Differentiating it from the range of constructs and strategies for measuring utility from a patient perspective,32-38 C-GUIDE represents a validated strategy for measuring utility from a clinician perspective. Our research showed that C-GUIDE measures the construct of clinical utility.

    View all citing articles on Scopus
    View full text