Health-related quality of life of informal carers in ALS: a systematic review of person reported outcome measures

An information specialist was consulted in developing a comprehensive search strategy across seven databases (Medical Literature Analysis and Retrieval System Online [MEDLINE], PsycINFO, Excerpta Medica Database [Embase], Cumulative Index to Nursing and Allied Health Literature [CINAHL], the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials [CENTRAL] and Google Scholar), utilising specific database combinations for systematic reviews [28]. Syntax was tailored per database and no restrictions were applied to publication date. This review involved two searches. Search 1, conducted on November 24, 2023, identified PROMs used to measure HRQoL (or an aspect of) in informal carers in ALS using related search terms. Search 2, conducted on April 4, 2024, sourced PROM development articles and literature on the measurement properties of the PROMs identified from Search 1, using related search terms and identified PROMs. Search terms in the current review included: (1) ALS (and derivatives); (2) a comprehensive list of informal carer terms; (3) a comprehensive search filter to identify questionnaires developed by the PROM Group at the University of Oxford [29]; (4) names of PROMs identified in Search 1; (5) a search filter developed by the COSMIN group for identifying articles reporting measurement properties of PROMs [30]. Additionally, as recommended by COSMIN methodology and consistent with other COSMIN reviews [31, 32]supplementary searches were conducted by screening the first 100 Google Scholar results for the names and acronyms of these PROMs. Finally, manual searches were conducted for PROM development articles. All searches were conducted by primary researcher (RB). Supplementary Material 1 details the review search strategy.

Article screening was conducted independently by two researchers (RB and JC) following predefined eligibility criteria (Table 1) using a hierarchical screening tool [33]. Search results were imported into EndNote 21 (Clarivate Analytics) to support a systematic, reproducible deduplication strategy [34]. Following deduplication, search results were transformed into Microsoft Excel (Microsoft Office, V.16.16.27) for title and abstract screening prior to full-text review. For title and abstract screening, one researcher (RB) reviewed all eligible articles, whilst a second researcher (JC) reviewed a random sample of 20% of titles and abstracts. Where disagreement occurred, this was resolved through retaining an article for full-text screening. For full text screening, two researchers (RB and JC) independently reviewed 100% of articles. Any discrepancy was resolved through discussion and reasons for exclusion were documented.

Multi-item, freely available, self-report PROMs or PROM subscales were eligible for inclusion if they measured a minimum of one component of HRQoL in adult informal carers of people living with ALS. Single-item PROMs, such as the EuroQoL Visual Analogue Scale [16]were not eligible for inclusion as psychometric standards require more than one item to permit rigorous evaluation [22, 23]. Copies of the HRQoL PROMs identified from Search 1 were independently screened by two researchers (RB and JC) to determine whether the PROM met predetermined eligibility criteria. This involved consideration of PROM content to determine whether aforementioned eligibility criteria was met. Where disagreement occurred, a third researcher (TS) ratified the inclusion decision. Reasons for PROM and PROM development article exclusion are listed in PRISMA diagram (Fig. 1).

Data extraction tools were developed according to COSMIN reporting guidelines [35] (see Supplementary Material 2). Data extraction was completed by one reviewer (RB). Methodological quality of included PROM development articles was independently assessed by two reviewers (RB and JC) with consensus reached via discussion. To rate methodological quality, each COSMIN standard (or item) was measured using a four-point scale from ‘Inadequate’, ‘Doubtful’, ‘Adequate’ to ‘Very Good’ [36]. Consistent with COSMIN methods [37]the final rating across COSMIN standards for each article was determined by the lowest rating assigned to any standard. For example, if any aspect of ‘PROM design’ was rated ‘Inadequate’, this yields an overall rating of ‘Inadequate’ despite presence of ‘Very Good’ ratings for other standards.

The assessment of content validity for each PROM involves evaluation of evidence from three sources: (1) the quality of the PROM development article; (2) the quality of PROM content validity articles; and (3) evaluation of PROM content by the research team. Relevance, comprehensiveness and comprehensibility ratings were made for each source of evidence independently by two researchers (RB and JC) with consensus reached via discussion. Individual ratings for content validity, and its constituent components of relevance, comprehensibility and comprehensiveness, were qualitatively synthesised using COSMIN rating synthesis rules [22] (Sheet 7, Supplementary Material 3). Using these rules, each PROM could receive an overall synthesised rating of ‘Sufficient’ (+), ‘Inconsistent’ (±) or ‘Insufficient’ (-). For example, if the PROM development article was rated ‘Insufficient’ (-) and the researcher rating was ‘Inconsistent’ (±) for comprehensibility, the overall synthesised comprehensibility rating would be ‘Insufficient’ (-). In the first instance, COSMIN rating synthesis rules [22] were used to combine scores for relevance, comprehensiveness and comprehensibility. When aforementioned synthesis rules could not be applied to scores, rating synthesis rules from previous COSMIN reviews [31, 38] were utilised (Supplementary Material 3).

Quality of evidence was independently rated by two researchers (RB and JC) using the COSMIN-modified GRADE approach [23] and rated as ‘High’, ‘Moderate’, ‘Low’ or ‘Very Low’. Evidence was initially rated as high quality, then downgraded according to four components: (1) risk of bias, (2) inconsistency, (3) imprecision and (4) indirectness [22]. ‘Low’ quality rating equates to high risk of bias, whilst a rating of ‘High’ equates to low risk of bias. Risk of bias, content validity and certainty assessments were considered when formulating recommendations for which PROM or PROM subscale, if any, were best suited to assessing HRQoL of informal carers of people living with ALS, considering current available evidence.

Search 1 generated 5198 records (Fig. 1). After duplicates were removed, 3518 records were screened via title and abstract. A total of 260 records were assessed for eligibility via full-text. 184 were rejected and 76 articles were included in this review. In Search 2, 2786 records were identified, from which 379 duplicates were removed and 2407 were screened via title and abstract. One hundred records were assessed for eligibility with nine articles eligible for inclusion. Cohen’s kappa of inter-rater reliability for full-text review for Search 1 and 2 was κ = 0.65, interpreted as ‘substantial agreement’ [39]. Additionally, 4292 records were screened from other sources (i.e., Google Scholar searches and manual searching for development articles). From these sources, 73 records were eligible for full text review, and 43 development articles were ultimately included. No new articles were found from Google Scholar that were not already identified in database searches. Overall, of the 12,276 records reviewed, 12,148 were rejected and 128 were accepted for inclusion in this review (85 articles providing evidence of measurement properties and 43 PROM development articles). A complete reference list of included articles is available in Supplementary Material 4.

From the 76 eligible full-texts from Search 1, 109 distinct PROMs were used with adult informal carers in ALS and 43 were eligible for inclusion (Table 2). Supplementary Material 5 contains the full list of 109 PROMs with reasons for exclusion. The number of PROMs used per article ranged from 1 to 11 with a median of 2 (IQR = 2–3). Individual PROMs were used in a total of 1–30 articles, with a median of 1 (IQR = 1–2). Of the included PROMs, the Zarit Burden Interview (ZBI) was used across the highest number of articles (n = 30), followed by the Hospital Anxiety and Depression Scale (HADS) (n = 24), Carer Burden Inventory (CBI) and Carer Strain Index (CSI) (n = 10). Supplementary Material 6 details full information on the frequency of PROM use in the included articles.

Ratings for PROM development articles for ten PROMs were extracted from a prior review [38] (Beck Depression Inventory [BDI], Caregiver Strain Index [CSI], Carer Quality of Life [CarerQoL], EQ-5D-5 L, Hospital Anxiety and Depression Scale [HADS], Short Form-12 [SF-12], Short Form-36 [SF-36], State Trait Anxiety Inventory-X [STAI-X], World Health Organisation Quality of Life-BREF [WHOQOL-BREF] and Zarit Burden Interview [ZBI]). Two development articles were used to assess four PROMs (Supplementary Material 4). PROM development articles produced ‘Inadequate’ ratings for all but three PROMs: the Close Persons Questionnaire (CPQ), EQ-5D-5L and Quality of Life at the End of Life–Family Carer Version (QUAL-E-Fam). These PROMs were rated as ‘Doubtful’ and surpassed an ‘Inadequate’ rating due to the presence of cognitive interview methods within PROM development and simultaneously were not scored down for other factors within PROM design or development.

No articles reporting on the content validity of HRQoL PROMs used with informal carers in ALS were identified. Therefore, the assessment of content validity was conducted by combining evidence from the PROM development papers and reviewer ratings, consistent with COSMIN methods for synthesising ratings [22] (Table 3). Overall ratings for relevance, comprehensiveness and comprehensibility were combined to produce ‘Inconsistent’ or ‘Insufficient’ overall content validity ratings for all 43 PROMs. Supplementary Material 3 contains COSMIN rating sheets. The CarerQoL [18] was the only PROM to achieve an ‘Inconsistent’ overall rating for comprehensiveness within this review.

The 11 PROMs receiving an ‘Insufficient’ rating for overall content validity were all rated ‘Inadequate’ for their respective development study. Those PROMs with an ‘Inconsistent’ rating either had better ratings for their respective development study or were rated more favourably in reviewer ratings (Supplementary Material 3). PROMs with highest overall content validity ratings and highest frequency of use are shown in Fig. 2.

Quality of evidence supporting content validity ratings was ‘Very Low’ for all PROMs with the exception of the CPQ, EQ-5D-5L and QUAL-E (Fam). Quality assessment scores are shown in Table 3. Consistent with the COSMIN-modified GRADE approach [22]all PROMs began with a ‘Moderate’ rating due to the lack of content validity articles with ALS carers. This baseline rating was adjusted based on the quality of evidence in PROM development articles. Quality ratings could have been universally downgraded to ‘Very Low’ due to ‘Indirectness’, as all PROMs within this review, with the exception of the CNS, were not developed with informal carers or people with ALS. PROMs were not downgraded further to ensure quality assessment could distinguish between PROMs based on the varying quality of their development articles.