INTRODUCTION

Patients with rare diseases and their caregivers often have limited evidence-based information to guide decisions about management and symptom relief.1 Patients are increasingly interested in and calling for a more proactive role as partners in clinical research.2–4 Engaging patients and other relevant healthcare stakeholders (e.g., patients, caregivers such as parents of children with rare diseases, advocacy organizations, and clinicians)5 in the planning and conduct of research (as distinct from increased patient involvement in clinical care6) is a promising approach for addressing evidence gaps for the management of rare diseases. Engagement may promote research that evaluates health outcomes that are both relevant to patients with rare diseases7 and useful for decision making.8–12 Engagement is feasible for many populations and research phases,13 and can involve patients as consultants, as collaborators, or as leaders.13, 14 Further, given their interest in advancing research, rare disease organizations15 could help facilitate patient engagement in clinical research beyond efforts by the research community.

We undertook a systematic literature review to assess the extent and nature of involvement of patients with rare diseases, their caregivers, and relevant organizations in research initiatives. Five key questions guided the review:

  1. 1)

    For what purposes (goals and stages of the research process) have patients and other relevant health care stakeholders been engaged in research on rare diseases?

  2. 2)

    In what ways have they been engaged? How were patients and other stakeholders identified and recruited for engagement? What approaches were used to obtain their input?

  3. 3)

    How does engagement affect the design, conduct, dissemination, and relevance of the research to patients with rare diseases?

  4. 4)

    What is the role of patient organizations in bringing researchers and patients together?

  5. 5)

    What are the challenges for engagement in research on rare diseases?

METHODS

Stakeholder Involvement

The review was guided by a multi-stakeholder technical expert panel (TEP) composed of patient representatives, clinicians, and researchers recruited through the informal professional networks of the authors. The TEP provided input on the key research questions, the definition of rare disease, the search strategy, key gray literature references, interpretation of study findings, and recommendations for future research.

Electronic Database Search

Eligibility Criteria

We included original studies of any design published in English, in which patients, caregivers, or other stakeholders participated in planning or conducting biomedical or health services research related to rare diseases. Studies available only in abstract form were excluded due to insufficient information. Studies were considered to be related to rare diseases if the focus was a condition considered to be rare according to lists maintained by the National Institutes of Health16 or the National Organization of Rare Disorders,15 or if the authors stated that their research focused on a rare disease. Research engagement was defined by active involvement in the planning or conduct of research (e.g., providing input on a research agenda, contributing to study design, identifying study comparators or outcomes, monitoring study progress, interpreting results, or disseminating findings).17 Enrolling patients only as passive participants (subjects) or actively involving them in clinical care were not considered research engagement. Non-original or summarized literature was used to identify additional studies.

Search strategy

The overall search strategy was developed by a reference librarian and methodologists with expertise in conducting systematic reviews to capture both clinical studies on rare disease(s) that engaged patients or other stakeholders and studies that evaluated engagement. This review covers the origin of the biomedical data bases through September 2013, searching: PubMed/Ovid MEDLINE, Ovid EMBASE; Ovid PsycInfo, Ovid Cochrane (Sys Rev, Methods, HTA), EBSCO, CINAHL, SCOPUS (social sciences content), Web of Science (multidisciplinary scientific content), and Business Search Premier, Academic Search Premier and Google Scholar. To identify studies relevant to research engagement, medical subject heading (MeSH) terms and text words were selected based on common indexing practices. We included, among others, terms related to patients and other stakeholders (e.g., “patient*,” consumer*, stakeholder*, etc.), engagement (e.g., “participa*,” “collaborat*,”, “partner*,” etc.) (See online Appendix for full search strategy). These engagement research search terms were compiled and tested repeatedly to enhance search sensitivity and identify all potentially relevant publications. Due to the lack of indexed search terms for rare diseases, as well as the large number and heterogeneity of rare diseases, the relevant literature on research engagement was hand-screened by the authors for relevance. For studies published prior to 2011, we capitalized on a previous systematic review by three of the investigators (HM, ZW, TE) concerning research engagement broadly, although not specific to rare diseases,9 that utilized the same database search strategy with respect to engagement in research. The reference list for this original review was screened to identify studies related to rare diseases. We also screened reference lists from eligible studies, used the PubMed ‘related articles’ feature to identify eligible studies, reviewed conference proceedings, and used SciSearch for publications that cited potentially eligible studies.

Gray Literature Search

To identify relevant resources not included in bibliographic databases, we used search engines Scirus and Sciverse (which contain scientific journal content, scientists’ homepages, courseware, preprint server material, patents, and institutional repository and Website information), Google, and Bing, in addition to other websites recommended by the TEP.

Study Selection

We organized initial references in citation files using Endnote software and removed duplicates. Titles and abstracts were screened independently by two reviewers against eligibility criteria using the DistillerSR software (Evidence Partners Incorporated, Ontario, Canada). All studies judged relevant by either reviewer were included for full text review. We conducted full text screening of potentially eligible studies in duplicate using a similar procedure. If both reviewers judged a study to be relevant, it was included. Disagreements in full text screening were reconciled through arbitration by a third reviewer (MH).

Data Extraction

Data were extracted from studies with a standardized form developed using DistillerSR. Data were extracted by one investigator and reviewed for accuracy by a second. The data extraction form was tested on a sample of included studies (n = 5) to assure consistent data extraction. We extracted the following data: study description, stage(s) of research in which there was engagement, purpose of research engagement, methods for identifying and recruiting patients/stakeholders for research engagement and for obtaining their input, roles of rare disease organizations, perceived challenges to engagement, and any reported outcomes of patient engagement.

Analysis

The included studies were assessed by two reviewers (with disagreements arbitrated by a third reviewer) regarding the extent to which information answering the key research questions was provided. Studies were classified based on whether or not engagement activities were described with sufficient detail and clarity for others to replicate the approach (‘sufficiently descriptive’ vs. ‘minimally descriptive,’ respectively). Studies were also classified based on whether or not there was a direct method for measuring and examining the effect of the engagement processes on the design and conduct of the study (evaluative vs. non-evaluative studies). Findings were synthesized by key research question.

RESULTS

The systematic review and gray literature search identified 35 relevant studies (Figure 1). All studies were observational; none employed experimental or quasi-experimental designs. Eleven studies used qualitative or survey methods to obtain input from patients,4, 7, 18–26 and 17 studies were narrative reports describing one group’s experiences with engagement.27–43 The remaining seven studies were descriptions of specific initiatives reported on websites44–50 (Table 1 and Table 2). The studies were mostly conducted in North America and Europe, and most focused on a specific rare disease rather than rare diseases in general.7, 18–21, 23–31, 35–38, 40–42, 47–50 Thirty studies reported on engagement conducted by the author(s),7, 18–21, 23–33, 35–42, 45–50 three described engagement conducted by others based on surveys or key informant interviews,4, 22, 43 and two broadly described contributions of patient organizations.34, 44 Nineteen reported on engagement of patients,7, 18, 20, 21, 23, 25–27, 32, 33, 35, 38, 39, 43, 45, 47–50 18 reported on engaging patient organizations,4, 19, 22, 27–33, 35, 37–39, 41–43, 46 13 reported engaging parents or other caregivers,19, 23, 24, 26, 32, 35, 36, 38–40, 43, 47, 50 and five reported engaging clinicians.20, 38, 39, 47, 49 Seven studies were classified as sufficiently descriptive7, 18, 19, 23–26 (see Box 1 for example of a sufficiently descriptive study). No studies were classified as evaluative, because none formally assessed outcomes related to engagement.

Figure 1
figure 1

Study screening and selection.

Full size image
Text Box 1.
figure 2

Illustrative Example of a Study that Was Sufficiently Descriptive with Respect to Research Engagement

Full size image
Table 1 Characteristics of Included Studies
Full size table
Table 2 Detailed Characteristics of Individual Studies
Full size table
  1. Key Question 1:

    For what purposes (goals, stages of the research process) have patients and other stakeholders been engaged in research on rare diseases?

The purposes of engagement included identifying patient-centered research topics or agendas,19, 23, 29, 36, 39 identifying outcomes important to patients or developing measures relevant to patients’ needs,7, 19, 20, 23, 25, 26 increasing recruitment or enrollment through development of patient-centered study designs,4, 18, 21, 24, 27, 30, 31, 41 and incorporating the patient perspective into study design19, 25, 35, 39, 40 (Table 2, Table 3)

Table 3 Engagement Summary: Stage, Purpose, Method, and Role of Rare Disease Organizations
Full size table

. Patients and other stakeholders were most commonly engaged in the preparatory stage (n = 19 for agenda setting)4, 7, 19, 20, 23, 25, 28, 29, 32, 35, 36, 39–43, 45, 49, 50 and study execution (n = 15 for study design and procedures,4, 18, 19, 21, 24, 26, 28, 29, 33, 35, 38, 42, 47, 50, 51 n = 12 for recruitment,4, 21, 27, 29–31, 35, 41–43, 47, 50 and n = 6 for data collection4, 21, 37, 42, 43, 50). Eleven studies reported engagement for research translation, with a greater focus on dissemination (n = 10)4, 32, 33, 35, 37, 40–43, 49 than on evaluation of findings (n = 3)21, 32, 49 or implementation (n = 0). Nineteen studies4, 19, 21, 22, 28–30, 32–35, 37, 40–44, 49, 50 described engagement at multiple stages, particularly those reporting on the role of patient organizations.33, 35, 41–43

  1. Key Question 2:

    In what ways have patients and other stakeholders been engaged in research on rare diseases?

Identifying Patients and Other Stakeholders for Engagement in Research

Eight studies reported information on how patients and other stakeholders were identified for engagement18, 19, 23–25, 27, 35, 48 (Table 4 [online]). All of these reported convenience sampling without an explicit selection process. Recruitment occurred through patient organizations,19, 23, 27, 35 clinics,18 self-referral on the internet,48 or a recruiting agency.24 No studies engaging patient organizations described how collaborations between research teams and these organizations were initiated.

Approaches for Patient and Other Stakeholder Input

Eleven studies referred to engagement activities based on consultation with patients and stakeholders. Obtaining input through workshops, focus groups, interviews, or Delphi methods was common.7, 18–20, 23–26, 36, 39, 45 One study obtained patient feedback on informational materials that would be presented to study subjects.19 Thirteen studies described more collaborative involvement in at least one stage of the research process; for example, including patients on governing or advisory committees,32, 33, 38, 42, 48 in developing study interview guides, study materials and websites;21, 35, 52 in collecting data (conducting interviews),4, 21 in reviewing research findings,21, 28 and in disseminating findings.4, 33, 37, 40–42 Stakeholder-driven approaches were reported; included here are examples of patients or advocacy organizations initiating new studies4, 29, 37, 41, 42, 46 and controlling the research process by setting the research agenda, having responsibility for data collection, and disseminating findings (e.g., Patient-Powered Registries43). Conversely, in four studies, patient organizations were engaged only for providing access to their members.27, 30, 31, 46 Four studies reported that patients or other stakeholders were prepared for engagement in research, ranging from an informational video24 to in-person workshops;21, 40, 48 none reported training researchers in engagement methods.

  1. Key Question 3:

    How does engagement affect the design, conduct, dissemination, and relevance of the research?

Four studies clearly articulated how they used input from patient engagement to inform their work, including identifying research topics for future study,23 the study design to use,19 the outcomes to assess,7, 19 and the domains to include on a patient-reported outcome measure26 (Table 3, Table 5 [online]). Five studies used either qualitative or survey methods to obtain information directly from patients about the outcomes that were important to them.7, 19, 23, 25, 26 These reports permitted rankings of symptoms by importance7, 25 and also clarified how patients describe symptoms.7, 19, 23 For example, due to patient input, these studies included measures of pain, stiffness, sleep, motor function, quality of life, and the economic impact of disease.

No studies empirically evaluated the effect of engagement on the design, conduct, dissemination, or relevance of the research. However, many studies reported perceived impact of engagement that was not measured and could not be confirmed. Some perceived benefits relative to other studies that did not use engagement were practical, such as improved access to patients or improved recruitment methods,4, 19, 29, 30, 35, 42, 43 improved study retention,19 more in-depth responses in interviews (when using patients to conduct the interviews),21 less time to complete the project,4 and incorporation of a broader range of outcome measures to inform future risk-benefit assessments for investigational treatments.49 Other perceived benefits were less tangible, such as increased relevance of the project,4, 29, 35 broader dissemination of findings,35 improved acceptance of research findings among end-users,29 greater mutual respect between scientists and patients,32 and increased transparency of and public confidence in the research.32, 33 Only one study articulated the benefits of engagement for the engaged partners, including increased awareness of the disease studied and greater access to health information.35

  1. Key Question 4:

    What is the role of rare disease organizations in bringing researchers and patients together?

Nineteen studies reported on engagement of patient organizations.19, 22, 27–33, 35, 37–39, 41–43, 46 However, no studies formally evaluated the role of patient organizations in connecting researchers and patients. Studies frequently acknowledged the role of rare disease organizations in providing access to or recruiting patients both for engagement and as study subjects,19, 22, 29–31, 35, 41 and in enhancing communication about research participation opportunities and research findings (e.g., through social media, websites, conferences, and newsletters).31, 35, 36, 41 In some cases, representatives from these organizations were directly involved in the research process, serving on advisory and regulatory committees28, 42 or providing input on study protocols,19, 22, 35 for example. Some studies reported that patient organizations initiated clinical trials or other studies of rare diseases.28, 29, 41

Several studies also highlighted broader support provided by rare disease organizations. For example, these groups initiated the organization of collaborative research networks,22, 28, 41 provided financial support for research infrastructure such as registries,22, 30, 43, 46 supported training activities,22, 34, 44 and developed policy statements regarding engaging patients in research.34

  1. Key Question 5:

    What are the challenges for engagement in research on rare diseases?

The studies included in this review provide insights into the process of research engagement. First, four of the studies stated that engagement requires substantial time and resources from both researchers and those who are engaged as partners.26, 29, 32, 35 One author notes that scientific groups face challenges in finding time to obtain and use patient input,32 and another author notes that patient organizations have limited resources to devote to research engagement versus other priority activities.35 Second, two studies stated that engaged partners must overcome barriers to engagement such as logistical issues (e.g., access to clinics) and health problems that may prevent participation.18, 31 Third, several authors expressed concerns about possible bias among patients and patient representatives chosen for engagement due to higher motivation, high educational levels, methods of recruitment (e.g., via internet or annual meetings) or possible conflicts of interest (e.g., relationships between patient organizations and industry groups).23, 25, 27, 30, 33 , 43 Other challenges identified include lack of researcher and patient experience in or training for engagement,25 , 32 an absence of policies facilitating engagement,25 the potential for both researchers and patient partners to undervalue engagement,25 ethical concerns (e.g., patient rights43), and the possibility of uncoordinated efforts by competing patient-led groups.43

DISCUSSION

Given the diversity of conditions and limited research resources associated with rare diseases, engagement of patients and other stakeholders in research has the potential to facilitate the study of relevant clinical questions and patient-centered health outcomes for rare disease populations. In fact, these issues were among the most commonly reported reasons for engaging patients or their caregivers. In contrast, the studies generally did not address the role of engagement for dissemination of research findings, even though engagement of patients and patient organizations in sharing research results may increase the reach of findings, which is particularly important in the context of treating rare diseases.

Most studies reported a consultative approach to engagement (i.e., unidirectional communication from patients to researchers with unknown influence on the research). We expect that collaborative approaches (i.e., bidirectional communication, shared responsibility for decision-making) may be more effective for advancing the study of rare disease, given the diversity of patient experiences. Although other researchers have reported similar expectations based on their experiences with research engagement in other settings (not specific to rare diseases), definitive evidence to support this is not yet available.9 , 53 Very few studies reported training researchers, patients, or other stakeholders for engagement. Other literature supports training as increasing the impact of engagement; while some training opportunities are available for patients43 and researchers,54 these are not specific to research on rare diseases.

This systematic review suggests that rare disease organizations play multiple roles in facilitating engagement of patients and other stakeholders in research. Nearly half of the identified studies involved patient organizations; these organizations met critical needs, including facilitating access to patients who may be too geographically dispersed or difficult to identify for participation in clinical trials. Patient organizations also provided broad support for patient-centered research in rare diseases, including wide communication about research opportunities and findings, financial support, formation of collaborative research networks, and creation of policies regarding engagement.

Many studies reported the effects of engagement, but none measured these effects in ways that permit estimation of the strength of the impact of the engagement efforts. The purported benefits included both practical effects (e.g., incorporation of a broader range of outcome measures, increased study enrollment) and intangible effects (e.g., increased relevance of the research, greater public trust). However, consistent with gaps in the broader literature on engagement overall,51 no studies evaluated the effect of engagement on the inclusion of patient-centered outcomes, the impact of the research findings, or their relevance for clinical decision-making. This lack of empirical evidence precludes drawing conclusions about the impact of engagement in research on rare diseases. This gap in the literature can be addressed in future studies that directly measure the impact of engagement on health outcomes.

This review identified additional evidence gaps, including the lack of standard reporting guidelines for information about engagement of patients and other stakeholders.13 In addition, very little is known about the characteristics of those engaged, the approaches for collaboration with these individuals or groups, or the impact of these characteristics on the research process or outcomes. Connecting researchers with engaged partners is a challenge. Although researchers can learn from successful engagement approaches for the study of more common conditions (e.g., HIV),55, 56 more information about how to identify and recruit patient and stakeholder partners would allow for both shared learning and greater transparency regarding possible biases.

Several initiatives may help to fill the evidence gaps. First, the Patient-Centered Outcomes Research Institute (PCORI)57 funds studies that address methods for improving engagement, for evaluating the impact of engagement on research outcomes, and for assuring that study questions and outcomes are meaningful to patients.58 Evidence regarding best practices for research engagement will also be generated by the Agency for Healthcare Research and Quality’s Community Forum project.59 Two initiatives will support infrastructure for patient-centered approaches to rare disease research, including PCORI’s National Patient-Centered Clinical Research Network60 and the National Institutes of Health’s Global Rare Diseases Patient Registry Data Repository.9 PCORI’s Advisory Panel on Rare Diseases may also provide guidance regarding promising approaches for fostering collaborative approaches to research and involving relevant patient organizations.61.

This review was limited by the small number of peer-reviewed studies that reported on research engagement in the study of rare diseases and the lack of detail about engagement. The findings from this review could be supplemented by interviews and/or surveys with a range of rare disease organizations to ensure comprehensive capture of research engagement activities. Our assessment of the effects of engagement relied heavily on the perceptions of the researchers who conducted the included studies, which have not been empirically validated. Moreover, this review could not fully capture the wide variation in research resources and infrastructure or the engagement capacity of patient groups across the spectrum of rare diseases. The studies included in this review provide limited insight into whether the successful approaches are generalizable to other rare disease organizations. Future research should examine differences in approaches to, and the impacts of, engagement for diverse types of stakeholders (e.g., children with rare diseases compared to their parents) and assess the authenticity (i.e., the extent to which patient and stakeholder input was actually incorporated into decision making) of approaches to engagement, as the impacts of engagement may depend on the quality and genuineness (i.e., the sincerity) of engagement interactions.

This systematic review found that patient engagement for guiding research on rare diseases tended to use a limited number of multi-purpose approaches. The published studies provide examples of approaches that could be adapted and further evaluated by other organizations, and also highlight the particularly important role of patient organizations in fostering engagement in research on rare diseases. While no insurmountable barriers were identified in this review, there were meaningful challenges. Building an evidence base that documents how different approaches to patient engagement facilitate or hinder research will inform guidance on how best to make clinical research on rare diseases more useful.