FormalPara Key Points

There are a number of approaches related to planning, selection, and stakeholder engagement when integrating patient-reported outcomes (PROs) into electronic health records (EHRs), each with advantages and disadvantages.

There is no single, most effective model for PRO-EHR integration that will work in all contexts.

Health systems should assemble the multidisciplinary expertise needed to weigh the advantages and disadvantages of each approach for PRO-EHR integration in their particular context.

1 Introduction

The growing measurement of patient-reported outcomes (PROs) and adoption of electronic health records (EHRs) presents an unprecedented opportunity to improve health care for patients and populations. PROs are direct reports from patients about their symptoms, functioning, health-related quality of life, or other aspects of their health that are captured without clinician interpretation using standardized PRO measures (i.e., instruments, questionnaires, surveys) [1,2,3]. EHRs are longitudinal electronic records of patient encounters and clinical data that patients, providers, and payers can access across facilities and devices [4]. The integration of PROs into EHRs can promote patient-centered care and advance quality improvement initiatives, research, and population health [5,6,7,8,9].

Despite the potential benefits of integrating PROs into EHRs, there are few best practices to help organizations achieve integration [10,11,12]. In collaboration with the International Society for Quality of Life Research (ISOQOL) and through a contract from the Patient-Centered Outcomes Research Institute (PCORI), Johns Hopkins University led a multidisciplinary team to create the Users’ Guide to Integrating Patient-Reported Outcomes in Electronic Health Records (“Users’ Guide”), which is available on the PCORI website (http://www.pcori.org/document/users-guide-integrating-patient-reported-outcomes-electronic-health-records). The purpose of the Users’ Guide is to provide a practical framework and examples for administrators, clinicians, researchers, information technology (IT) professionals, and others wishing to integrate PROs into their EHRs. It offers guidance on 11 key questions related to integration, and for each question, describes advantages and disadvantages of different options, identifies research gaps, and lists references and resources. Importantly, many of the options presented in the Users’ Guide are not mutually exclusive, and it is often possible and advisable to pursue more than one approach.

The Users’ Guide was developed between August 2015 and May 2017. In phase 1 of the project, a steering group was assembled to help clarify goals, develop strategies, and identify topics to be addressed. The steering group, PCORI, ISOQOL, and others circulated the topic list for public comment. In phase 2, working group members were identified and formed into writing teams, each responsible for creating one of the 11 Users’ Guide sections addressing a key question about PRO-EHR integration. There was an in-person meeting to discuss outlines for each section, and writing teams then developed drafts. The drafts were presented during a series of conference calls with the combined steering and working groups (see “Acknowledgements”). The final sections were circulated for public comment, with section authors encouraged to give due consideration to the feedback received and to revise as they deemed appropriate. The Users’ Guide was released in time for a May 2017 PCORI-hosted public meeting in Arlington, VA.

In this editorial, we present the 11 key questions addressed in the Users’ Guide in three thematic groups: Planning, Selection, and Engagement. For each theme, we describe the key questions and their important considerations and takeaways. Finally, we offer recommended next steps for organizations wishing to pursue PRO integration into an EHR.

2 Overview of Guidance for Integration

2.1 Planning

When planning for the integration of PROs into EHRs, it is important to consider what strategy will be used, how the integrated system will be governed, ethical and legal issues, and how data from multiple EHRs can be pooled across organizations. Each consideration is described below, with takeaways summarized in Table 1.

Table 1 Main takeaways for integration themes
Full size table

The primary planning consideration involves determining how the PRO data will be integrated into the EHR. The options range from “full” integration, where PROs are collected from the patient and reported directly within the EHR, to “minimal” integration, where staff or clinicians manually push collected PRO data into the EHR (e.g., paper scanning, clinical note entry), with many options in between. Full integration allows patients, providers, researchers and administrators to access and act upon PRO and clinical data side by side, but manipulating elements already incorporated into the system (e.g., questionnaire language, timing of administration) may require a local EHR IT team or other external vendor, potentially constraining the customization of PROs and necessitating the prioritization of clinical needs over research potential. Minimal integration is likely to cost less up front and may allow for greater customization of PRO collection and research independence from the clinical care system, but it can lead to redundancy in data collection as researchers, clinicians, and others gather the same data in their preferred format, and produce data that are less structured and less accessible in the EHR.

Governance involves determining who will make decisions about the selection, implementation, analysis, data sharing, and use of PROs in EHRs. Key questions include the degree of centralization of governance and stakeholder diversity. A centralized governance model grants decision-making power to a single entity, whereas a distributed model allows departments or even individuals to make local decisions. A centralized model facilitates system-wide initiatives, and makes regulatory oversight easier, but it can inhibit local innovation and slow deployment of condition-specific and tailored PRO measures, due to bureaucracy. A distributed model offers greater local and PRO measure flexibility, but loses the centralized system’s coordination and oversight functions. With any model, the diversity of stakeholder involvement is the next important governance decision. One option is for the entity that currently governs the EHR to also take responsibility for decisions about PROs. A broader option is to include additional stakeholders such as patients and their advocates, quality improvement officers, researchers, IT specialists, and senior administrator champions. This latter option can add complexity to decision-making, but it also supports patient-centered care and recognizes the spectrum of perspectives involved in PRO collection, analysis, and use.

Ethical and legal issues may arise around the collection and re-use of PRO data in EHRs. The intended use of the PRO data—patient care, research, or population surveillance—affects decisions around protection of patient information. Depending on state and organizational legal and ethical requirements, organizations might choose to provide PRO measures to patients with no explanation of their purpose, which is expedient, but will reduce patient buy-in (and therefore participation) and raise ethical questions. On the other end of the spectrum is to follow the processes around informed consent for human subject research, which might be most respectful to patients, but would differ from usual patient care and is likely to decrease participation and therefore risk limiting the generalizability of the data that are collected. There are many options in between these extremes (e.g., a PRO-specific information sheet), so organizations should seek ethical and legal advice to determine the most appropriate option for their context.

Although not necessary for EHR integration, it is prudent to consider early on how data from different organizations’ PRO-EHR systems might be pooled in support of population health, quality improvement, or research efforts. Planning for pooling involves identifying a data warehouse to store the shared data and a data model to outline the data to be shared. Data warehouses can be centralized, where all data from each local EHR are stored in one location, or distributed, where data are stored locally and only summary data are shared with a central governing body. The centralized option is technically easier and facilitates cross-group data analysis, but safeguards to protect patient-identifiable information are critical when data are shared both locally and centrally. In addition, health care organizations may have concerns regarding how the data might be used (e.g., for competitive advantage). The distributed option has less risk of breaching identifiable information, but it is technically complex and impedes secondary data analysis across groups. Either option entails the additional challenge of requiring local sites to agree to structured data formats; thus identifying a data model is also essential for successful pooling. Attributes of data models include the feasibility of patient-level analyses, de-identification capabilities, and types of clinical domains, among many others. There are many potential models, but some options include PCORnet v3.0, Consolidated Clinical Document Architecture (CCDA) R2, Shared Health Research Informatics Network (SHRINE), and a project-specific ad hoc model [13,14,15,16]. Each model has its advantages and disadvantages in terms of requirements and suitability for PRO data.

2.2 Selection

To select PROs for EHR integration, it is first important to identify which patient population to target for PRO data collection based on the intended use of the data in the health care system. The next step is to choose specific outcomes and their measures. Considerations for selection are described below, and takeaways are summarized in Table 1.

There are two broad approaches for selecting patients for PRO measure administration once the intended use of PRO data is established: tailored and population-wide. The tailored approach involves capturing PROs from patients with specific health conditions, or who receive specific treatments, or who are identified using other clinical data within the EHR (e.g., lab results, pharmacy data). A reasonable first step for the tailored approach is to identify or hypothesize a clinical, public health, or quality-of-life need and then select the patient population accordingly. The tailored approach has the advantage of producing data that is potentially more pertinent to individual patient management, and combining PRO data with other clinical data in the EHR can facilitate health screening efforts. The disadvantages of the tailored approach include a reduced ability to make population-level comparisons, and a dependence on the completeness and accuracy of documentation in the EHR (e.g., encounter coding, problem lists), as well as the EHR’s functionalities (e.g., ability to link with pharmacy data) to accurately identify patients. Population-wide options include administering PRO measures to an entire patient population for which a health system or department is accountable, or administering to all patients in certain settings, like primary care or specialty clinics. The advantages of a population-wide approach are that it creates a culture for PRO collection among patients and clinicians and can be useful for population-level analyses and system-wide quality improvement efforts. The main disadvantage is a limited ability to collect health condition-specific information (depending on the care setting), which is information that typically feels most relevant to patients and actionable for clinicians. Patient response burden can be a challenge for either approach, particularly for patients who receive care in many settings or who respond to screening tools that trigger longer surveys. The EHR can be used to automate and coordinate processes to avoid repeated testing, and the selection of PRO measures can include computer adaptive test (CAT) options, among other features, which can ensure questions are relevant to individual patients, therefore minimizing burden.

Once a patient population is identified for PRO measurement, selecting outcomes to measure involves choosing among multiple possible PRO domains and weighing health condition-specific or more generic options. Potentially relevant domains include (1) symptoms, such as anxiety or pain, (2) functioning, or having the physical, psychological, and social functioning needed to carry out daily activities, (3) social health/support, or the patient’s perspective on their social environment, (4) general health perceptions, or the patient’s perception of their overall health, and (5) health-related quality of life, a construct that combines many of the above domains. Each domain has its advantages and disadvantages. Information on symptoms, for example, is highly actionable in a clinical context, while data on general health perceptions or health-related quality of life can be less actionable, but can also be easier to compare across populations and can inform patient management by offering insight into behavior and health status. Considered another way, PROs can be categorized as relevant for a specific health condition (e.g., PROs for psoriasis might include symptoms like itching, redness, or burning) or as more generic or applicable to both patients and healthy individuals (e.g., health-related quality of life, functioning). Condition-specific PROs tend to be highly actionable in a clinical context with more evidence available to support their use in practice, and the relevance of this kind of PRO is often readily apparent to the patient. The disadvantage of condition-specific PROs is that patients with multiple conditions might experience response burden. More generic PROs might limit response burden because they are applicable across conditions as well as better facilitate population-level comparisons, but they can be less actionable and their purpose might be less apparent to patients. It may be appropriate to choose PROs from multiple domains, using condition-specific and/or generic PROs in different circumstances, given these advantages and disadvantages.

The final task is to identify specific PRO measures for administration to patients. There are many criteria for measure selection, including the availability and attributes of existing PRO measures, desired degree of standardization or pooling of data across settings, EHR integration, stakeholder perspectives, resources required, and impact on clinical practice workflow. Existing PRO measures can be well-validated and reliable, for example, but costly to license. Some questionnaires might have been tested in multiple settings and languages, but be too complex or costly to implement in a particular patient population. Including patient, provider, and/or researcher input as stakeholders can improve patient-centeredness and potentially the use of PRO measures, but can also lead to contradictory perspectives. It is difficult to meet all criteria, but the Users’ Guide offers considerations and questions for each criterion to aid in the selection process.

2.3 Engagement

Patients, providers, researchers, administrators, and IT experts are stakeholders who will likely engage regularly in different roles with the integrated PRO-EHR system. Appropriate stakeholder engagement is essential to ensure PROs are integrated in EHRs in a way that meets the needs of these various users. Specific engagement considerations include how, where, and with what frequency patients will respond to PRO measures, how to display PRO data in EHRs, how clinical teams will act upon PRO data, and how to train, support and incent clinical teams and patients to incorporate PRO data into care. Each consideration for engaging stakeholders is described below, and takeaways are summarized in Table 1.

PROs are only as useful as the quality of the data collected, which makes the patient, by definition, the most important stakeholder to engage in the data collection process. Appropriate patient input is necessary to alleviate questionnaire burden, maximize response rates, and limit missing or nonsensical data. Synchronizing questionnaires across clinical areas can streamline the collection process and reduce the number and frequency of questionnaires for the patient to complete. Other ways to reduce burden include remote—rather than in-office—deployment of questionnaires, and offering multiple options for completion, like paper surveys or patient portals. The EHR system can help monitor PRO measure completion and prompt clinical teams to follow-up on incomplete surveys. PRO-EHR system programming can also prevent nonsensical data submissions or require response fields before PRO measure submission to avoid missing data. The technological functionalities of the PRO-EHR system, however, determine the feasibility of these strategies to engage patients in the collection of high-quality data. Health systems therefore need to balance a reasonable patient burden and desired data quality with their technological reality to determine the how, where, and frequency of PRO data collection.

After data collection, the display and accessibility of PRO data influence the engagement of patients, clinicians, and even administrators or researchers. The PRO-EHR system can display data at the individual or population level in various ways (e.g., through the patient portal, in clinical notes, in distinct places in the EHR). Data can take the form of numeric scores or graphs, cross-sectional analyses or longitudinal trends. Access to this PRO information can empower patients to monitor their own conditions, prompt care planning with clinicians, and enable managers and researchers to study health care quality. For this access to be effective, it is important to aid interpretation of the PRO scores and ensure that PRO reporting fits seamlessly in the clinical workflow. Decisions about the display and accessibility of PRO data help determine the ultimate clinical utility of the data.

The next goal is to make it as easy as possible for clinicians to act upon the PRO results. Providing notification of new or important information while minimizing unnecessary disruption to clinical workflow is a necessary balance to establish. When a patient completes a PRO measure, the data can be incorporated passively into the medical record without notifying anyone, or the EHR can offer standard or conditional active notifications and decision support. Standard notifications might simply indicate that new PRO data are available for review, while conditional notifications might trigger only if a PRO score or response is concerning. The modality of notification is another relevant consideration. Emails or clinical messages through the EHR that clinicians can review at times of their choosing are less intrusive, whereas texts or pages are more intrusive. Emails are the least secure option and lack systematic documentation of the notification, whereas messaging within the EHR is secure and documented. Texts or pages enable a timelier response. Another consideration is who should receive the notification: the clinician who ordered the PRO measure, the clinician scheduled for the next visit, the primary care provider (PCP), a patient navigator or administrator, or the patient’s choice of provider, among other options. The “ordering” clinician is the default receiver for most clinical tests, and is likely to be most interested in the PRO results. The clinician seeing the patient next may be best positioned to act upon the PRO data, but some systems cannot identify these types of providers. PCPs in contrast are usually easy to identify, but excessive notifications for results the PCPs did not order or visits they did not conduct can lead to little or no attention being paid to these notifications (i.e., “alert fatigue”). Navigators who triage PRO data to the appropriate clinician could reduce clinician burden, but could also slow response times. Patients might have thoughts about which of their providers should receive the PRO data, but that also might not be the case. Ultimately, too many notifications or alerts, and particularly those that do not feel actionable, can significantly reduce the utility of the PRO.

Even if a PRO-EHR system has all the necessary technological functionalities, the data are useless unless patients and clinicians have the training, support, and incentives they need to engage. Patients need to appreciate the relevance of PROs to their care, have access to basic technical support, and be able to use PRO data for self-monitoring. The health system can use marketing materials or scripts for office staff to convey relevance, but patients are likely to need more personalized conversations with clinicians or other care team members. Patients can learn the technical requirements for PRO measure submission in these same contexts, and support phone lines or emails can help patients troubleshoot issues remotely. There are fewer established methods to facilitate self-monitoring among patients using PRO data, but some might include decision support tools or algorithms. The clinical team and other office staff in turn need training and support to appreciate, interpret, and act on PRO data. This training might occur during a departmental meeting at the launch of PRO collection in a clinical setting and continue with on-going support at the team or individual level. Written guidelines, e-learning modules, or regular small group meetings can all help, particularly with interpretation and action, but these forums range from least to most time-intensive. Identifying staff and clinical team members as stakeholders in the redesign of clinic workflow can facilitate the use of PROs, but this effort can be time-consuming. There needs to be a culture change in the health system to support PRO use among both patients and clinicians, and the precise mix of support must adapt to the local context. Ultimately, the more patients see the PRO data being used to inform their care, the more likely they are to complete the questionnaires. Finally, although not specifically covered in the Users’ Guide, the availability of services (e.g., mental health) to address the needs identified by PROs is another important consideration affecting the ability of both clinicians and patients to act on PRO results.

3 Recommendations

Many of the actions and decisions needed to initiate PRO-EHR integration fall into the purview of health system leadership and IT specialists. We take a step beyond the Users’ Guide, and draw upon discussions from the May 2017 public PCORI meeting, to make the following recommendations to these stakeholders, following the three themes employed in this paper. Case studies that illustrate how various health systems have integrated PROs into EHRs are also available in Advances in the Use of Patient Reported Outcome Measures in Electronic Health Records, available on the PCORI website (https://www.pcori.org/assets/2013/11/PCORI-PRO-Workshop-EHR-Landscape-Review-111913.pdf) [10].

3.1 Planning

In planning the implementation of PROs within an EHR, it is necessary for decision makers to establish goals for both local implementation and higher-level coordination. At the local level, leaders should identify the stakeholders crucial to PRO-EHR integration within their organization and develop a marketing plan with the value proposition for each stakeholder in mind. The input and buy-in of these stakeholders at the early planning stages of integration can set the initiative on the best course for success in the implementation and resulting value of the effort.

In anticipation of a future need to pool PRO data across EHR systems, decision makers are encouraged to use open source data standards and create and implement PRO measures within them. For example, classifying PRO measures using CCDA or Logical Observation Identifiers Names and Codes (LOINC) will allow sharing or pooling of data outside of the organization. Metadata to document how the PRO data were collected and information about the practice setting should also be standardized and collected. If decision makers are already working with other organizations, they should establish rules for engagement for a central data repository and network of sites. There is a national need for PRO scientists to develop crosswalks across PRO measures to facilitate analyses.

3.2 Selection

Given resource constraints, health care organizations will face many questions about how to select specific PRO measures for use by patients and clinicians, and how to deploy them. We encourage organizations and researchers to develop a broad selection framework that addresses cost, burden, efficiency, quality, transparency, care, and patient outcomes.

3.3 Engagement

It is crucial to engage patients so that they are motivated to provide valid responses to PRO measures. Similarly, it is important to engage clinicians so that they pay attention to PRO scores, discuss the results with patients, and act on them when appropriate. A key step to achieve these goals is to develop guidelines for interpretation and action for patients and clinicians.

It is the right of patients who complete PRO measures to obtain the results. To do so, institutions should implement policies that provide easier access to the results, including displaying them within patient-facing platforms. It is also advantageous to design systems to provide patients with greater control of when and how (e.g., email, patient portal, paper) they complete PRO measures, similar to other communications with providers.

4 Conclusion

We have summarized the main considerations and takeaways for PRO-EHR integration from the key questions that the Users’ Guide covers in far more detail. A well-designed and integrated PRO-EHR system can help to promote patient-centered care, population health, quality improvement efforts, and health services research. The Users’ Guide can help patients, providers, and managers realize this potential for their own health systems, aided and informed by researchers and IT professionals. A health system wishing to integrate PROs into their EHR must consider patient and provider burden, effect on clinical workflow, and the technological functionalities of their PRO-EHR system, among many other considerations. Stakeholder engagement is crucial to address each of these potential challenges. There is no one-size-fits-all solution for integration, nor is there a way to maximize any one consideration without minimizing some others. A health system that is firmly committed to optimizing patient outcomes should devote time, resources, and expertise to plan and engage to gain the full benefit of integrating PROs into their EHR, and into the care of their patients.