First steps in PROMs and PREMs collection in Wales as part of the prudent and value-based healthcare agenda

Clinicians worldwide are experts at treating diseases and conditions, with new advances leading to continued changes in healthcare provision. While the main purpose of any healthcare system is to promote, restore and/or maintain health, arguably improving health ultimately aims to enhance health-related quality of life (HRQoL) [1]. Improved health may be achieved by a range of methods including better disease prevention, and improving curative, supportive and palliative care. While objective measures such as disease, survival and infection rates (for example) can be readily assessed by clinical teams, quality-of-life measures such as subjective wellbeing can only be assessed by patients themselves [2]. In order to identify what matters most to patients in relation to their health, we need to identify the symptoms, impacts and outcomes of greatest importance to them. This can help us support improved HRQoL which specifically refers to the health aspects of quality of life, generally considered to reflect the impact of disease and treatment on disability and daily functioning [3].We should listen to patients to understand the experience of care that they have received so we can improve provision wherever possible. Increasingly, patients are being asked to provide their views on their health and healthcare experiences using questionnaires or tools called patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs). PROMs measure patient symptoms and health-related quality of life from the patient’s perspective, often before and after (and sometimes during) treatment to identify any related changes [4]. PROMs were originally used in research but have been adopted into more widespread practice, particularly notable perhaps in England and Sweden. In Sweden, the National Quality Registers which collect patient-level data on treatment and outcomes were collecting PROMs by 2002 [5]. Swedish Quality Registers are now obliged to include PROMs for high-level certification and many report examples of how they are used for quality improvement initiatives such as shared decision making [6]. National PROMs collection started in England in 2009, initially focussing on four clinical areas: hip replacement, knee replacement, varicose vein surgery, and hernia surgery, and aims to enable change by identifying good practice, financially penalising poor providers and facilitating transparency [7] (PROMs collection in varicose vein and hernia surgery was discontinued in October 2017). As well as facilitating improved care on an individual patient level by providing an individualised service based on patient-reported data [8], co-ordinated PROMs collection can identify areas with relative good or poor outcomes through benchmarking, which can facilitate improved clinical performances [9]. It can also provide collated quality-of-life datasets to support decision makers such as funders by allowing healthcare providers to assess the effectiveness and cost effectiveness of care. However, despite a few large-scale initiatives, it has been suggested that PROMs collection is often sub-optimal and fragmented with limited co-ordination [10].

As their name suggests, PREMs measure patients’ experiences of care and may include questions about how well information was explained to them, whether they had opportunities to ask questions, and whether staff was polite [11]. PREMs can also be used to improve services by identifying areas with good and poor practice to drive service improvement. Research has suggested that there may be a direct relationship between patients’ experiences and their outcomes, particularly in relation to trust and the level of communication with their clinician [12].

In NHS Wales, PROMs and PREMs have been collected for a number of years, but this has primarily been driven via small local initiatives with a restricted approach to co-ordinated collection. This has reduced opportunities for shared learning from outcomes data. This limited uncoordinated collection has also restricted our ability to more broadly invite all of our patients to tell us about their experiences and outcomes of care. In order to facilitate standardised collection across Wales, the PROMs, PREMs and Effectiveness Programme (PPEP) was initiated [21]. This ambitious programme of work aimed to develop and roll out an electronic platform to collect PROMs and PREMs from all secondary care patients across Wales using a co-ordinated, top-down approach. As well as supporting the principles of co-production to help involve patients more in decisions about their care, unified collection and linkage of patient outcome data with clinical and administrative data have huge potential in supporting the VBHC agenda [22].

In early 2016, a successful grant application was made to the WG’s efficiency through technology fund to develop an electronic PROMs and PREMs data collection platform. Here, we provide a brief overview of the groups involved, methods used for tool selection and system integration.

To facilitate and support the development and roll out of the platform, a core team of staff was brought together under three primary work streams.

Here, we present the results of our work to develop an electronic platform which uses recognised tools to collect patient-reported data in a form which facilitates both individual patient-level use and cohort analysis.

A fully integrated national electronic data collection platform has been successfully deployed for use in HBs and Trusts in Wales with the first data collected in June 2016. Uptake across HBs has varied as different clinical areas have implemented use at their own pace, with some organisations choosing to use alternative systems to meet specific needs. However, all but one health board in Wales are currently using the national electronic platform for PROMs collection in some capacity. Methods of collection vary with most HB’s collecting only for specific conditions such as lung cancer or orthopaedics, while Cardiff & Vale UHB opted to roll out large-scale collection whereby all patients are invited to complete the generic tool. This is collected alongside relevant condition-specific tools where available. An overview of current collection on the platform is available in Table 1. As detailed previously [21], two methods of collection currently exist, one designed so that patients can complete surveys on electronic devices in-clinic, and another “remote” system allowing collection on electronic devices in any setting. As the “remote” collection is connected to the HBs local patient administration system, programming within the system means that once patients are assigned to a relevant clinical pathway, e.g. arthroplasty, our collection system automatically invites them to complete a PROM for that condition at the scheduled time points. The patient acknowledgement letter includes an invitation to complete the PROM via a web-link with a unique identifier which enables them to log in. Once the PROM is completed, the data are stored in a national repository and a copy added to the Welsh Care Records Service for access by clinical teams. PROMs collection is embedded seamlessly throughout the patient journey, with the system automatically sending out further PROM invitations at predetermined time points. The system also allows ad-hoc completions with the invite to complete the PROM remaining open so that once invited onto the system patients can complete the PROM at any time and repeat this at will. This means they do not need to wait until a request is sent to them and can complete one as needed, e.g. during a clinic visit. This is particularly useful for conditions which may require regular clinic appointments at times during the treatment journey. Patients can be on more than one pathway at any time as illustrated in Fig. 2.

Data are stored centrally at NWIS and available to individual HBs in order to maximise use on a patient level and population basis. The “in-clinic” system is not connected to the local patient administration systems, therefore, involves a member of the clinical team logging onto the system to select the correct tool for the patient to complete. As a result, PROMs data collected with this method are not added to the patient record; however, patient responses are still made available via a standalone clinical portal which allows clinicians to see the completed PROM.

While this collection method has less functionality and therefore increased limitations compared to the “remote” system, it is available as an alternative for clinics where remote collection is not yet available. It was originally developed to pilot the system, and intended as a temporary measure only. This system also has an increased risk of error due to the potential for the clinic staff to load the incorrect PROM or patient information. There is also an additional burden on clinic staff to facilitate this system, and it relies on local internet connections. As the “remote” collection extends, the “in-clinic” system will become obsolete and will be retired in the near future.

For both methods of collection, collected data are stored centrally at NWIS and available to all individual HBs in order to maximise use of their own data on a patient level and population basis, and for data linkage analysis. The dataset as a whole is available for national use.

The PPEP has recently been adopted into the national VBHC Programme in Wales within the Value in Health (ViH) team, to support its approach to delivering prudent healthcare and achieving the best possible healthcare outcomes for our population with the resources that we have [71]. As it extends across Wales, data collection and analysis support work to identify and reduce unwarranted variation in services and outcomes by using PROMs to pinpoint where variation occurs in different areas of care. These include variations in pre-operative health and outcomes of treatment. This knowledge can subsequently lead to positive service delivery changes and is illustrated by independent work carried out in ABUHB where the use of PROMs and activity-based costing was used to identify the treatment pathway that offered better outcomes at a lower cost within the memory assessment services [19]. Although the data we currently hold is too immature to be used in high-level decision making, we are able to start to use it on an individual patient level while it grows, allowing us to developing our understanding.

ABUHB is also collecting PROMs on a commercial system set up in parallel with the national platform, and this has offered flexibility and additional learning opportunities. Under VBHC and the ViH Programme, the national platform will be one of an array of patient communication systems with various uses, and there will continue to be a number of collection systems in place to offer choice to users. Importantly, data will be collected to a common standard and fed into a national data repository for linkage and analysis. The aim to collate this data on a national basis is supported by a recent WG Health Circular which requires all NHS Wales health boards and trusts to consistently submit clinical audit and PROMs data, and to support the flow of this data to NWIS for national use [72].

In order to utilise the data collected under the umbrella of ViH, a range of stakeholder groups continue to be involved in the evolving analytical plan. As well as analytical teams within individual health boards, there is ongoing input from the ViH in-house analytical team, colleagues from the Finance Delivery Unit (FDU) and both technical and data visualisation teams within NWIS. These teams are able to bring a range of skills and experience to the programme which will allow us to consider and deal with ongoing complications including data quality, mixed modes of administration, goals, case-mix adjustment, data linkage and standardised rules and guidelines for analysis [29]. We continue to work with clinical and analytical teams across Wales to ensure that we are able to identify priority areas and produce meaningful outputs.

The national platform continues to improve, and current developments within ViH focuses on data visualisation to support shared, informed decision making. PROMs responses available within the patient record will allow clinicians to visualise an individual’s PROMs responses over time and highlight where symptoms or problems are improving or worsening using data visualisations such as Red, Amber, Green alerts and arrow systems. These can allow clinicians to easily see how symptoms are responding to treatment and help identify changes in aspects of health. As well as allowing clinicians to monitor changes, it also provides a clear visual tool to support one-to-one discussions and shared decision making with patients in the clinic environment. This work is being piloted in lung cancer and will allow clinical teams to closely monitor patient symptoms to help focus the clinic appointment on the most relevant discussions and treatment decisions. Data dashboards are also being finalised which allow clinical teams to visualise the collated data available for specific conditions so that they can easily assess groups of patients at different stages of care and compare variables such as gender, stage of disease (if applicable), health board and age. This work has involved close collaboration between technical teams and clinicians from each speciality in order to produce a bespoke output for each specialism. A screenshot of the prototype of one data dashboard is available as an example in Fig. 3. As we aim to increase the use of the system to support an improvement in response rates, the use of a text reminder system has been piloted. This can send out automated messages to patients reminding them to complete a PROM before their appointment and is being tested to assess the effect on response rates.

We continue to improve the usability of the system and dataset through a constant process of review and refinement. Work is planned to improve the patients’ view of their own data so they can also view longitudinal completions.

The programme has overcome a range of technical and practical difficulties throughout implementation and is functioning as intended, integrating with other clinical platforms. The use of PROMs during the clinic appointment is putting patients at the centre of care by ensuring that they are fully involved in consultation discussions and decision-making processes. Imminent developments such as the introduction of data visualisation tools will make it easier to utilise and illustrate data in an accessible and meaningful way. Evidence suggests that sharing data derived from PROMs collection can improve patient engagement, encouraging patients to be more engaged and participate more actively in their healthcare [22].

The “in-clinic” collection system is still in use in some areas and lacks some of the functionality of the remote system. Ongoing work to maximise the availability of the remote collection system will allow teams to migrate over to this system so that the in-clinic system can be retired. Currently, generic PROMs data collection has dominated, and despite its growing use, pre- and post-treatment condition-specific data are limited. To facilitate case-mix and cohort analysis for decision making, it is important to have larger datasets which take time to accumulate. This may delay use of the data for some purposes while the data mature. In addition, the data are complex and are available to a number of end users; therefore, it is important that methodologies are developed to facilitate reproducible reporting and that reporting methods are transparent and consistent to prevent discrepancies between organisations.

There are remaining difficulties in accurately determining response rates due to a number of complex factors [29], but it is estimated that our response rates are currently low while the platform has been overcoming initial developmental challenges. This has allowed us to progress the platform to its current status without raising expectations prematurely. Now that it is widely available we will increase patient and clinician outreach to improve response rates. Electronic data collection may disadvantage some users [73], with a 2018 realist synthesis identifying a “digital divide”, leading to inequities in the ability to access and gain from digital innovation [74]. While the reasons for this are complex, factors include education, income and generational status as well as chronological age. Similarly, findings from a recent UK study suggest that older people and those with a long-standing illnesses or disabilities are less likely to have internet access [75] which may be a barrier to their ability to interact with the system. Further work is required to understand the factors leading to potential digital inequalities so that steps can be taken to mitigate against them.

As noted, there has been uneven implementation across the HBs with some collecting across all conditions, some in specific specialities and one currently not using the national platform at all but using an alternative commercial system. A number of HBs are using the national platform and commercial systems in tandem. This has led to additional complexities in collecting data into a single repository both from technical and data protection viewpoints. The numerous different patient administration systems (even within a single health board), levels of engagement, and requirements for additional functionality have been among the barriers to a more comprehensive roll out. However, the different options for collection have allowed health boards to develop their systems in a way that best suits their own needs, priorities and technical requirements. We have focused initial work in areas where clinical staff has been most engaged to allow us to test functionality, identify issues within smaller scale pilots and showcase outputs. The national system is available across all Wales for HBs to use if they choose, and its development has, thus, met the original aims of the programme. We are continuing to work with clinical and technical teams to improve functionality and clinical interest. As uptake spreads, additional electronic collection platforms such as commercially available systems and additional PROMs tools will become available which over time will allow groups to have a flexible approach and support growth of national collection.

The use of an electronic system keeps costs low as no manual data input is required. This also reduces the risk of human error and time spent on data cleaning [76], while providing instantaneous feedback. The nationalisation of collection has also reduced costs and burdens related to PROM licensing as we have been able to negotiate licences to cover collection across a large geographic area and prevented the need for each user (e.g. health board) to obtain individual licences. With the expertise of the teams involved, we have been able to translate and validate most tools in-house. With recent quotations from external providers to translate and validate a single tool for in the region of $7,000, this provides a significant cost saving. Teams within the programme have provided expert support in the technical development and implementation of the platform, and there is capacity to develop targeted reports for clinical teams and national audiences.

Although electronic PROMs capture has potential drawbacks regarding accessibility, data show that use amongst older people, for example, has increased substantially in recent years [73]. In order to help reduce access issues and to improve usability, our system allows a third party, such as a family member, friend, carer or clinical staff to assist with completion. We have also supported a number of clinics in purchasing electronic tablets to use in hospital waiting rooms so patients can complete PROMs while they wait for their appointment. In some areas, this is being further supported by volunteers to help patients who require assistance. It is likely that with time, internet access and IT literacy will increase which may improve our collection rate. We also plan to provide access to paper PROMs in some areas to improve collection rates.

Clinicians and healthcare providers worldwide are encouraged to collect patient data with groups such as ICHOM developing standard sets to facilitate collection and comparison [77]. The practicalities of this can be challenging for busy clinical teams with already limited resources. The national programme has allowed clinical groups in Wales to implement PROMs collection with minimal input, and feedback from clinical groups has been positive with patients also providing encouraging feedback on the process.

Traditional PROMs collection such as the NHS England National PROMs programme has been remote from direct patient care [78]. Our system is directly linked to patient care as data are easily accessible to the patient and their clinician through platforms such as the Welsh Care Records Service and NWIS’ Clinician’s Portal. This can prove beneficial in the clinic environment [79]. As well as providing an additional use of the data, this helps make PROMs collection more relevant, which supports change management.

We have proven that Despite challenging, it is feasible to incorporate electronic PROMs collection across organisations with different clinical systems, and to introduce their use to inform daily clinical care and decision making. For example, findings from the data on differences in outcomes by implant brand will allow colleagues in NHS Wales to make informed decisions during their procurement processes. As the data grow, we will be able to identify which healthcare services offered in Wales provide the best outcomes based on data provided by patients themselves. We are collecting data from a large number of patients across a broad range of clinical conditions. For some of these specialisms, PROMs use in clinical care is novel, and our work throughout the country is providing us with an insight into how our patients feel in a way that has not yet been fully investigated outside the research arena.

The amalgamation of different collection platforms under VBHC has allowed us to share knowledge and experiences across Wales. The development of data visualisation tools and data dashboards, plus the use of virtual reviews illustrates how PROMs collection can be an effective method of enhancing communication between the clinician and patient, both within and outside the clinic setting. We have involved clinical teams throughout the development of our programme including on the reporting and data visualisation work. There is currently significant interest from staff who are keen to access the PROMs data to inform their work. The use of virtual reviews is spreading which has the potential for additional clinics to prioritise clinic appointments and to free up clinic space for those in most need. As we start to resume routine care in the NHS following the Covid-19 outbreak, the use of remote methods of assessing health has become even more beneficial to prioritise care and reduce the need for unnecessary hospital visits. As we continue to embed PROMs into direct patient care, we will work with clinical teams and patients to raise the profile of the national PROMs platform, to improve the response rate and increase awareness of the potential of the system to improve patient care. As there are a number of clinical systems across Wales, setting central triggers for PROMs collection is challenging and may not be the most appropriate way forward if we wish to remodel care. Our experience to date suggests that communications should be locally facilitated but driven according to nationally agreed schedules to ensure that data are comparable whilst providing flexibility to fit local requirements and systems.

Perhaps most importantly, we have learned that patients want to be able to track their own outcomes and see the progress they are making over time. This would appear to be one method of involving patients in the process and providing an incentive to complete their PROMs. Data visualisation work will provide further opportunity to continue to improve communications with patients through involving patients in design processes to ensure that data are being presented in an accessible and engaging format. Work which allows patients to measure their own health and health changes against other similar patient cohorts may also help to improve engagement. Our ongoing work will gather patient feedback on their experience of this process.