The impact of patient-reported outcome measures in clinical practice for pain: a systematic review

Previous reviews examining PROMs in clinical practice have found studies to be heterogeneous [1, 16, 19] finding meta-analysis to be unjustified; therefore, meta-analysis was deemed inappropriate for this review [20]. This review used critical interpretive synthesis (CIS), a method of synthesis developed from meta-ethnography. CIS was developed as an alternative to traditional quantitative systematic reviews or qualitative syntheses, because researchers and healthcare professionals must examine diverse bodies of evidence to resolve complex problems within health care. CIS was thus designed to use both qualitative and quantitative literature to assemble arguments from all the available evidence, despite varying study designs [21, 22]. Synthesising the results of qualitative and quantitative research improves the understanding of a complex phenomenon by viewing it from multiple perspectives; trials can identify the effectiveness of an intervention in a certain context, with qualitative studies and surveys further exploring the potential impact of an intervention through participants’ views and lived experiences [23]. CIS also includes papers of high and low methodological quality, as all may have at least some relevance, although this is accounted for in the synthesis process [21]. The interpretive stages of CIS (outlined below) permit theoretical concepts from a diverse body of literature to be combined in order to generate a richer understanding of the phenomenon of interest.

This review followed established guidance regarding search strategies, inclusion and exclusion criteria, and data extraction [24, 25]. CIS guidance suggests literature searches should be broad and flexible and multiple methods were used to obtain relevant studies [21]. Several relevant databases were searched in January 2015: Medical Literature Analysis and Retrieval System (MEDLINE); Excerpta Medical Database and Allied and Alternative Medicine (EMBASE); PsycINFO; Cochrane Library; Web of Science; and PsycARTICLES. Terms included derivatives of patient-reported outcomes and clinical practice (see Table 1). The search was restricted to items published after 1985, when PROMs emerged in the literature [26]. Additional searches were conducted on: Google Scholar, the UK Clinical Research Network Study Portfolio website, bibliographies on obtained studies, and key author publications.

Study selection was predetermined by inclusion and exclusion criteria (see Table 2). During the screening process, one article was translated from Portuguese to determine eligibility. Full texts were examined and a list of potential studies generated by one reviewer, before two reviewers finalised studies for inclusion. See PRISMA flowchart in Fig. 1 [27]. The initial search was very broad, and it generated a lot of irrelevant studies; however, CIS encourages a broad and inclusive approach. The aim was not to obtain a representative sample but to obtain a comprehensive sample of all papers that met the inclusion criteria.

This review used CIS to synthesise the emerging concepts underlying the potential impact of using PROMs in clinical practice. Table 3 depicts the stages of synthesis. CIS mapped the qualitative literature against the quantitative to balance the inherent limitations of each method and provide further explanations of the results (for example of mapping see Table 4). Data interpretation by three reviewers refined constructs.

The CERQual tool was used to assess confidence in evidence for each of the concepts generated during the synthesis [28]. CERQual helps reviewers judge if the concepts are representative of the phenomenon being studied. The CERQual has four components which contribute to assessing the confidence of each review finding: methodological limitations, relevance, coherence, and adequacy of data (see Table 6). Examining the components, an overall assessment was made on the confidence on the concepts [28]. A final table was then developed summarising the concepts from the synthesis and the CERQual assessments. This assessment of confidence fits with the principles of CIS, which assembles arguments from all available evidence despite varying study designs and methodological quality. By using the CERQual assessment, we were able to formally assess confidence in the assembled constructs and overall synthesised arguments.

To examine methodological quality and risk of bias of the primary studies [29], questions were extracted from the Mixed-Method Appraisal Tool (MMAT) [30]. The MMAT provided a single method of analysing methodological quality for all studies, rather than applying various checklists to different studies [30]. The MMAT allows for studies to be assessed according to study design, and each is then evaluated on four criteria [30]. For example, randomised controlled trials are assessed on their randomisation, blinding, outcome data, and drop-out, with qualitative research being assessed on sources of data, analysis, context, and researchers’ influence. Other tools were examined for relevance to the review, but were deemed inappropriate due to the heterogeneity of the results, not allowing for assessment of the quality of the research in respect to the study design [31, 32]. This assessment provided an overview of study quality and methodological implications of the study, which was used when synthesising the study results. The two MMAT screening questions were modified to include the five appraisal prompts used for judging study quality in CIS [21].

One prominent use of PROMs was to assess patients’ pain. Clinicians from various backgrounds, including physicians, nurse practitioners, and physical therapists, suggested that the purpose of PROMs was to assess the patients’ pain and quantify the impact of their pain [34, 35]. PROM data were also seen as a useful way to view pain within the context of a patients’ life [36, 37]; illustrated in the following quote from a nurse using PROMs in a hospital setting: It is important to assess and take into account the thresholds of physical pain for each different individual on different occasions and how it is impacted by cultural and physiological factors [37]. Collectively, the qualitative literature suggested that PROMs were thought to provide a positive method of gathering essential information from patients. However, there is little information on participant characteristics or recruitment for these studies, so this finding may not reflect the population of interest, patients with non-malignant pain.

In one qualitative study, orthopaedic surgeons raised concerns over PROM data, seeing the data as highly subjective and questioning the patients’ ability to provide “objective” data on their pain [38]. A quote from a surgeon provides a powerful illustration of this: Getting patients to fill out forms is grossly inaccurate in my book… the patient 9 time(s) out of 10 wouldn’t understand what hip pain is [38].

Kazis et al. [39] explored physicians’ views through a survey on the contribution of health status reports generated from PROMs. The majority of clinicians felt that PROMs impacted overall patient assessment in some or all of their consultations and the reports contributed to medical history taking. Thirty-eight per cent of clinicians also felt that the reports contributed to physical examination during some or most of their consultations. Other clinicians felt that no contribution was made to overall patient examination, medical history taking, or physical examination. However, not all of the clinicians surveyed had been sent the health status reports and used them in practice, although some of their patients had completed the PROMs as part of an RCT. Their lack of experience using PROMs may have significantly influenced their views on how PROMs contribute to patient assessment.

The outcomes related to this construct are inconclusive. Clinicians had mixed views when surveyed on whether PROMs may contribute to patient assessment. Similarly, in the qualitative studies clinicians suggested PROMs had benefits in this area but also voiced concerns about the validity of PROMs.

Clinicians felt that PROMs made valuable contributions to the decision-making process surrounding care. Across three qualitative studies, clinicians including medical doctors, surgeons, and nurses claimed that PROMs facilitated shared decision-making [33, 37, 38]. This is demonstrated in a quote from a Swedish healthcare provider, after PROMs had been implemented into their clinic for two years: Work is smoother, it is much easier to form an opinion and decisions are easier to make [33]. PROMs were thought to provide useful information to choose an appropriate treatment for a patient and develop a treatment plan.

PROM scores also enabled clinicians to provide individualised treatments based on patients’ needs and direct them to appropriate care [34, 37]. Within a study on nurses’ use of PROMs, a nurse stated: This method is of great value in the performance/assistant of planning so we can assign a more expressive care in relation to the pathology and the patient as a whole. Thus, seeking to minimise the patient’s suffering and pain [37]. Using PROMs in decision-making enabled clinicians to feel they had enough information to develop individualised treatment plans.

PROMs were also used in the decision-making process to enable clinicians to set functional goals with patients. Two case series and a case report examined how PROMs were used for goal setting [34, 36, 40]. PROMs provided baseline data on patients’ current situation and then used to anticipate change and set goals.

No studies quantitatively tested the hypothesis that using PROMs improves shared decision-making. However, the qualitative literature does suggest that shared decision-making improves and decisions are increasingly individualised with PROMs.

The synthesis suggested that PROMs had an impact on the therapeutic relationship between patients and clinicians through improving communication and patient engagement regarding their care.

A case report demonstrated how PROMs were used to improve communication between patient and physical therapists and start dialogue regarding their care [34]. Although the authors did not provide adequate details of the procedure and analysis, other studies demonstrated similar findings. For example, in one study both patients and clinicians believed that using PROMs changed the clinician–patient interaction, as this patient explained: The system made it possible for the provider and I to talk about the important issues [33]. In a survey of primary care providers (PCPs), all using PROMs in their clinical practice, 76% felt satisfied that the PROM measuring pain helped patients participate in their pain management [35].

Other qualitative findings also suggested that clinicians believed PROMs enabled patients to get involved in their care. This included identifying patient concerns and engaging patients in self-management [33, 35, 36]. One nurse stated: I see the implementation of the pain scale as a way to humanize care, where we can stop relying on machines and turn to the patient; to what he is saying and feeling. Giving them an active voice and a right to express themselves [37]. This humanisation of care, aided by communication and patient engagement, was thought to improve the relationship between patients and clinicians. Similarly, in a survey of doctors (some of whom had experienced PROMs and some of whom had not) the majority felt that PROMs contributed to the doctor–patient relationship, although the survey did not examine whether this contribution was positive or negative [39]. However, qualitative literature suggests that PROMs may facilitate interactions, aid communication, and promote individualised care. It is through these processes, that PROMs may improve the therapeutic relationship.

Several studies demonstrated using PROMs for the tracking of patient progress, using the scores from PROMs to evaluate treatment and change treatment plans accordingly.

A survey found that 53.3% of PCPs were satisfied that the PROM helped them to understand patient progress [35]. A case series also suggested that information from PROMs was used to track progress [40]. Finally, this use of PROMs was also demonstrated by nurses: This scale is important in the sense of monitoring the evolution of the intensification of pain and even to what point the treatment is being beneficial to the patient [37].

Despite these findings, only 39.9% of PCPs felt satisfied that PROMs helped them to modify a treatment plan [35]. Several clinicians from two studies did not feel that the PROMs helped them modify a treatment plan [35, 38]. Several surgeons raised concerns over the information provided from PROMs, one surgeon stated: I just think there is a lot of effort being put in there for not a lot of surgical gain from my perspective [38]. Thirty per cent of PCPs surveyed on PROM use were dissatisfied regarding PROMs to help them to modify a treatment plan [35]. However, these surveys specified neither previous treatments nor future planned treatments.

Nonetheless, clinicians from several studies reported that PROM scores did influence treatment plans, and this was done on both an individual patient level and clinician level. A qualitative study on surgeons, PROMs encouraged two clinicians to reflect and change their clinical practice [38]. Individually patients’ treatments were also affected, one nurse stated: It is (sic) tool that allows us to quantify the pain our patient is feeling with more accuracy, and rethink whether or not the therapy being given is really effective in treating that individual [37].

As part of the construct on tracking progress and evaluating and changing treatment, two sub-constructs were generated: using PROMs to change patient medication use and using PROMs to change referrals to other clinicians and health services. One case report suggested PROM scores were used to refer the patient to another service [34]. Doctors surveyed on PROM use had conflicting opinions; 50% of doctors felt that health status reports (generated from PROM data) did not contribute to patient referrals, and 54% of doctors felt that reports did not impact on medication decisions [39]. However, not all doctors had used PROMs in practice.

Five studies tested the impact of PROMs on medication decisions. One study found that 17% of patients had analgesia altered and 6% of patients had an additional dose of analgesia after PROMs had been implemented across a hospital [41]. Another study, which issued nurses with training on PROMs and implemented PROMs across a cardiac surgery ward, found that after training and implementation, patients had higher morphine consumption [42]. In comparison, three studies showed no significant differences in medication across intervention and control groups [39, 43, 44]. No significant differences were found in additional treatment [43], arthritis referrals [39], or reducing doctor visits [39].

The effect PROMs have on tracking patient progress, evaluating and changing treatment is unclear. Surveys and interviews with clinicians identified mixed views, with additional conflicting results from trials testing the impact of PROMs on referrals and medication use.

Studies suggested that PROM use might influence patients’ health status, pain levels, and satisfaction. Two trials tested the impact of PROMs on patient outcomes, but no significant differences were found between the intervention and control groups on patient satisfaction [39, 45] or health status [39].

PROMs were also hypothesised to impact pain levels. Ravaud et al. [45] conducted a cluster-RCT; three wards were assigned to the intervention group and three wards assigned to control; the intervention group received education on pain and assessing pain with a visual analogue scale, and the scale was then used within the intervention wards. Pain significantly decreased in the intervention group compared to control (d = 0.1796 [0.0643–0.2949] p = −0.038) [45]. An additional study assessed whether pain assessment through PROMs changed clinical practice; case coordinators in the intervention group received training on PROMs and PROMs scores were put into a summary sheet for patients and clinicians, showed no significant differences between intervention and control groups for pain levels [44]. However, the intervention group did show some benefit in pain levels; they reported less pain related to strenuous activity at follow-up (d = 0.4253 [0.054–0.7966] p ≤ 0.05) [44].

There is no definitive evidence as to whether PROMs have an impact on health status, with only some studies showing significant differences. Studies showed no effect on patient satisfaction. Additionally, no studies examined adverse effects on patient outcomes.