In clinical trials, patient-reported outcome measures (PROMs) improve communication between patients and clinicians, resulting in quality of life and survival benefits [1
]. Implementation strategies are key to translating health innovations into practice [4
], but little is known about the optimal implementation strategies for PROM integration in medical oncology outpatient departments.
Care delivery transformation within the clinical environment of an outpatient oncology clinic is complex [5
]. This complexity comes from the many levels of the larger health ecosystem [8
] into which outpatient oncology wards are embedded [9
], as well as the diverse needs of cancer patients undergoing a multitude of treatment protocols and follow-up [10
]. Teams of multidisciplinary health professionals may be involved in each patient’s care and they may have competing priorities with diverse knowledge, attitudes and beliefs [11
]. In randomised controlled trials (RCTs) testing the value of PROMs, the clinical trial coordinator often played an invisible role facilitating uptake of the PROM innovation with patients and staff teams (e.g. conducting training sessions and running clinic-level reports) [13
]. However, in routine clinical practice, there may not be investment in a dedicated implementation support role to support this complex process [15
]. To implement a new intervention into this type of an environment requires insight into the interplay of the innovation (PROMs), the context and setting and the people and roles involved [16
The multiple stakeholders required to implement PROMs include patients, families, nursing staff, administrative staff, medical staff, allied health professionals and records managers, all need to be engaged in using PROMs [5
]. Their involvement is required to ensure initiating, handling, interpreting, and responding to PROMs as part of routine care delivery [17
], thus making this a complex healthcare intervention [19
We used the implementation science framework “Integrated Promoting Action on Research Implementation in Health Services” (i-PARIHS) to examine factors impacting the use of the PROM version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) [20
] during patient care visits.
i-PARIHS is a framework specifically designed for implementing complex interventions in healthcare [21
] and is unique in emphasising the role of a Facilitator as the central construct of successful implementation. Like other determinant frameworks in implementation science [22
], i-PARIHS can be used to help understand and overcome the multiple influences impacting implementation. Based strongly on experiential learning and empirical evidence, whilst drawing on a range of behaviour change theories, i-PARIHS can be used to plan, guide and understand complex implementation in healthcare [23
]. This paper reports on a pilot implementation case study using the constructs of i-PARIHS (see Box 1
The aims of this paper are to
describe this case study of PROMs implementation in routine oncology practice using the i-PARIHS framework [21
], to better understand what conditions are critical to enable integration of this intervention into the clinic;
describe the Facilitator role and how it was used to overcome barriers to PROM implementation;
determine changes in clinician and staff perceptions of acceptability [4
] and knowledge of PROMs, and perceived skills in interpreting and using PROMs with patients and
summarise lessons learned to inform implementation, sustainability, scale and spread of future PROM projects.
Box 1: The intervention design
The PROMs intervention consisted of patients completing the PRO-CTCAE in the waiting room of two oncology clinics in Australia, and clinicians reviewing patient responses and acting on burdensome symptoms. The core-set PRO-CTCAE measures cancer-specific symptoms such as constipation, diarrhoea, nausea, vomiting, shortness of breath, fatigue, anxiety, low mood, pain, peripheral neuropathy, and other symptoms . Patients completed the PRO-CTCAE items via a touchscreen workstation in the waiting room. A paper copy of the patient’s PROM report was automatically printed, then immediately available to the care team for use during the visit. The PROM report was also imported into the patient’s scanned electronic medical record for future reference. The immediate availability of standardised symptom reports has been shown in RCTs to improve symptom detection, more consistent management of symptoms, quality of life and mortality [13
Implementation took place in two oncology outpatient clinics in a large metropolitan hospital in Queensland, Australia. This hospital provides oncology services for the metropolitan area as well as specialist referral services for patients across the state.
To comprehensively understand PROM implementation issues presented in the literature [25
], and to inform selection of an implementation framework, we previously undertook a scoping review of systematic reviews [19
]. This review identified key themes of (1) engaging clinicians, (2) integration of technology for PROMs and (3) strategies to respond to PROMs data [14
]. It was identified that these themes correspond to core constructs of the i-PARIHS framework (Recipients, Innovation
), and so it was chosen as the guiding framework for our pilot study [30
]. In this report, we present i-PARIHS terms in italics
Each core construct of i-PARIHS has a sub-set of potential factors that can become enablers or barriers for implementation that can be addressed using Active Facilitation,
which has been demonstrated to increase uptake of research evidence in primary care [24
] and is “how” the implementation happens. Central to the i-PARIHS framework, Facilitation
is “the construct that activates implementation through assessing and responding to characteristics of the Innovation
and the Recipients
(both as individuals and in teams) within their Context
ual setting” [21
, p. 6].
For this study, an internal novice Facilitator
(NR), an oncology nurse experienced as a clinical trial coordinator, conducted study activities pre-, during- and post-implementation. Consistent with the concept of the Facilitator’s journey
], the novice Facilitator
engaged with a expert Facilitator
(AM) and a local multidisciplinary peer group of clinician–implementers for reflection (and peer learning) every 1–2 months during the implementation. She also had structured academic support from a research team with university academic affiliations to the setting.
The scoping review of systematic reviews [14
] theme of engaging with clinicians aligned with the i-PARIHS Recipients
construct of “who” receives the intervention. Whilst the Recipients
included clinicians, patients and health administration staff, this case study focuses on the views of the clinical staff, as the group who reviewed the PROM, interpreted patients’ responses and responded to achieve the anticipated clinical outcomes of the intervention. Clinical staff in the setting included nurses, doctors (specialist oncologists and trainees) and a range of allied health professionals, including dietitians, physiotherapists, occupational therapists, speech therapists, psychologists and social workers.
Within the i-PARIHS framework, potential Recipient
enablers and barriers include factors, such as motivation
, value & beliefs
, clinical consensus
, local opinion leaders
, existing data sources
, skills and knowledge
, time and resources
, learning environment
, collaboration and teamwork
, power and authority
and professional boundaries and networks
In the scoping review of systematic reviews [14
], the theme of technology integration for PROMs aligned with i-PARIHS’s construct of the Innovation.
This is “what” is implemented and is described in more detail in Box 1. Potential barriers and enablers within the Innovation
construct include underlying knowledge sources
, degree of fit (compatibility or contestability)
, degree of novelty
, likely boundaries
and relative advantage
The scoping review of systematic reviews theme [14
] of strategies to respond to PROMs data aligned with i-PARIHS’s Context,
“where” the implementation takes place and encompasses the characteristics of the setting.
Enablers and barriers considered under the Context
domain included characteristics of the inner and outer context. Inner Context
factors included formal and informal leadership support
, past experience of change
, mechanisms for embedding change and learning
and evaluation and feedback processes. Outer Context
factors included organisational priorities
, leadership and senior management support
, systems and processes
, history of innovation and change,
and absorptive capacity
(the ability to absorb change in the Context
This study used a descriptive mixed methods case study design nested within a prospective trial of PROMs implementation in two oncology clinics [33
]. The research protocol of the trial is presented elsewhere [35
Data collection and analysis
Qualitative data were coded by the Facilitator
] using a content analysis [36
], informed by the i-PARIHS framework [21
]. Data collected by the Facilitator
over a 12-month period was informed by the research protocol. These data included field notes comprising structured observations of activity for context assessments and process mapping [21
], weekly case report forms and transcribed quotes from Recipients
as memos. Results from the pre-implementation findings had been previously shared (and amended for accuracy) with Recipients
to plan the implementation and during the iterative evaluation cycles, then also reviewed by staff post-implementation. Additionally, free text responses on questions about perceived knowledge, perceived usefulness and perceived enablers and barriers in a clinician survey completed pre- and post-implementation were included as a qualitative dataset [35
]. These qualitative data were collated for the evaluation by NR, and informed discussions with an expert Facilitator
To determine if each factor was a barrier or enabler from the Facilitator perspective, the novice Facilitator (NR) and expert Facilitator (AM) separately identified and rated factors along a visual continuum of barrier to enabler, and then combined their ratings by consensus. Consensus discussions were summarised into templated tables, and illustrative examples of facilitation strategies were agreed on by discussion. Final tables were reviewed for face validity by a third rater, the lead senior academic team member not directly involved in the health service (MJ). Whilst we recognise that such judgements are inherently subjective, potential biases on the part of the Facilitator (NR) were thus mitigated through a number of mechanisms, including the cycles of evaluation with participants, reflexive monitoring, expert facilitation discussions and collaboration with a wider academic team.
Quantitative data included a survey adapted from Rouette et al. [35
] used to elicit clinician and staff perceptions of PROM acceptability [34
], knowledge and utility of PROMs, and enablers and barriers. Surveys were distributed to nursing, medical and allied health professionals involved in providing care to patients, before and after implementing the intervention. Data were summarised by discipline group, reporting the proportion of respondents who agreed or strongly agreed with each statement and are presented in a table for comparison.
The potential conflict of interest that could come from the novice Facilitator’s relationship with participants was addressed in the design of the study, including the ethical considerations that were integrated into the study protocols. For the anonymous staff surveys, participation was considered consent and staff could decline participation in surveys simply by not completing them. During the implementation phase, staff in-service education was provided, marketing flyers distributed and information sheets provided. These resources explicitly stated that participation was voluntary and staff could opt out at any time in writing on case report forms, or by simply not participating in the implementation work.
This study evaluated implementing PROMs into the routine care of an oncology outpatients’ department using the i-PARIHS framework. The i-PARIHS domains of Facilitation, Innovation, Recipients
were useful for identifying barriers and enablers so that tailored strategies could be developed and put in place. The use of an implementation science framework in a research study offered new insights into the characteristics of implementation in differing Contexts
]. Using i-PARIHS provided a systematic way to organise observations, provide new insights into how research findings can be translated and how they impact delivery of care in routine oncology health services [16
]. This can inform future larger studies, which in turn can be used to improve outcomes in large populations of patients [40
By using i-PARIHS, we were able to identify, measure and report on how Facilitation
ameliorated barriers to implementation. Core Facilitation
strategies included education, awareness raising, and audit and feedback [41
took a dynamic and flexible approach balanced with the formal structure of audit and feedback
that acted to inform and regularly engage with Recipients
(care teams). Similarly, in previous work, Harvey et al. [40
] in the FIRE study found that regular meetings were important for implementation to offer goal-focussed practical support, technical assistance and interactive problem solving.
is a recognised implementation strategy [43
], which uses a relational approach to problem solving and may involve trained personnel external to the healthcare context [40
]. In our study, the novice Facilitator
was part of the local staff, but was supported by an experienced Facilitator
and team to identify and reflect on appropriate strategies [21
]. The support of an experienced Facilitator
was valuable whilst the novice Facilitator
developed increased awareness, skills and knowledge. Besides providing verbal guidance and problem solving, the novice Facilitator
had to be physically present in the clinic to support the clinicians, problem solve to make the PROM intervention fit the Context
and maintain implementation momentum. This demand on the Facilitator
illustrates the tension between a Facilitator
doing the work rather than enabling implementation [43
]. By keeping up momentum through workflows, the Facilitator
enabled the experience of using a PROM in clinical practice for staff [44
]. However, for sustainability, the intervention would need to be fully integrated into routine workflows before scaling up.
implementation strategies resulted in a shift in staff perceptions about the usefulness of PROMs, their knowledge about PROMs and how they could use them practically to improve their clinical care. Interestingly, whilst nursing knowledge and confidence improved markedly post-implementation, allied health professionals reported somewhat lower knowledge and confidence post-implementation. Whilst the reason for this is unclear, the very high self-reported knowledge and confidence pre-implementation may have reflected lack of experience applying PROMs within this complex patient group, with more conservative estimates after experiencing the PROM implementation. Additionally, the doctors and nurses may have had more exposure to the PROMs which may have increased their knowledge and confidence. Staff engagement with PROMs has been identified as essential for implementation success [29
Staff surveys showed that the acceptability of PROMs increased from pre- to post-implementation. The construct of acceptability has been described in the literature as dynamic and changing as Recipients
experience an innovation/intervention and gain a deeper understanding of “what” is involved [4
], and this matched our experience in the case study. In the post-implementation survey sample, the perceived “usefulness” (relative advantage
) of PROMs increased as clinicians started to report that they understood better the value of PROMS and reduced barriers to PROMs use [4
We also found that the collection of formative data in our case study was essential to enable successful facilitation. Staff responded to quantitative data of completion rates and staff acknowledgement rates with accompanying observational data, such as changes to workflows in response to events taking place in clinical areas. This use of data assisted in giving all Recipients a deeper understanding of the factors at play in the Context, and insight into the importance of the role staff have in the facilitating implementation.
Implementation science theories, such as i-PARIHS, can provide rich data to support and expand existing frameworks for PROMs. For example, Van der Wees and colleagues described a sequence of steps for choosing and implementing PROMs (goal setting, selecting PROs and PROMs, developing and testing of quality indicator(s), and implementing and evaluating the PROM(s) and indicator(s)) [8
]. Our case study shows that it may be worthwhile to combine PROM-specific frameworks with implementation science theories to better characterise the ways in which PROM attributes (e.g. validity and reliability) may interact with the implementation process. Implementation science theories systematise barriers so they can be linked to the broader implementation literature [47
], directly link barriers to evidence-based support strategies [48
] and provide a way to examine contextual factors influencing implementation (e.g. organisational readiness to change and clinic culture) [48
Whether implementation is successful or not, there is much to learn when it is systematically described, and this knowledge is disseminated [49
We recognise some limitations with the evaluation. Data used to inform the analysis of the i-PARIHS constructs included observations collected by the Facilitator (NR) in site journals and field notes. This may be biased both by her perspective as a clinical trial nurse within the study context, and as the Facilitator, and are intrinsically subjective judgements. There is a balance in an implementation evaluation between the deep knowledge of context available to an internal Facilitator, and the risk of bias inherent in holding this tacit knowledge. Mechanisms to increase objectivity included contemporaneous notes, triangulation of data sources, reflexive monitoring with a research team, including members internal and external to the Context, and regular reflection with an expert Facilitator external to the project team. Staff surveys were anonymous, and used as cross-sectional data, and it is unknown where there is some overlap between respondents at both time points. The exact numbers of staff surveys distributed to each professional group was not able to be accurately captured, only the total number distributed due to the non-identifiable approach to data collection.
The detailed description of the Recipients
highlights the importance of local factors for implementation success, which means that findings may not be generalisable to other clinics or settings. However, these findings could inform approaches to implementation in other settings as the enablers and barriers had consistency with the findings of the initial scoping review [14
The i-PARIHS framework was useful to systematically inform and iteratively plan the implementation of PROMs into routine oncology care. The four core i-PARIHS constructs (Facilitation, Recipients, Innovation and Context) all needed to be addressed to allow successful implementation, with Facilitation having a particularly important role moderating the other domains of Recipients, Innovation and Context. In this study, the PROMs intervention was acceptable to many staff, but complex barriers mean that sustainability may not have been achieved, and teams may struggle to maintain PROMs once the facilitation role has been withdrawn. Further research into efficient implementation and sustainability of PROMs is required to ensure lasting benefits.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.