Introduction
Methods
The evidence-based KLIK PROM portal
CFIR domain | CFIR determinants | Reasons for successful implementation |
---|---|---|
Intervention characteristics | Evidence Strength & Quality | The evidence of KLIK is emphasized in the training for clinicians [34] |
Intervention characteristics | Trialability | KLIK started small and has found its way, step by step, in many hospitals and has scaled up to adult healthcare and other countries A license agreement is signed at the start, which can be ended and therefore undo the implementation if needed The implementation process and workflow are adapted according to the wishes of every multidisciplinary team, as the KLIK team experienced that a ‘one size fits all’ approach was not feasible |
Intervention characteristics | Design Quality and Packaging | Clear and direct available feedback of PROMs on a well-designed dashboard The design of the KLIK PROM portal is evaluated positively, both by clinicians and patients [35] A strength of KLIK is the design of the PROM feedback and the variety of options [36] Optimization of the PROM feedback in KLIK is an ongoing process, based on scientific knowledge [37] and user experience |
Outer setting | Cosmopolitanism | Worldwide, there is increased motivation for the use of PROMs in clinical practice, e.g., Value-Based Healthcare supports the use of PROMs, which facilitates the implementation climate The KLIK expert team shares common experiences with other hospitals through collaborations and networks (e.g., ISOQOL, PROMIS, research projects, implementation in many Dutch hospitals and the UK). Therefore, the KLIK PROM portal is increasingly well known and more visible for interested stakeholders |
Outer setting | External Policy & incentives | Former research showed lack of formal agreements, such as policy and work plans on using KLIK at a hospital level [29]. However, this is changing, because from a governmental perspective, collecting PROMs or using Routine Outcome Monitoring for benchmarking purposes is increasingly encouraged or even obligated |
Inner setting | Goals and feedback | During the KLIK training goals on implementing PROMs are clearly communicated, as previously different expectations were noticed (e.g., discussing PROMs in the consultation room versus collecting PROMs for research purposes), which may hinder the implementation Clinicians receive feedback regarding the implementation process during the annual evaluation meetings |
Characteristics of individuals | Knowledge & Beliefs about the intervention | Multidisciplinary teams initiate implementation themselves and are, therefore, motivated to use KLIK. However, some clinicians of a team may have a negative attitude and show resistance, because they do not know the added value of using PROMs in clinical practice. The KLIK training provides knowledge of underlying principles and helps to generate enthusiasm |
Characteristics of individuals | Self-efficacy | The KLIK training provides clinicians with knowledge, tools, and skills to feel competent to implement KLIK in their practice. However, there could even be more emphasis on training communication skills, as some clinicians report low confidence in discussing psychosocial topics with their patients Research shows that most clinicians have sufficient knowledge to use KLIK as intended [29] Current focus is on empowering patients to discuss PROMs with their clinician, for example by developing educational videos |
Design
A. Retrospectively describing the most prominent determinants of successful KLIK PROM implementation using CFIR
B. CFIR-ERIC Implementation Strategy Matching Tool to identify current barriers of the KLIK PROM portal implementation
Results
A. Retrospectively describing the most prominent determinants of successful KLIK PROM implementation using CFIR
B. CFIR-ERIC Implementation Strategy Matching Tool to identify current barriers of the KLIK PROM portal implementation
CFIR domain | CFIR barrier | What is already done? | Why still a barrier? |
---|---|---|---|
Intervention characteristics | 1. Relative advantage | Overall, clinicians working with the KLIK PROM portal think it is a valuable tool to monitor PROMs in their patients [12, 29, 35] The advantages about the KLIK PROM portal and discussing PROMs in the consultation room are spread during conferences and in scientific papers | Some stakeholders are reluctant to change and do not see the advantages of using PROMs, or suggest that alternative solutions (e.g., administering PROMs using EHRs or on paper) can be used |
Intervention characteristics | 2. Adaptability | KLIK is a very flexible system where many individual wishes of the multidisciplinary teams can be met (e.g., different PROMs for different patient groups at different timeslots, for different ages, various forms of feedback different for specific clinicians) To make KLIK as user friendly as possible. For example, KLIK is available in different languages and proxy reported PROMs are offered for patients with disabilities [13] | Clinicians prefer the intervention as tailored as possible. A standard set of PROMs is currently being offered to patients automatically based on age and patient group, not yet on an individual patient level (selecting specific PROMs per individual patient per visit) |
Intervention characteristics | 3. Complexity | The KLIK PROM portal is easy to use as a result of its origin in pediatrics. A recent evaluation study shows that the majority of clinicians (72%) think KLIK is easy to use [35] Together with the use of the KLIK PROM portal, hospitals receive support and advise from the KLIK expert team during all steps of the implementation (see Fig. 3) | For some clinicians KLIK remains complex to use, for example if they are not familiar with ICT. In addition, it requires additional actions, because clinicians need to actively motivate patients and sometimes send out the PROMs to their patients |
Intervention characteristics | 4. Cost | KLIK is being offered at low costs, as we are a non-profit organization, and alternative portals are often more expensive | Within healthcare there are often insufficient financial resources. Therefore, some teams still decide to refrain from using PROMs because of the additional costs |
Inner setting | 5. Structural characteristics | In general, KLIK is being implemented bottom-up, where small multidisciplinary teams show their interest in using KLIK | Hospitals are large organizations, and obtaining permission to change existing workflows can be a long process The board of the hospital might not be aware of bottom-up processes and can, therefore, be perceived as a barrier in larger scale implementation |
Inner setting | 6. Tension for change | Champions (clinicians who are enthusiastic about using KLIK) can explain the added value of using PROMs in clinical practice and persuade colleagues in trying out KLIK as well | Some clinicians do not see the current situation (not using PROMs in clinical practice) in a need of change |
Inner setting | 7. Compatibility | At the start of the KLIK implementation, the KLIK expert team advices on how to fit KLIK best into the existing workflow Recently, in four hospitals, a front-end integration with KLIK and the EHR (Epic© and HiX©) is realized | A study showed that a perceived barrier for stakeholders was compatibility (24% of clinicians indicated that the KLIK method did not fit well with current routines) [29]. To make it better fit with existing workflows, KLIK should be integrated into the EHR in all hospitals |
Inner setting | 8. Organizational incentives & rewards | Clearly communicate incentives (e.g., communication tool, improvement of quality of care, data can also be used for scientific purposes) of using PROMs in clinical practice for both patients and clinicians | Sometimes there are no incentives in the opinion of multidisciplinary teams and they, therefore, do not promote the use of PROMs in clinical practice |
Inner setting | 9. Leadership engagement | License agreements are signed by an authorized signatory and it therefore approves the implementation | Key organizational leaders or managers could show more commitment and involvement in KLIK by promoting it actively. In addition, in the current situation, they are not held accountable for implementation of the innovation |
Inner setting | 10. Available resources | KLIK has received several grants for the implementation and developed a business model to provide financial resources for the KLIK expert team in addition to the external resources | There is no structural funding yet for the KLIK expert team. To continue implementing KLIK, we are currently working on a new business model where we are not dependent on external funds, but can provide the use of KLIK at low costs For clinicians, money or time to discuss PROMs can be a barrier in implementing PROMs |
Characteristics of individuals | 11. Individual stage of change | When clinicians experience benefits from implementing PROMs, they become more enthusiastic By training clinicians, the skills necessary to implement and discuss PROMs are provided During the annual evaluation meetings we identify clinicians that do not perceive enough benefits or forget using KLIK. These meetings keep the clinicians focused on the goal of discussing PROMs From the perspective of patients, information letters, flyers, and educational videos are provided to give them the skills to complete and discuss PROMs. In addition, focus groups are held to explore their experiences regarding KLIK, in order to further optimize KLIK | Clinicians that do not feel skilled or enthusiastic about using the innovation in a sustained way are resistant to use the intervention. Feedback from patients includes that PROMs are not discussed by the clinician, they sometimes do not see the added value, and PROMs can be long and repetitive |
Process of implementation | 12. Champions | Most teams have a champion (an individual who support the KLIK implementation in a way that helps to overcome indifference of resistance by key stakeholders) who is motivated to start implementing KLIK for their patients | Some champions seem to have insufficient influence to convince their colleagues |
Process of implementation | 13. Engaging (Key stakeholders) | Clinicians are involved in the entire implementation process Also patients are more and more involved in the KLIK PROM portal, e.g., by asking their opinion in both qualitative and quantitative studies, developing educational videos to prepare them for the outpatient consultation, and by collaborating with patient associations | Patient engagement can be increased, for example, at the start of the implementation to explore the relevant PROs for patients |
Matching ERIC strategies to CFIR barriers
CFIR barriers | Cumulative Percent | 1. Relative advantage | 2. Adaptability | 3. Complexity | 4. Cost | 5. Structural Characteristics | 6. Tension for Change | 7. Compatibility | 8. Organizational Incentives & Rewards | 9. Leadership Engagement | 10. Available Resources | 11. Individual Stage of Change | 12. Champions | 13. Key Stakeholders |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ERIC strategies | ||||||||||||||
Identify and prepare champions | 449% | 45% | 23% | 30% | 12% | 27% | 48% | 21% | 25% | 41% | 4% | 44% | 67% | 63% |
Promote adaptability | 312% | 24% | 73% | 40% | 16% | 23% | 17% | 45% | 4% | 9% | 4% | 28% | 11% | 17% |
Assess for readiness and identify barriers and facilitators | 310% | 24% | 31% | 30% | 16% | 36% | 35% | 34% | 13% | 14% | 13% | 12% | 15% | 38% |
Alter incentive/ allowance structures | 305% | 28% | 0% | 7% | 44% | 18% | 22% | 10% | 71% | 32% | 17% | 32% | 7% | 17% |
Conduct local consensus discussions | 287% | 24% | 31% | 7% | 4% | 14% | 43% | 41% | 8% | 27% | 0% | 20% | 26% | 42% |
Inform local opinion leaders | 261% | 28% | 15% | 13% | 12% | 14% | 39% | 3% | 17% | 18% | 0% | 28% | 44% | 29% |
Access new funding | 226% | 10% | 0% | 3% | 72% | 5% | 0% | 3% | 38% | 9% | 78% | 0% | 4% | 4% |
Tailor strategies | 218% | 17% | 35% | 27% | 12% | 18% | 13% | 38% | 17% | 5% | 9% | 8% | 4% | 17% |
Create a learning collaborative | 218% | 7% | 23% | 33% | 8% | 18% | 9% | 14% | 13% | 5% | 9% | 28% | 19% | 33% |
Identify early adopters | 217% | 17% | 27% | 20% | 8% | 23% | 13% | 10% | 13% | 9% | 0% | 24% | 41% | 13% |
Discussion
-
Recently, more and more evidence has become available for the relative advantage of implementing PROMs [42, 43]. We incorporate this information in the training to clinicians (step 4 in Fig. 2) and in the information we send to interested stakeholders to overcome this barrier. This might also affect the barrier tension for change.
-
To overcome the barrier of structural characteristics, creating awareness within the board of hospitals to facilitate larger scale implementation can be an opportunity. This might also affect the barrier leadership engagement.
-
Regarding engaging key stakeholders, patients and patient associations should be more involved in e.g., selecting PROs and PROMs and choices regarding frequency (step 1 in Fig. 2).