A computerised decision support system for cardiovascular risk management ‘live’ in the electronic health record environment: development, validation and implementation—the Utrecht Cardiovascular Cohort Initiative

Development of the CDSS started with defining the project and its incentive. It is well established that CDSSs developed together with clinicians are used more frequently than those developed by industry alone [6, 8]. We created an expert group comprising clinicians, data scientists, epidemiologists, ICT architects, and user experience and user interface designers. This expert group defined the scope of our pilot as a ‘cardiovascular risk management (CVRM) dashboard in the EHR environment of the University Medical Centre Utrecht, scalable to other types of EHR for use in routine clinical practice and accessible to patients’.

A project plan was drafted defining aims on timing, content, and functionalities. The CVRM dashboard should provide the treating physician with ‘live’ information in the EHR environment during the patient visit. The content comprised cardiovascular risk factors (age, sex, medical and family history, smoking, blood pressure, lipids, kidney function, glucose intolerance), estimated 10-year cardiovascular risk, guideline-compliant strategies for (non-)pharmacological treatment to optimise risk factors, and an estimated change in 10-year risk of cardiovascular disease (CVD) if treatment goals are adhered to [1, 9]. It was requested that treatment benefit be incorporated into the risk calculation in such a way that clinicians and patients could switch these prospects ‘on’ and ‘off’ to explore the effects of certain interventions. Risk assessment for CVD is calculated differently for patients with different (co-)morbidities. These risk factors, scores, and guideline-compliant strategies together formed a decision tree with business rules. In these business rules we determine which risk assessment methodology to use for patients with different co-morbidities, such as (1) with no previous cardiovascular events, HEART score; (2) with a history of one or more cardiovascular events, SMART score; (3) with diabetes, ADVANCE; and (4) elderly patients, Elderly score. Each of these methodologies defines treatment targets, thus suggestions for therapy can be generated from the profile and risk score combined. The decision tree was derived from the current cardiovascular guidelines. Possible conflicts were discussed until a consensus was reached in an expert group comprising 12 cardiovascular specialists.

Administrative support was requested, integrating a print function and a copy-to-consult function to the CDSS. Direct links to the manuscripts and guidelines were added to allow physicians easy access to sources of evidence for risk scores and treatment recommendations. Lastly, the CDSS had to be accessible via patient portals.

The technical framework of a CDSS scopes all technical aspects from data collection (and storage) to user interface. This technical blueprint comprises information on data location, location and functionalities of the final application, and connections and steps in-between. In our project, important criteria for the selection of a technical structure were (1) visualising the CDSS within the EHR and (2) sustaining scalability. Therefore, we chose a web-based application.

The practical use of the CDSS is described in Fig. 1. First, a user logs in to the EHR and opens the patient record after EHR-based authentication. By clicking on a link within the EHR, the CDSS is activated. The connection between the data within the EHR and application is established via a URL. Prior to authorising access three security checks are conducted: a single sign-on Windows authentication, user authentication via GET parameters (which guarantees that this user can access this patient data) and a patient identifier via an encrypted token.

Data on the cardiovascular risk profile are extracted from the EHR. Four structured query language (SQL) queries are used to obtain necessary information from the EHR. These four queries are aimed at questionnaire information, measurements, laboratory results, and the medication register.

During the development we used a data lake, a repository of the required anonymised data in its natural format and with enough data storage space and adequate processing speed to develop the CDSS. After the developmental phase and tests on accuracy and processing burden of the queries on the EHR, the dashboard was linked directly to the back end of the EHR, which made data storage within the data lake unnecessary. Our application calculated risk scores and provided suggestions for therapy based on predefined business rules. The application returned the complete dashboard via a viewer. When treatment suggestions were made, these were accompanied by clickable boxes that showed the risk score with or without the proposed treatment. The application calculated the risk modification due to the intervention and returned these data to the viewer via an application program interface.

The expert group decided on essentials and preferences regarding functionalities and user interface.

The user interface, the part of the CDSS the end-user finally works with, was designed taking user experience, characterised by personalisation, accessibility, compatibility and overall intuitiveness into account [10]. The expert group discussed the final interface layout, taking into consideration that personalisation of a tool increases uptake amongst end-users, but decreases scalability. Patient preferences were studied by a patient panel (n = 67).

In many CDSSs, data obtained during routine clinical practice are the main source of input data. Clinical data are generated by caregivers in the EHRs but are not primarily intended for further use in either research or feedback. CDSS quality depends on the ‘raw’ data and so ‘garbage in—garbage out’ illustrates the problem well [11].

The Centre for Circulatory Health of the University Medical Centre (UMC) Utrecht initiated the Utrecht Cardiovascular Cohort (UCC) [12]. All first-time patients visiting one of the departments of the Centre for Circulatory Health at the UMC Utrecht for the evaluation of a symptomatic vascular disease or an asymptomatic vascular condition are invited to participate. Information on the cardiovascular risk factors data set, based on Dutch CVRM Guidelines, is collected from all patients at all departments and registered in a structured format in the EHR. The detailed methodology has been published elsewhere [12]. All data used for development of our CDSS were extracted from the files of patients who gave informed consent for research including the use of their routine clinical care data.

During the dashboard development phase validation studies were conducted in a test environment, i.e. a copy of the EHRs. Validation included data completeness (i.e. defined as the proportion of information provided versus information needed) and data accuracy (i.e. defined as agreement between the value presented in the dashboard and the ‘raw’ value in the EHR) [11, 13]. These parameters were assessed for risk factor levels, risk estimates, and medication prescription. In addition, checks were performed on the suggested treatment approach given the risk factor levels and risk estimates following the 2016 CVRM guideline, and the estimates of effect when completely adherent to the suggested treatment approach (e.g. the estimated absolute risk reduction when smoking was stopped) [1, 14]. This process was repeated until the accuracy was 100%.

We set out to develop a CDSS for use in routine clinical practice in order to improve adherence to CVRM guidelines with the ultimate aim of improving patient healthcare. The planning and development started with writing a project plan to define the scope and requirements with a dedicated multidisciplinary team. This was followed by data work-up, including quality checks, development of methods to improve data completeness, and the writing of business rules to incorporate individual variables and models into decision trees. The technical design was then developed, incorporating aspects of user experience and interface design, ICT architecture and programming. Finally, implementation involved consideration of factors that influence embedding, ICT support and maintenance, sustainability, generation of scientific support, and end-user testing. Based on end-user testing, improvement of the prototype is currently ongoing.

A CDSS uses data collected in routine clinical practice. Whereas most imaging, laboratory, and several other test result data are structurally placed in predefined locations in the EHR, clinical notes are often unstructured [17]. This is an important potential source of inaccurate and/or missing data. For example, systolic pressure is found in several different fields in the EHR. This can either be solved at the source (involving many people in a hospital, calls for education, training, and feedback) or by using search-engine approaches that extract these measurements [10].

In medical practice clinical notes, usually free text, are still the cornerstones of medical reporting, as these allow flexible registration of clinical reasoning with case nuances, and the procedure is less time-consuming [18]. Unstructured text is even worse than unstructured numerical data, such as blood pressure. Spelling errors, domain-specific abbreviations, and idiosyncrasies in a complex context make free text very difficult to analyse computationally. Although text-mining algorithms are available, extensive efforts go into that approach before accurate and complete data are retrieved. Although general text-mining algorithms are available for use, the approach taken should be tailored to the specific situation at each hospital, which hampers scalability.

Missing data, irrespective of its underlying cause, prohibits use of risk-prediction algorithms. Right now, in the worst case with no data, we can impute missing data parameters with the mean from a training data set and generate the average population estimate. But this could cause a bias towards the population average. More advanced statistical approaches for imputations are readily available, but the models for imputations on an individual patient level have to be developed, validated and then integrated in the dashboard approach such that in the end the dashboard nearly always works in routine clinical practice (unless there is too much missing data), provides valid and adequate information, and also informs the treating physician which information was missing and was estimated using statistical modelling. Presently, within the CVRM dashboard project we are exploring imputation and machine-learning approaches to arrive at flexible prediction algorithms and compare the performance of the obtained models with respect to accuracy and impact on clinical decision-making [19].

Traditionally, prediction rules on cardiovascular events provide 10-year risk estimates. Yet, risk-factor interventions usually have a lifelong perspective rather than a 10-year horizon. Lifetime benefit seems a more intuitive and understandable measure. Recently, it has become possible to estimate lifetime risks and individual treatment benefits with externally validated risk scores for patients with established CVD (SMART-REACH risk score) and for patients with diabetes mellitus (DIAL score). These scores, as well as 10-year risk scores, are available at the U‑Prevent website (http://​www.​U-Prevent.​com) [9, 15, 20‐23].

Our approach to the development of a CDSS facilitates the use of any other prediction rules, e.g. prediction of 30-day mortality, 1‑year mortality, or the balance between risk of haemorrhage and reduction of ischaemic events.

Although it is plausible that an appropriate CDSS leads to improved adherence to guidelines and improved patient care, evidence of clinical benefit in this regard seems modest. A recent systematic review by Groenhof and co-workers reported on 22 studies, almost all conducted in primary care [24]. We concluded that CDSSs are related to some improvement in CVRM, but heterogeneity limits overall conclusions. Since the complexity and application of CDSSs can vary greatly, there is no one-size-fits-all approach in testing safety and effectiveness[24].

Moreover, alternative design considerations need to be taken into account for evaluation. Because of the risk of contamination in a classical randomised control trial (RCT) with patients as the randomisation unit, a cluster randomised trial seems the most common design in CDSS studies. But sample size calculations need to allow for intra-cluster correlations. Also, with heterogeneous clusters, statistical performance can be lower. A stepped-wedge cluster RCT could be a solution to this problem: this involves random and sequential cross-over of clusters from control to intervention until all clusters are exposed [25].

Lastly, other outcomes such as patient experiences—as patient empowerment was positively associated with CDSS effect—and reasons as to why clinicians deviate from the guidelines should be evaluated [24, 26].

The rapid developments in information technology also need to be addressed from legal and ethical perspectives. Currently, CDSSs are defined as a medical device in the medical device requirements (MDR) of the European Union [27]. The MDR is the prelude to the Medical Device Directive that will come into effect in 2020, indicating that medical software like a CDSS requires CE certification. The criteria for CE certification of a CDSS are multi-interpretable, but tools that provide a diagnosis, prognosis or therapy advice usually require CE certification. CE-certification processes are, however, only minimally adapted to evaluate devices such as CDSS, characterised by a combination of software, medical evidence (risk scores, other models) and medical practice. Guidance, starting with the developmental phase, by an organ specialising in CE certification, seems appropriate.

In general, CDSSs can be considered as a component of the clinical infrastructure integrating care, data analytics, scientific evidence, and clinical best practice (guidelines) [5].

This project showed a proof of concept for application of data analytics in clinical care. More advanced data analytics techniques such as probabilistic data integrity checks to automatically determine erroneous data and machine learning, potentially improving predictions by learning from complex and non-linear interactions, and previously latent auxiliary variables [28]. Also, decision-tree formulations may be suboptimal because they may assume unrealistically simple situations, whereas in practice patients have multiple problems and treatments might be based on multiple guidelines [2, 29]. More flexible decision guidance, including mitigation frameworks to allow for concurrent processing of multiple guidelines and Markov random field methods are potential solutions to this problem [29, 30]. Practical implementation and adoption of such approaches should first be explored.

If accessible for evaluations, CDSSs can provide data for evaluation of care, including benchmarking of subgroups of clinicians. Furthermore, a CDSS provides the opportunity to identify eligible patients for scientific studies, including pragmatic trials. In this way, CDSSs contribute to development of a learning healthcare system: a healthcare system where care and science are connected via a continuous cycle of data collection in care, data analytics and evaluation, interpretation and implementation [31].