Performance of a simplified HEART score and HEART-GP score for evaluating chest pain in urgent primary care

This study involved a retrospective, observational cohort of consecutive patients (≥18 years) evaluated for chest pain at a large regional primary care facility in Alkmaar, the Netherlands in 2017. The facility is responsible for out-of-office-hours urgent primary care for 245,000 inhabitants. Evaluation involved anamnesis, physical examination, and 12-lead electrocardiogram (ECG), at the discretion of the treating physician. Follow-up information was obtained from electronic health records from the GP, and outpatient, admission or discharge notes from the ED/hospital.

The simplified HEART score consists of: history, ECG, age, and risk factors. For the HEART-GP score a fifth element is added, which is based on the GP’s sense of alarm, as shown in Tab. 1. For interpretation of the history element, we relied on the approach previously reported by Mahler et al. [11, 12]. In their study the history element depends on balancing low- and high-risk features. We presumed the absence of a high-risk symptom when such a feature was not recorded in the electronic health records by the treating physician.

The primary outcome of interest is the occurrence of a major adverse cardiac event (MACE) occurring within 6 weeks of initial contact with the GP. MACE is defined as a composite consisting of death from any cause, acute coronary syndrome (ACS), or coronary revascularisation.

Study personnel visited the out-of-office-hours primary care facility as well as the affiliated primary care practices in the Alkmaar region to collect baseline and follow-up information from electronic health records. Baseline data included sex, age, medical history, and use of relevant medications. Data were collected and processed using a secure, web-based, electronic data capture platform (Castor EDC, Amsterdam, The Netherlands). Further information on the methodology used for data collection can be found in a methodology paper published previously by our group [16].

We expressed diagnostic accuracy for the simplified HEART and HEART-GP scores for detecting 6‑week MACEs at various thresholds as sensitivity, specificity, accuracy, positive and negative predictive values (PPV, NPV), with 95% confidence intervals (CI). We displayed the overall discriminatory properties using C‑statistics.

During the study period, a total of 770 patients were evaluated by a GP for chest pain. We had to exclude data from 83 of these patients who objected to sharing medical data for research purposes (in the wake of the introduction of new European data protection regulations). Of the remaining patients, we could not obtain follow-up information on 23 (3.3%), which left us with a study population of 664 patients. The baseline characteristics of these patients are shown in Tab. 2. Overall, the median age was 48 years, and 56.9% were female. Risk factors for cardiovascular disease were common (39.8%), of which hypertension (25.5%) had the highest prevalence. Symptom characteristics were also different, with MACE cases more often having heavy/pressure-type chest pain with radiation, nausea and diaphoresis, and less often localised pain that is reproducible by palpation.

A total of 32 (4.8%) patients suffered a MACE within the first 6 weeks after consultation (Fig. 1). Of those 6 died (5 from cardiovascular causes), 6 patients had an ST-segment elevation myocardial infarction, 14 non-ST-segment elevation myocardial infarction, 4 unstable angina, and 2 patients underwent coronary revascularisation. Apart from MACEs, there were also 10 cases of heart failure, 7 cases of pulmonary embolism, and 1 patient with a (non-fatal) aortic dissection who underwent supracoronary aortic replacement surgery. A complete list of events can be found in the Electronic Supplementary Material (Table S1).

After initial evaluation, GPs urgently referred a total of 157 (23.6%) patients to the (cardiac) ED, 74 by ambulance and 83 with self-transportation. Of those, a total of 26 had a MACE within 6 weeks (PPV 16.6%, 95% CI 13.7–19.9%). A total of 6 patients were not referred but still had a MACE within 6 weeks (NPV 98.8%, 95% CI 97.6–99.4%). The sensitivity and specificity were 81.3%, 95% CI 63.6–92.8% and 79.3%, 95% CI 75.9–82.4%, respectively.

The distribution of the simplified HEART and HEART-GP scores and the occurrence of MACEs can be found in Fig. 2. Overall, the occurrence of MACEs was rare in those patients with a low score on the simplified HEART (1/346 = 0.29% for score ≤1) or HEART-GP (1/371 = 0.27% for score ≤2), and increased to 75% in those with the highest documented simplified HEART score (=6/8 points) or HEART-GP score (=8/10 points), respectively. When assessing the individual components, patient history, ECG abnormalities, age, and risk factors were all associated with MACEs (Electronic Supplementary Material, Table S2). As shown in Fig. 3, the simplified HEART and HEART-GP scores had C‑statistics of 0.86, 95% CI 0.80–0.91 and 0.90, 95% CI 0.85–0.95, respectively. The diagnostic performance of the simplified HEART and HEART-GP scores at various thresholds (1–5) is summarised in Tab. 3. In short, the NPV was at or above 99% when applying referral thresholds of 3 points (or lower) for the simplified HEART score and 4 points (or lower) for the HEART-GP score, respectively. The number of false-negative cases remained low (≤5 cases) when applying a threshold of ≤3 points for the simplified HEART score, or ≤4 points for the HEART-GP score.

We found a lower number of missed MACEs when using a simplified HEART score of ≥2 points (1 missed case, 0.15%) or a HEART-GP score of ≥3 or ≥4 points (1 (0.15%) or 3 (0.45%) missed cases) as a referral threshold, instead of unassisted physician assessment (6 missed cases (=0.90%)). This improved safety comes at the expense of additional referrals. For a simplified HEART score of ≥2 points this would lead to 175 (332 vs 157, 50.0% vs 23.6%, p < 0.001) additional referrals when compared with physician assessment. For the HEART-GP score, a threshold of ≥3 points would lead to a total of 136 additional patient referrals (293 vs 157, 44.1% vs 23.6%, p < 0.001). For a HEART-GP score of ≥4 points there would be 29 additional referrals (186 vs 157, 28.0% vs 23.6%, p = 0.08). Finally, when comparing unaided physician performance with a high-threshold referral strategy, such as a HEART-GP score of ≥5 points, we would see fewer referrals (110 vs 157, p < 0.001), but also more missed cases (9 vs 6).

Our study involved the clinical presentation and clinical course of consecutive patients with chest pain in urgent primary care, which curtails the risk of selection bias. The study involved a relatively large number of patients and was conducted in a large-scale urgent primary care centre, involving over a hundred GPs, and is therefore likely a representative sample. Prior studies have found that particularly the history element is prone to subjective interpretation. To minimise this heterogeneity, we applied a rigorous approach in which we scored high- and low-risk features as previously described by Mahler et al. [11]. These assessments were made by experienced investigators who were blinded as to the final diagnosis and/or outcome. The limitations of the study are as follows: the study was retrospective in nature, and we presumed absence of a symptom when a symptom or other element was not recorded by the treating physician. The number of MACEs is limited, and we can therefore not rule out a certain degree of imprecision in regard to the diagnostic performance of the studied risk scores. Another limitation is selective clinical work-up and follow-up, which may have led to verification bias. Finally, a mentionable number of GPs refused to provide follow-up data of their patients because of the ‘opt-out-plus’ design of the study, or expressed liability concerns due to the recently implemented European data protection regulations.

Previously, our group conducted a survey among ≈300 GPs to establish what they would perceive as an acceptable rate for missed MACEs among patients who present with acute-onset chest pain [8]. Most GPs would be willing to accept missing 0.5–2.5% of cases, while at the same time keeping the referral threshold to a maximum of 50 ‘unnecessary’ referrals for each ACS case. Based on our study, the simplified HEART score would likely not be of added value. A threshold of ≥2 points would result in too many referrals, whereas a threshold of ≥3 points would not lead to a substantial reduction in the number of missed cases. The HEART-GP score seems more promising, either using a threshold of ≥3 points (higher referral rate, but very low rate of missed cases), or ≥4 points (29 additional referrals and 3 fewer missed MACEs).

A number of studies have been conducted to construct a clinical decision rule over the past three decades. In the late 1990s Grijseels et al. developed a decision aid for ruling out ACS in general practice [17]. Risk assessment in this aid was based on ECG parameters and high-risk features (male sex, past medical history of coronary artery disease) and symptoms (presence of radiation of pain and/or nausea/sweating). This score was recalibrated by Bruins Slot et al. in 2011 [18]. These studies showed mediocre discriminatory properties (C-statistic 0.66–0.72), and unaided clinical judgement provided a better overall fit (C-statistic of 0.75), with poor agreement in risk estimation (in half of cases) [6, 17, 18]. Recently, a 2-week flash-mob study was performed among Dutch GPs in which the Marburg Heart Score was evaluated for its properties for ruling out ACS in patients referred for suspected ACS [19]. Overall, the diagnostic properties in terms of predictive values of the Marburg Heart Score, as for the other risk assessment tools, were not superior to unaided GP assessment.

In order to uncover the full potential of the HEART score, or other risk scores, the availability of a reliable point-of-care (POC) troponin test is pivotal [6]. In the pre-hospital (ambulance) setting the use of troponin resulted in an improved performance of the HEART score (C-statistic of 0.74 vs 0.65) [20]. The ambulance-based ATTICA trial is now evaluating whether patients with a low HEART score (including troponin) could be safely deferred to primary care [21]. An urgent primary care study that evaluated the HEART score (URGENT) was terminated prematurely, as the POC troponin was retracted (and sold) by the manufacturer [22]. Overall, 37 cases could be analysed, of which 10 were referred (4 cases of ACS), and 1 case of ACS was missed (among 27 non-referred patients). The missed case was the result of a breach in protocol. Seen in this light, the findings of this pilot study are promising, and future efforts to evaluate the HEART score should be encouraged when a reliable, time-efficient, POC troponin test becomes available. Based on the findings of our study, the HEART score should perhaps be modified to also include the GP’s sense of alarm.