BACKGROUND

Risk prediction tools aid physicians to objectively evaluate the probability of coronary artery disease (CAD) among patients presenting with chest pain. Such decision support is particularly useful at a clinical setting where the actual disease prevalence is low, such as at the primary healthcare setting1.

The pre-test probability of CAD reflects a continuum of risk. When reaching a shared decision to refer a patient for further cardiac investigations, one should take into account individual risk appetite and also consider the trade-offs, namely between correctly diagnosing disease versus unnecessary added tests in the otherwise healthy.

However, conventional methods used to evaluate and compare various prediction models, namely the discrimination and calibration statistics, are not intuitive enough to aid decision-making in routine clinical practice.

The Duke Clinical Score2 (DCS), CAD Consortium Score3 (CCS), and Marburg Heart Score4 (MHS) are commonly used prediction models for CAD diagnosis. To date, the clinical implications of using these risk scores have not been compared in a primary care setting. It is also not known which tool is best calibrated for use in an Asian population.

OBJECTIVE

We aimed to develop and validate a diagnostic prediction model for CAD in Southeast Asians using clinical parameters readily available in primary care, and to compare the performance and clinical utility of three existing prediction tools (DCS, CCS, and MHS) against our new model.

METHODS

We report our study in accordance with the TRIPOD statement5.

Study Sites

Singapore is an urbanized island-state in Southeast Asia with a multi-ethnic population of 5.7 million6. SingHealth Polyclinics (SHP) is a provider of subsidized primary healthcare services. Its network of eight polyclinics covered 1.8 million attendances in 20187. Its affiliate, the National Heart Centre Singapore (NHCS), is the largest local tertiary referral center for cardiovascular (CV) diseases.

Participants

A prospective cohort study was conducted on consecutive patients who attended all SHP branch clinics for chest pain. They were stable clinically and were subsequently referred for cardiac evaluation at NHCS between July 2013 and December 2016. Those with (a) existing or prior history of CAD, (b) acute coronary syndrome such as unstable angina and evolving acute myocardial infarction, and (c) age below 30 years were excluded. Ethical approval was obtained from SingHealth Centralised Institutional Review Board (CIRB2018/2851). All participants provided informed consent.

Study Procedure

Patients completed an interviewer-administered questionnaire and underwent resting electrocardiogram (ECG). Electronic medical records (EMR) were accessed to determine clinical history and laboratory test results. Patients without investigations in the preceding year had fasting blood tests taken upon enrolment to determine their lipid and glucose levels.

The patient and his attending doctors (primary care physician and cardiologist) were blinded to the CAD pre-test probability (PTP) results, computed using the various models tested. All subsequent cardiac investigations at NHCS were determined at the clinical discretion of the reviewing cardiologist.

Definition of Predictors

Chest pain was classified clinically as typical, atypical, or non-anginal8. Patients indicated on a diagram the region of their chest discomfort. To determine if chest pain was reproducible upon palpation, a trained investigator palpated the same area in which the discomfort was reported.

Diagnoses of type 2 diabetes mellitus (T2DM), hypertension, or dyslipidemia were retrieved from the EMR. Newly diagnosed T2DM or dyslipidemia was also based on fasting blood glucose ≥ 7 mmol/L (126 mg/dL) or fasting total cholesterol ≥ 5.2 mmol/L (200 mg/dL) respectively. Smoking was defined as current (tobacco product use within the last 6 months), former, or never. Ethnicity was self-reported and categorized as Chinese, Malay, Indian, or “others.”

ECGs were reviewed by an investigator who was blinded to the case records. Results were classified according to the Minnesota code (Appendix in the Supplementary Information).

Outcome Measures

The primary outcome was diagnosis of significant CAD, defined as (a) ≥ 70% luminal stenosis of at least one major coronary artery or ≥ 50% left main stenosis (based on either catheter-based or CT coronary angiography), or (b) clinical diagnosis of CAD in patients without coronary angiography. All clinical diagnoses were independently adjudicated by an investigator who was blinded to the diagnosis of the attending cardiologist. Discrepancies in diagnoses were arbitrated independently by another cardiologist in the study team.

At 1 year of follow-up, matching was done at the respective national registries (Appendix in the Supplementary Information) for mortality and major adverse cardiovascular events (MACE). MACE includes non-fatal myocardial infarction (MI), non-fatal stroke, and coronary revascularization (coronary artery bypass grafting and/or percutaneous coronary intervention). Data on revascularization was obtained from EMR and phone interviews conducted using standardized scripts.

Risk Instruments Tested

The Duke Clinical Score (DCS) predicts critical ischemia (≥ 75% coronary stenosis, or significant left main stenosis). The score was derived from a single tertiary center cohort of patients presenting with chest pain, and who underwent cardiac catheterization between 1969 and 19792.

The CAD Consortium Score (CCS) was developed from a pre-existing database, sourced from a consortium of 18 hospitals across Europe and the USA. It comprises basic (CCS-basic) and clinical (CCS-clinical) models, which can be used in a stepwise manner to predict risk as more clinical information unfolds. CAD was defined as ≥ 50% luminal stenosis3.

The Marburg Heart Score (MHS) was derived from a multicenter primary care cohort in Germany. Patients presented to their general practitioner with chest pain. CAD status was determined by expert panel consensus at 6 months follow-up4.

Refer to details in Appendix Table 1 in the Supplementary Information.

Sample Size Calculation

We envisioned our clinical prediction rule to contain about 10 predictors. Using the rule of thumb of 10 events per predictor, 100 cases of CAD would be needed. CAD prevalence was estimated to be 10%. A 35% dropout was factored. The calculated sample size was 1350. We aimed to recruit an additional 600 patients to allow for internal validation.

Statistical Analysis

The cohort’s baseline characteristics were summarized as frequencies and percentages for categorical variables and mean ± standard deviation (SD) for continuous variables.

Model Development

We developed the PRECISE (Predictive Risk scorE for CAD In Southeast Asians with chEst pain) risk score for CAD in Southeast Asians. Independent t test and chi-square test were used to determine variables associated with CAD. Clinician input and p < 0.05 determined the variable retention in the final regression model. Odds ratios (OR) and their 95% CI were calculated. Univariate and multivariate logistic regression analyses were performed to determine the final independent predictors of CAD. We used complete-case analysis to handle missing data for predictors and outcome.

Values predicted by the PRECISE score belong to a range from 0 to 1, and represent the probability of CAD. The model is:

$$ p=\frac{1}{1+{e}^{-y}} $$

where p is the predicted probability of CAD, e is a base of natural logarithm, and y is a linear combination of variables (xi) and their estimators (bi) included in the model:

$$ y={b}_0+{b}_1{x}_1+{b}_2{x}_2+\dots +{b}_n{x}_n $$

For binary predictors (e.g., dyslipidemia status), xi = 1 if present, and 0 if absent. For ternary predictors (e.g., type of chest pain, in which categories are mutually exclusive), if xi = 1 (e.g., typical pain), then xi + 1 = 0 (e.g., atypical pain).

We created two independent risk calculators:

  • PRECISE-S, a simple model comprising demographic and clinical variables, and

  • PRECISE-C, a clinical model that included additional variables from resting ECG.

Model Discrimination, Validation, and Calibration

Model discrimination (the ability to differentiate between those with and without disease) was measured by the AUC and its 95% confidence interval (95% CI).

When evaluating an association within any given data set, the apparent strength of that association may be overestimated because of idiosyncrasies of the data. To correct for this over-optimism (so-called internal validation), bootstrapping9 was used as a resampling technique.

Here, 829 (50%) new data sets were used by randomly sampling selected subjects from the main data set, with replacement. Next, stepwise multivariable logistic regression analysis was performed on each of these 829 data sets, by considering the models with significant univariable p values (p < 0.1). Internal bootstrap validation (829 bootstrap samples) was used to provide optimism-corrected estimates. It was applied to each of the imputed data sets. The optimism is the decrease in model performance between the bootstrap and the original samples, which can adjust the developed model for overfitting. The corrected calibration slope was used as a shrinkage factor for the regression coefficients, and AUC with 95% CI corrected for over-optimism was estimated.

Model calibration (the agreement between predicted and observed outcomes) was expressed graphically, with observed risks plotted on the y-axis against predicted risks on the x-axis. The corresponding calibration intercept and slope were calculated. The calibration slope evaluates the spread of the estimated risks and has a target value of 1. The calibration intercept is an assessment of calibration-in-the-large, and has a target value of 0. Perfect calibration shows predictions lying on the 45° line of the calibration plot (i.e., a slope of 1 and intercept of 0).

Performance of Comparator Models

We quantified the predictive performance of DCS, CCS (basic and clinical models), and MHS using the original equations (as published)2,3,4. The respective AUC and calibration plots were presented.

Risk Stratification

We stratified the cohort into low, intermediate, and high CAD risk groups, using empirical risk thresholds of 5% and 50% respectively. In clinical practice, patients at low risk are managed expectantly, and those at intermediate risk should be referred for further cardiac investigations, while those at high risk may receive invasive diagnostic tests (e.g., cardiac angiography) from the outset10.

To offer insights to the use of each predictive model in clinical practice, we performed reclassification analysis and calculated the net benefit.

Reclassification Analysis

Reclassification was performed by cross-tabulating the probability classification of patients using PRECISE-C against other risk models. Reclassification was regarded correct if the predicted probability using PRECISE-C was closer to the observed probability of CAD compared to DCS, or CCS-clinical.

Net reclassification improvement11 (NRI) quantifies changes in risk classification when using different models.

$$ \mathrm{NRI}={\mathrm{NRI}}_{\mathrm{case}}+{\mathrm{NRI}}_{\mathrm{non}-\mathrm{case}} $$

where

NRIcase:

= P (up | case) − P (down | case)

NRInon-case:

= P (down | non-case) − P (up | non-case)

NRI reports differences in proportions of patients moving “up” and “down” for cases and non-cases separately. “Up” refers to moving to a higher risk category. “Down” refers to moving to a lower risk category. “Case” refers to patient with CAD; “Non-case” refers to patient without CAD. NRI indicates changes to a patient’s risk category across the stipulated cutoffs, which translates into modification in treatment recommendations. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. In conclusion, NRI values were applied to compare the reclassification capacities of the various models across our pre-determined risk thresholds.

Net Benefit Analysis

We used a threshold probability (Pt) of 5% as the cutoff between low- and intermediate-risk groups. For any chosen threshold probability, there is an associated number of “true positives” and “false positives.” The trade-off between benefits (of detecting CAD among “true positives”) versus harm (of unnecessary cardiac tests among “false positives”) is expressed in terms of net benefit (NB)12. The unit of NB is “true positive.”

$$ \mathrm{Net}\ \mathrm{Benefit}=\frac{\mathrm{True}\ \mathrm{Positives}\hbox{--} \mathrm{False}\ \mathrm{Positives}}{N}\times \frac{Pt}{1\hbox{--} Pt} $$

We calculated and compared NB for each risk model. The model with the highest NB demonstrates the highest clinical value.

Statistical analysis was performed using IBM SPSS version 25.0 and SAS version 9.4. A p value of < 0.05 was taken to be statistically significant.

RESULTS

Participants

A total of 1858 patients were recruited, of which 1658 had complete outcome data (mean age 56.7 ± 11.1 years, 809 males, 1374 Chinese). Among those excluded were 179 who had dropped out and 21 who were withdrawn.

Prevalence of CAD in our cohort was 9.5% (n = 158). Eighty-six (54.4%) patients had evidence of stenosis on catheter-based angiography, 2 (1.3%) had CT evidence of stenosis, 55 (34.8%) had positive stress test and deemed to have clinically significant CAD, and 14 (8.9%) were diagnosed clinically to have CAD by their cardiologist and started on treatment.

Demographic characteristics are summarized in Table 1. The participant flow diagram is presented in Figure 1. Figure 2 summarizes cardiac investigations at specialist follow-up.

Table 1 Demographics and Clinical Characteristics of Study Population
Full size table
Fig. 1
figure 1

Flow diagram of patients from recruitment until 1-year follow-up.

Full size image
Fig. 2
figure 2

Diagnostic evaluation of patients for coronary artery disease.

Full size image

1-Year Follow-up

At 1 year, 1 (0.6%) out of the 158 patients classified as “CAD positive” had died of CV cause, and 81 (51.3%) developed MACE. In comparison, only 1 (0.1%) out of the 1500 patients classified as “CAD negative” had died of CV cause, and 13 (0.9%) developed MACE. Of the 179 dropouts, 1 (0.6%) died of non-CV cause and 1 (0.6%) developed MACE.

Predictors of Coronary Artery Disease

After multivariable analysis, significant predictors of CAD were older age (OR 1.03 (1.02–1.05), p = 0.001), male (OR 5.75 (3.63–9.11), p < 0.001), diabetes mellitus (OR 1.82 (1.20–2.76), p = 0.005), hypertension (OR 1.64 (1.12–2.42), p = 0.012), smoker (OR 2.08 (1.30–3.34), p = 0.002), typical chest pain (OR 3.95 (2.40–6.50), p < 0.001), chest pain radiating to neck (OR 3.18 (1.50–6.74), p = 0.003), Q waves on ECG (OR 2.77 (1.54–5.01), p = 0.001), and ST-T changes on ECG (OR 1.74 (1.07–2.83), p = 0.027) (Table 2).

Table 2 Univariate and Multivariate Predictors of Coronary Artery Disease
Full size table

The final equation for the simple model (PRECISE-S) is:

$$ y=-6.632+\left(\ 0.035\ast \mathrm{Age}\ \right)+\left(\ 1.694\ast \mathrm{Male}\ \right)+\left(\ 0.613\ast \mathrm{Diabetes}\ \right)+\left(\ 0.542\ast \mathrm{Hypertension}\ \right)+\left(\ 0.791\ast \mathrm{Smoker}\ \right)+\left(\ 0.063\ast \mathrm{ExSmoker}\ \right)+\left(\ 1.395\ast \mathrm{Typical}\ \mathrm{Pain}\ \right)+\left(\ 0.877\ast \mathrm{Atypical}\ \mathrm{Pain}\ \right)+\left(\ 1.143\ast \mathrm{Pain}\ \mathrm{Radiating}\ \mathrm{to}\ \mathrm{Neck}\ \right) $$

The final equation for the clinical model (PRECISE-C) with resting ECG parameters is:

$$ y=-6.714+\left(\ 0.033\ast \mathrm{Age}\ \right)+\left(\ 1.75\ast \mathrm{Male}\ \right)+\left(\ 0.597\ast \mathrm{Diabetes}\ \right)+\left(\ 0.497\ast \mathrm{Hypertension}\ \right)+\left(\ 0.733\ast \mathrm{Smoker}\ \right)+\left(\ 0.07\ast \mathrm{ExSmoker}\ \right)+\left(\ 1.374\ast \mathrm{Typical}\ \mathrm{Pain}\ \right)+\left(\ 0.875\ast \mathrm{Atypical}\ \mathrm{Pain}\ \right)+\left(\ 1.157\ast \mathrm{Pain}\ \mathrm{Radiating}\ \mathrm{to}\ \mathrm{Neck}\ \right)+\left(\ 1.020\ast \mathrm{Q}\ \mathrm{waves}\ \mathrm{present}\ \right)+\left(\ 0.552\ast \mathrm{ST}-\mathrm{T}\ \mathrm{changes}\ \mathrm{present}\ \right) $$

The prediction models are available as an online probability calculator. (https://webapps.duke-nus.edu.sg/tools/PRECISE; Appendix Figure 1 in the Supplementary Information)

Performance of Risk Scores

Discrimination and Validation

AUCs for PRECISE-S and PRECISE-C are 0.808 (95% CI = 0.776–0.840) and 0.815 (95% CI = 0.782–0.847) respectively. In the bootstrapped validation cohort, PRECISE-S and PRECISE-C resulted in AUC of 0.825 (95% CI = 0.782–0.868) and 0.841 (95% CI = 0.799–0.883) respectively.

AUCs for DCS, CCS-basic and CCS-clinical models, and MHS were 0.795 (95% CI = 0.759–0.831), 0.756 (95% CI = 0.717–0.794), 0.787 (95% CI = 0.752–0.823), and 0.661 (95% CI = 0.621–0.701) respectively (Table 3).

Table 3 Comparison of Performance of Risk Scores
Full size table

Calibration

The calibration intercept and slope are 0.025 and 0.503 for PRECISE-S, and − 0.044 and 2.00 for PRECISE-C. The calibration intercept and slope were − 0.037 and 0.313 for DCS, 0.014 and 0.400 for CCS-basic, and 0.013 and 0.382 for CCS-clinical (Fig. 3a–e).

Fig. 3
figure 3

Calibration plots for a PRECISE: simple model, b PRECISE: clinical model, c CAD Consortium Score: basic model, d CAD Consortium Score: clinical model, and e Duke Clinical Score.

Full size image

Pre-test Probability Scores

Using DCS, 60.9% of patients were classified as intermediate risk, for which guidelines recommend further cardiac investigations10. Using the CCS-basic and CCS-clinical, 76.5% and 70.2% of patients were grouped as intermediate risk respectively. In comparison, the use of PRECISE-S resulted in 51.0% of patients being classified as intermediate risk, while 47.8% were classified as low risk. PTP ranged between 0 and 67% with PRECISE-S. With PRECISE-C, 48.8% of patients were classified as intermediate risk and 49.8% were classified as low risk. PTP ranged between 0 and 78% with PRECISE-C (Appendix Table 2 in the Supplementary Information).

Reclassification Analysis

73.1% of patients were classified into a different risk category when PRECISE-C was used instead of DCS. 32.3% of patients were classified into a different risk category when PRECISE-C was used instead of CCS-clinical. The NRIcase and NRInon-case for PRECISE-C versus DCS were − 75.0% and 72.9% respectively. The NRIcase and NRInon-case for PRECISE-C versus CCS-clinical were − 26.7% and 30.4% respectively (Table 4; Fig. 4; Appendix Tables 3 and 4 in the Supplementary Information).

Table 4 Net Reclassification Improvement of CAD Pre-test Risk for PRECISE: Clinical model vs Duke Clinical Score and CAD Consortium Score: Clinical model
Full size table
Fig. 4
figure 4

Reclassification flow chart. a Duke Clinical Score vs PRECISE: clinical model. b CAD Consortium Score: clinical vs PRECISE: clinical model.

Full size image

Net Benefit Analysis

At the pre-determined threshold probability of 5%, NB of PRECISE-S and PRECISE-C was 0.061 and 0.063 respectively. Taking PRECISE-C as an example for illustration: The primary care physician is willing to refer 20 “at-risk” patients for tertiary evaluation in order to find 1 patient with CAD (i.e., 5% threshold probability). He decides to use PRECISE-C as a clinical decision support tool to identify patients with ≥ 5% PTP of CAD for referral. With the aid of PRECISE-C, he refers a total of 1000 patients for tertiary evaluation, out of which a net of 63 patients are “true positive” for CAD (i.e., a net of 1 “true positive” out of every 16 patients referred).

In comparison, NB for DCS, CCS-basic, and CCS-clinical was 0.056, 0.060, and 0.065 respectively.

DISCUSSION

We developed and validated a clinical decision support tool (PRECISE: Predictive Risk scorE for CAD In Southeast Asians with chEst pain) for the diagnosis of CAD in Southeast Asians presenting with chest pain. Our methods conformed to the TRIPOD5 reporting framework. We also evaluated the performance of existing CAD risk calculators in our population.

The American College of Cardiology/American Heart Association13 and European Society of Cardiology10 guidelines strongly recommend PTP calculation to select at-risk patients for further investigations. Patients with low pre-test risk do not benefit from routine additional testing, while those with intermediate pre-test risk are most likely to benefit from an initial non-invasive test.

PRECISE has demonstrated good discriminatory ability (C-statistic > 0.8 for both PRECISE-S and PRECISE-C), and these results were consistent upon internal validation (bootstrap C-statistic > 0.8 for both models).

Overall disease prevalence can affect the probability of CAD. Latest ESC guidelines publish PTP estimates (based on age, gender, and chest pain type) at approximately one-third of those reported in the 2013 guidelines, in part due to lower CAD prevalence in recent cohorts14,15,16. For our cohort using PRECISE-C, PTP ranged between 0 and 78%, comparable to data from Foldyna et al.14 The latter found a PTP range of between 2 and 48% in patients with stable chest pain referred for non-invasive testing. Their prevalence of CAD was 13.9% (defined as ≥ 50% coronary stenosis) and 6.1% (if defined as ≥ 70% coronary stenosis), similar to the CAD prevalence of 9.5% in our study. Our results also mirror the CAD prevalence (10.5–11.2%) reported on populations in Europe and the USA17.

External Validation of Comparator Models

Based on our results, the DCS, CCS-basic, and CCS-clinical models demonstrate acceptable discrimination (C-statistic 0.795, 0.756, and 0.787 respectively) in our population, while MHS shows poor discrimination (C-statistic 0.661). However, DCS, CCS-basic, and CCS-clinical models are poorly calibrated for Southeast Asians. The calibration plots for the abovementioned models show that they tend to systematically over-estimate the probability of CAD in our primary care population. Over-estimation of PTP will result in increased and unnecessary cardiac investigations.

Prevalence of CAD in the tertiary-based Duke cohort was 66%2. Hence, it is likely that DCS would over-estimate the CV risks18 of patients in the community. Indeed, Cheng et al.16 externally validated DCS in a contemporary population (2003–2010) who had undergone CT angiography. They observed a substantially lower CAD prevalence than what was predicted by DCS (10% actual vs 42% predicted risk of CAD, defined as 70% coronary stenosis).

The predictive accuracy of CCS was found to be good, when externally validated in predominantly white populations19. In our study, although CCS demonstrated acceptable discrimination, it was found to be poorly calibrated for Southeast Asians.

CAD prevalence among Germans in the MHS4 cohort was 15%. Although MHS was developed from a primary care population, it showed poor discriminatory value (C-statistic 0.661) when validated in our Southeast Asian setting. The Marburg Heart study was limited by incomplete tertiary evaluation for the entire study population, resulting in uncertainty in those labeled as “CAD negative.” Furthermore, differing magnitudes and interaction of genetic, environmental, and cultural, as well as socioeconomic risk factors for CAD may result in differences between Western and Asian cohorts20, 21.

Clinical Implications

When PRECISE was used (instead of CCS-clinical or DCS), a greater proportion of patients were classified as low CAD risk (defined empirically as < 5%); 49.8% were classified as low risk using PRECISE-C, versus 22.9% and 6.3% respectively when CCS-clinical or DCS was used. These findings are consistent with the ESC 201910 report of PTP over-estimation by existing risk models.

A 5% risk threshold was chosen as the cutoff between low- and intermediate-risk groups. At the primary care level, patients with low PTP (< 5% risk) are managed expectantly, while those at intermediate (or higher) risk should be referred for further investigations and management at the tertiary setting. In patients with low CAD risk, deferment of routine investigations can reduce healthcare cost, and has also been shown to be safe (< 1% annual risk of CV death or MI)14, 15.

We quantified the clinical impact of choosing PRECISE over DCS or CCS-clinical using NRI and NB. This would make it more intuitive for the practicing clinician to judge the implications of choosing one model over another.

Reclassification analysis showed that PRECISE-C was able to correctly reclassify 100% of patients compared to CCS-clinical. The bulk of these were reclassified to a lower risk category (Appendix Table 4b in the Supplementary Information), likely due to lower CAD prevalence in our cohort (9.5% vs 29% in CCS cohort)3, and lower diagnostic thresholds used in the CCS cohort (≥ 50% coronary stenosis vs ≥ 70% coronary stenosis or ≥ 50% left main stenosis). Our threshold was selected as it better corresponds to myocardial ischemia.

At the pre-determined threshold probability of 5%, both PRECISE-S and PRECISE-C demonstrate superior NB when compared with DCS and CCS-basic. However, CCS-clinical was found to have higher NB than PRECISE-C; We calculated a net of 65 “true positives” out of every 1000 patients referred for tertiary evaluation (with CCS-clinical), versus a net of 63 “true positives” out of every 1000 patients (with PRECISE-C).

Clinical Significance

Patients presenting with chest pain need to be managed safely, and in a timely and objective manner. Although clinical judgment still remains critical, PRECISE can provide useful clinical decision support. With the use of PRECISE CAD risk calculator, patients at intermediate CAD risk can be referred for routine cardiac evaluation, while those at high CAD risk can be given earlier access to a specialist. Finally, patients assessed to have low CAD risk can continue to be managed expectantly at the primary care level.

PRECISE calculator incorporates readily available clinical variables and aligns to current ACC and ESC guidelines10, 13. Substantial literature evidence underpins the independent associations between variables in this model and presence of CAD20, 22,23,24. PRECISE assesses CAD risk with or without ECG data. This broadens its utility in resource-poor primary care settings within Southeast Asia, where ECG facilities or expertise in ECG interpretation may be lacking.

Other proofs of value in using PRECISE, such as cost-savings and patient safety, can be evaluated in a randomized controlled trial.

Limitations

CAD diagnosis was based on variable but valid investigations in this real-world pragmatic study. Although only 11.6% of all patients underwent coronary angiogram, it is noteworthy that a further 68.6% received stress imaging and 8.4% underwent exercise treadmill testing. Our findings reflect the real-world setting, where the majority of patients are in the intermediate PTP strata, and hence undergo stepwise assessment: first with non-invasive functional testing, followed by invasive coronary angiography if needed. Non-invasive functional tests for ischemia typically have better rule-in power10 for CAD than CT coronary angiogram among patients with intermediate PTP.

Our independent adjudication of the CAD diagnosis was a deliberate measure to reduce risk of misclassification. Furthermore, the composite 1-year mortality and MACE among those classified as “CAD negative” was verified to be very low (< 1%).

As expected of a primary care population, CAD prevalence in our study was low. Due to the relatively small number of “CAD positive” patients (n = 158), only internal validation was performed. PRECISE is well calibrated to predict CAD at lower levels of risk (< 20% CAD risk). However, at higher CAD risk levels, there is poor concordance between the predicted and observed risks. These results reflect the inherent nature of our primary care patients.

The inclusion of only patients who were referred to cardiology for further evaluation may have led to selection bias. Bias may also have been introduced by missing data for predictors and outcome (Appendix Table 5 in the Supplementary Information). Another limitation is that our calibration method (i.e., calculation of the calibration intercept and slope) is considered a “Level 2” method in the calibration hierarchy25.

PRECISE has been developed for use in primary care practice. In high prevalence settings, our model may underestimate the actual CAD risk. Further external validation studies are needed to calibrate the performance of PRECISE for use in higher prevalence settings (e.g., outpatient clinic of a tertiary hospital).

This study focused mainly on Chinese, Malay, and Indian ethnic groups. While it did not cover other Asian ethnicities, these aforementioned ethnic groups represent significant proportions of populations living in Malaysia, the vast Indonesian archipelago, minority groups in Thailand, and Indochina in Southeast Asia and beyond to mainland China and India.

CONCLUSIONS

Existing chest pain risk scores have limitations when applied in an Asian population. The novel PRECISE score, based on readily available clinical information, has been developed and validated to predict CAD in Southeast Asians who report of chest pain. PRECISE demonstrates clinical benefits when used as a decision support tool at the primary care setting.

SUMMARY BOX

What is already known on this topic

  • Current ACC/AHA and ESC guidelines for stable coronary artery disease strongly recommend the calculation of pre-test probability to select at-risk patients for further investigations.

  • Existing diagnostic prediction models have been found to over-estimate the risk of coronary artery disease.

What this study adds

  • We developed the PRECISE calculator using readily available clinical variables such as age, gender, type of chest pain, underlying metabolic risk factors, and ECG data.

  • PRECISE is the first diagnostic prediction model for CAD in Southeast Asians that is designed for use in a primary care setting.