Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks

Highlights

• An automatic coronary calcium scoring method in coronary CT angiography is proposed.

• The method uses paired convolutional neural networks to identify calcified voxels.

• The method allows fast and accurate coronary calcium detection in CT angiography.

• Ensembles of paired convolutional networks outperformed individual models.

• Automatic coronary calcium scoring in CTA allows cardiovascular risk stratification.

Abstract

The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular events. CAC is clinically quantified in cardiac calcium scoring CT (CSCT), but it has been shown that cardiac CT angiography (CCTA) may also be used for this purpose. We present a method for automatic CAC quantification in CCTA. This method uses supervised learning to directly identify and quantify CAC without a need for coronary artery extraction commonly used in existing methods.

The study included cardiac CT exams of 250 patients for whom both a CCTA and a CSCT scan were available. To restrict the volume-of-interest for analysis, a bounding box around the heart is automatically determined. The bounding box detection algorithm employs a combination of three ConvNets, where each detects the heart in a different orthogonal plane (axial, sagittal, coronal). These ConvNets were trained using 50 cardiac CT exams. In the remaining 200 exams, a reference standard for CAC was defined in CSCT and CCTA. Out of these, 100 CCTA scans were used for training, and the remaining 100 for evaluation of a voxel classification method for CAC identification. The method uses ConvPairs, pairs of convolutional neural networks (ConvNets). The first ConvNet in a pair identifies voxels likely to be CAC, thereby discarding the majority of non-CAC-like voxels such as lung and fatty tissue. The identified CAC-like voxels are further classified by the second ConvNet in the pair, which distinguishes between CAC and CAC-like negatives. Given the different task of each ConvNet, they share their architecture, but not their weights. Input patches are either 2.5D or 3D. The ConvNets are purely convolutional, i.e. no pooling layers are present and fully connected layers are implemented as convolutions, thereby allowing efficient voxel classification.

The performance of individual 2.5D and 3D ConvPairs with input sizes of 15 and 25 voxels, as well as the performance of ensembles of these ConvPairs, were evaluated by a comparison with reference annotations in CCTA and CSCT. In all cases, ensembles of ConvPairs outperformed their individual members. The best performing individual ConvPair detected 72% of lesions in the test set, with on average 0.85 false positive (FP) errors per scan. The best performing ensemble combined all ConvPairs and obtained a sensitivity of 71% at 0.48 FP errors per scan. For this ensemble, agreement with the reference mass score in CSCT was excellent (ICC 0.944 [0.918–0.962]). Aditionally, based on the Agatston score in CCTA, this ensemble assigned 83% of patients to the same cardiovascular risk category as reference CSCT.

In conclusion, CAC can be accurately automatically identified and quantified in CCTA using the proposed pattern recognition method. This might obviate the need to acquire a dedicated CSCT scan for CAC scoring, which is regularly acquired prior to a CCTA, and thus reduce the CT radiation dose received by patients.

Introduction

Cardiovascular disease (CVD) is the global leading cause of death. The amount of coronary artery calcification (CAC) as quantified in cardiac CT – the calcium score – is a strong and independent predictor of CVD events (Yeboah et al., 2012).

In a clinical cardiac CT exam, a calcium scoring CT (CSCT) scan and a coronary CT angiography (CCTA) scan are typically both acquired. The CCTA scan is used for stenosis detection or identification of non-calcified plaque, and the CSCT scan is used to determine the calcium score (Hecht, 2015). However, it has been shown that CAC may also be quantified in CCTA. In a study by Pavitt et al. (2014), 85% of patients with a high calcium score in CSCT also had a high calcium score in CCTA (specificity 99%). Moreover, Mylonas et al. (2014) showed excellent agreement between CVD risk categories based on calcium scoring in CCTA and categories based on calcium scoring in CSCT (Cohen’s linearly weighted $\kappa =0.93$). A recent survey reported typical radiation doses of 1 mSv for CAC scoring in CSCT (Messenger et al., 2015), while modern techniques allow CCTA acquisitions with 1.5 mSv radiation dose (Al-Mallah et al., 2014). Hence, performing calcium scoring in CCTA and omitting acquisition of the CSCT scan could reduce the radiation dose of a cardiac CT examination by 40–50% (Voros and Qian, 2012).

In clinical practice, CAC is standardly quantified in CSCT by manual identification of groups of connected voxels in the coronary artery that are above a 130 HU threshold and subsequent automatic 3D region growing (Agatston et al., 1990). This procedure is not applicable to CCTA, due to intravascular contrast material that typically enhances the arterial lumen well beyond 130 HU (Figs. 1(b) and (f)). Hence, higher global detection thresholds, ranging from 320 HU (Otton et al., 2012) to 600 HU (Glodny et al., 2009) have been proposed to emulate CAC scoring in CCTA. However, these fixed thresholds do not consider variations in lumen attenuation in CCTA, which might occur depending on protocols, scanners or contrast agents (Figs. 1(c) and (g)). This variation can be taken into account by using patient-specific or scan-specific attenuation thresholds, based on HU values taken from a ROI in the ascending aorta (Mylonas et al., 2014) or the proximal coronary arteries (Pavitt et al., 2014) (Figs. 1(d) and (h)).

Manual identification of CAC in cardiac CT requires substantial expert interaction, which makes it time-consuming and infeasible for large-scale or epidemiological studies. To overcome these limitations, (semi)-automatic calcium scoring methods have been proposed for CSCT (see e.g. Išgum et al. (2007); Kurkure et al. (2010); Shahzad et al. (2013); Wolterink et al. (2015a) and Ding et al. (2015)). Wolterink et al. (In press) provide a comparison of (semi-)automatic methods for calcium scoring in cardiac CT exams. Similarly, methods have been developed for automatic calcium scoring in CCTA. These methods typically require a (semi)-automatically extracted segmentation of the coronary arteries. Based on this segmentation, CAC has been identified as deviation from a trend line through the lumen intensity (Wesarg, Khan, Firle, 2006, Ahmed, de Graaf, Broersen, Kitslaar, Oost, Dijkstra, Bax, Reiber, Scholte, 2014), as voxels in the extracted arteries with intensities above a patient-specific HU threshold (Teßmann et al., 2011), or as deviations from a model of non-calcified artery segments (Eilot and Goldenberg, 2014). Mittal et al. (2010) did not use a model or threshold to identify CAC, but trained classifiers to identify CAC lesions along an extracted coronary artery centerline. Coronary artery tree extraction methods generally show good performance, but they have been reported to fail in patients with complex anatomy, in the distal segments of the coronary arteries, in scans with motion or noise artifacts and in scans with occlusions in the coronary arteries. In addition, severe CAC deposits affect the performance of artery extraction algorithms, restricting their applicability in CAC identification (Schaap et al., 2009). Manual correction of incorrectly segmented coronary arteries is often time-consuming and tedious.

We propose identification of CAC without initial coronary artery tree extraction. In contrast to previously proposed methods, our algorithm uses supervised learning to directly identify CAC in CCTA. Supervised learning using nearest-neighbor, SVM and randomized decision tree classifiers has been previously applied to CAC identification in CSCT (e.g. (Isgum, Prokop, Niemeijer, Viergever, van Ginneken, 2012, Wolterink, Leiner, Takx, Viergever, Isgum, 2015, Shahzad, van Walsum, Schaap, Rossi, Klein, Weustink, de Feyter, van Vliet, Niessen, 2013)). However, these methods cannot be applied in CCTA, as they classify potential CAC lesions, extracted using a clinical 130 HU threshold. In CCTA, it is non-trivial to distinguish between CAC and attenuated lumen, and the application of a predefined single detection threshold to extract potential CAC lesions is not feasible. Instead, the proposed method identifies CAC voxels to segment lesions.

CAC voxel identification in CCTA is a challenging and extremely unbalanced classification problem. The proposed algorithm therefore first limits the volume-of-interest (VOI) to a bounding box around the heart, extracted using our previously proposed algorithm (de Vos et al., 2016). Thereafter, voxels in this VOI are classified using convolutional neural networks (ConvNets). Recently, ConvNets have been successfully used in natural image classification, image segmentation and object detection. In addition, they have been used in several medical image analysis tasks, for example knee cartilage segmentation (Prasoon et al., 2013) lymph node detection (Roth et al., 2014), brain tissue segmentation (Stollenga et al., 2015), and pulmonary nodule classification (Ciompi et al., 2015). In the proposed algorithm, ConvNets automatically extract texture features from triplanar 2.5D or volumetric 3D input samples, which are combined with spatial features derived from a normalized coordinate system defined in the VOI. To classify voxels as CAC or non-CAC, a pair of ConvNets is used. These ConvNets are linked by training and together are called a ConvPair. The first ConvNet identifies voxels likely to be CAC. Such voxels are further classified by the second ConvNet, which distinguishes between CAC and CAC-like negatives. We propose a purely convolutional ConvNet architecture, which allows for fast evaluation times and can be directly applied to arbitrarily sized CCTA images. In addition, we present experiments showing that combinations of different architectures can achieve higher CAC identification performance than individual architectures.

We have previously proposed a method for CAC scoring in CCTA using a combination of a ConvNet and a Random Forest classifier (Wolterink et al., 2015b). This work extends our previous work in several ways. First, the classification procedure has been modified. Our previously proposed method used a ConvNet for voxel classification and a Random Forest classifier for lesion classification. The current method uses two sequential ConvNets for voxel classification. Second, in our previous work, candidate voxels for classification were selected based on the image intensity histogram. In the current work, we classify all voxels within the VOI, regardless of intensity, hence no assumptions are made about CAC HU values. Third, location features were previously extracted using a time-consuming elastic registration preprocessing step. In the current method, this registration step is omitted in favor of our very fast ConvNet-based bounding box detection technique (de Vos et al., 2016). Fourth, in our previous work we only evaluated triplanar 2.5D input with one input size. In the current work, we provide a comparison between 2.5D and volumetric 3D input, between input with different sizes, as well as experiments with ensembles combining these input representations. Fifth, the ConvNet architecture in our previous work required a time-consuming scan algorithm with many redundant operations for neighboring candidates. Here, we use a purely convolutional network for efficient voxel classification. Finally, in this work an evaluation on a substantially larger set of scans has been performed, and a thorough comparison with clinically used CSCT CAC scores, as well as interobserver variability, are provided.