Performance Evaluation of CCTA-AI for Coronary CT Angiography

0
0
0

Performance Evaluation of Deep Learning-Based Post-Processing and Diagnostic Reporting System for Coronary CT Angiography: A Clinical Comparative Study

Coronary computed tomography angiography (CCTA) has emerged as a cornerstone for non-invasive diagnosis of coronary artery disease (CAD), offering detailed anatomical insights, particularly stenosis grading. However, the post-processing and interpretation of CCTA data remain labor-intensive, requiring cardiovascular imaging professionals to manually analyze images, segment coronary arteries, and quantify stenosis. In China, this process averages 30 minutes per case, creating bottlenecks in clinical workflows. To address these challenges, this study introduces CCTA-AI, a deep learning-based system designed to automate post-processing and diagnostic reporting. The research evaluates two critical aspects: the efficiency gains offered by CCTA-AI and its diagnostic accuracy compared to conventional methods.

Development and Training of CCTA-AI

The CCTA-AI system was trained on a dataset of 10,410 CCTA cases collected from 18 hospitals, divided into training (7,287 cases), tuning (2,082 cases), and validation (1,041 cases) sets. A multi-tiered annotation process ensured high-quality training data:

  1. Initial Quality Control: Cases with poor image quality were excluded by junior graders (>2 years of experience).
  2. Detailed Annotation: Senior graders (>5 years of experience) labeled the aorta, coronary arteries, and atherosclerotic plaques.
  3. Final Verification: Expert graders (>10 years of experience) validated annotations for accuracy.

The algorithm integrated convolutional neural networks (CNNs) for vessel segmentation, plaque detection, and stenosis quantification. The system processes raw CCTA data into structured reports, including 3D reconstructions, curved planar reformats, and stenosis percentages, within a unified interface (Figure 1).

Clinical Validation: Methodology

Study Cohorts

  • Diagnostic Accuracy Cohort: 335 patients underwent both CCTA and invasive coronary angiography (ICA) within six months. ICA served as the reference standard.
  • Time Efficiency Cohort: 350 consecutive CCTA cases were analyzed to compare processing durations between manual and AI workflows.

Imaging Protocols

CCTA scans were performed using three CT scanners:

  1. GE Revolution 256-row CT
  2. Philips Brilliance 128-row CT
  3. GE LightSpeed VCT 64-row CT

Parameters included prospective ECG-gating, 0.625 mm slice thickness, and iodinated contrast injection (5–6 mL/s depending on patient weight). ICA was conducted using a GE Inova 2100 system, with quantitative coronary angiography (QCA) software for stenosis measurement.

Comparison Workflows

  • Conventional Workflow: Eight radiographers performed post-processing (vessel segmentation, plaque labeling), followed by stenosis grading by four radiologists.
  • CCTA-AI Workflow: Fully automated post-processing and stenosis quantification without human intervention.

Analytical Metrics

  • Time Efficiency: Post-processing durations for both workflows.
  • Diagnostic Performance: Sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and area under the receiver operating characteristic curve (AUC) for detecting ≥50% and ≥70% stenosis at patient-, vessel-, and segment-levels. Coronary segments were classified using the Society of Cardiovascular Computed Tomography 18-segment model.

Results

Time Efficiency

CCTA-AI reduced post-processing time dramatically:

  • Manual Workflow: 160 ± 100 seconds per case.
  • CCTA-AI: 40 ± 900 seconds per case (P < 0.0001).
    The AI system successfully processed 99.7% of cases, failing only in one instance due to an anomalous coronary artery origin unrecognized during training.

Diagnostic Accuracy

Patient-Level Analysis
  • Sensitivity:
    • ≥50% Stenosis: CCTA-AI = 89.3% vs. Professionals = 82.6% (P = 0.019).
    • ≥70% Stenosis: CCTA-AI = 72.4% vs. Professionals = 62.0% (P = 0.038).
  • Specificity:
    • ≥50% Stenosis: CCTA-AI = 55.9% vs. Professionals = 71.2% (P = 0.006).
    • ≥70% Stenosis: CCTA-AI = 64.5% vs. Professionals = 85.5% (P < 0.00001).
  • AUC:
    • ≥50% Stenosis: CCTA-AI = 0.76 (95% CI: 0.70–0.81) vs. Professionals = 0.84 (P = 0.002).
Vessel-Level Analysis
  • Sensitivity:
    • ≥50% Stenosis: CCTA-AI = 74.6% vs. Professionals = 76.8% (P = 0.061).
    • ≥70% Stenosis: CCTA-AI = 54.7% vs. Professionals = 57.1% (P = 0.615).
  • Specificity:
    • ≥50% Stenosis: CCTA-AI = 80.8% vs. Professionals = 90.0% (P < 0.0001).
    • ≥70% Stenosis: CCTA-AI = 87.8% vs. Professionals = 95.4% (P < 0.0001).
Segment-Level Analysis
  • NPV:
    • ≥50% Stenosis: CCTA-AI = 93.3% vs. Professionals = 95.6% (P = 0.003).
    • ≥70% Stenosis: CCTA-AI = 97.6% vs. Professionals = 98.2% (P = 0.074).
  • Specificity:
    • ≥50% Stenosis: CCTA-AI = 87.2% vs. Professionals = 93.7% (P < 0.0001).

Discussion

Efficiency and Workflow Integration

CCTA-AI’s 75% reduction in processing time addresses critical inefficiencies in high-volume settings. Its compatibility with CT scanners from multiple vendors (GE, Philips) underscores robustness, enabling broad clinical adoption.

Diagnostic Performance

While CCTA-AI demonstrated superior sensitivity over professionals, its lower specificity highlights a trade-off. The high NPV (88.8–97.6%) suggests utility in ruling out significant stenosis, potentially streamlining triage. At the patient level, CCTA-AI’s AUC (0.76) matched preliminary radiologist assessments but lagged behind arbitrated results, indicating AI’s role as an adjunct rather than a replacement.

Challenges and Limitations

  1. Segment-Level Discrepancies: Lower specificity at the segment level may stem from mismatches between AI’s rule-based segmentation and subjective human judgments.
  2. Calcium Artifacts: Heavy calcifications, excluded during analysis, remain a hurdle for automated systems.
  3. Anomalous Anatomy: Failure in one case underscores the need for expanded training data on rare coronary variants.

Clinical Implications

In resource-constrained environments, CCTA-AI could prioritize cases requiring urgent review, reduce radiologist workloads, and accelerate reporting. Its quantitative outputs may also mitigate inter-observer variability, a persistent issue in stenosis grading.

Conclusion

This study validates CCTA-AI as a transformative tool for CCTA workflows, offering significant time savings and reliable stenosis detection. While diagnostic accuracy approaches radiologist performance, its lower specificity and segment-level limitations necessitate human oversight. Future iterations could enhance specificity through advanced plaque characterization and integration with hemodynamic data.

doi.org/10.1097/CM9.0000000000001913

Was this helpful?

0 / 0