Brief ReportExternal validation of the CRAX2MACE model
Introduction
Single-photon emission computed tomography myocardial perfusion imaging (SPECT MPI) is commonly used for risk stratification in patients with suspected coronary artery disease (CAD). Regional myocardial ischemia has been established as an important predictor of cardiovascular events.1 However, the prevalence of myocardial ischemia in patients undergoing SPECT imaging has diminished over time.2 As a corollary, consideration of other clinical and imaging information is increasingly important. However, to date holistic risk-prediction models, integrating ischemia with other imaging and clinical variables, have been underutilized.3
The cardiovascular risk assessment for major adverse cardiac events (MACE) at 2-year (CRAX2MACE) model incorporates several clinical and imaging variables to predict 2-year risk of MACE.4 The model can be calculated quickly with routinely available information and demonstrated excellent discrimination for MACE with area under the receiver operating characteristic curve (AUC) of 0.79 during internal validation, significantly outperforming quantitative perfusion alone.4 However, a recent study questioned the external validity of the CRAX2MACE score, demonstrating AUC of 0.612 for a composite outcome of all-cause mortality, non-fatal myocardial infarction (MI), or late coronary revascularization.5
While external validation frequently demonstrates reduced predictive performance due to differences in patient populations, the reported accuracy was significantly lower than would be expected by using quantitative perfusion assessment alone. Accordingly, we performed an external validation study of the CRAX2MACE model using data from our institution.
Section snippets
Study population
This was a retrospective study of 2,985 patients who underwent SPECT MPI as part of clinical care between September 1, 2014 and December 31, 2018 at the University of Calgary with available follow-up for MACE. CRAX2MACE was derived to predict outcomes in patients with suspected CAD as described in detail previously.4 The score was derived using a logistic regression model in 3,896 patients and validated using 1,946 patients from a single center, imaged between 2001 and 2008. Details of the
Patient population
In total, 2,985 patients were included with median age 67.4 (IQR 59.2 to 75.0) and 1,625 (54.4%) male patients. During the first 2 years of follow-up, MACE occurred in 243 (8.1%) patients (first event 49 late revascularization, 46 myocardial infarct, 148 deaths). Table 1 compares patient characteristics of patients who experienced MACE compared to those who did not.
Prediction performance
Figure 1 shows AUC for 2-year MACE. The AUC for CRAX2MACE (0.710, 95% CI 0.677-0.743) was significantly higher compared to stress
Discussion
We performed an external validation study of the CRAX2MACE model using a large retrospective cohort. We identified better discrimination for 2-year MACE with the CRAX2MACE model compared to quantitative perfusion alone. Additionally, the model demonstrated good predictive performance for hard cardiac events and good calibration between predicted and actual risk. As expected, our results demonstrate reduced accuracy compared to the original derivation study. However, the overall accuracy
New Knowledge Gained
The CRAX2MACE model demonstrates good discrimination of 2-year MACE and acceptable goodness-of-fit and calibration in an external patient population. The CRAX2MACE model is simple to apply in clinical practice and outperforms quantitative analysis alone
Conclusion
CRAX2MACE had high predictive performance for 2-year MACE in an external population. To improve the prognostic accuracy and relevance of CRAX2MACE in an era of decreasing ischemia, the model may benefit from further adaptation.
Disclosures
Dr. Miller reports research support and consulting with Pfizer. Dr. Slomka participates in software royalties for QPS software at Cedars-Sinai Medical Center. The remaining authors have no relevant disclosures.
Funding
This research was supported in part by Grant R01HL089765 from the National Heart, Lung, and Blood Institute/National Institutes of Health (NHLBI/NIH) (PI: Piotr Slomka).
References (15)
- et al.
5-Year prognostic value of quantitative versus visual MPI in subtle perfusion defects
JACC Cardiovasc Imaging
(2020) - et al.
Temporal trends in the frequency of inducible myocardial ischemia during cardiac stress testing
J Am Coll Cardiol
(2013) Prognostic value of gated myocardial perfusion SPECT
J Nucl Cardiol
(2004)- et al.
Prognostic significance of previous myocardial infarction and previous revascularization in patients undergoing SPECT MPI
Int J Cardiol
(2020) - et al.
Visually estimated coronary artery calcium score improves SPECT-MPI risk stratification
IJC Heart Vasc
(2021) - et al.
Automated quantification of myocardial perfusion SPECT using simplified normal limits
J Nucl Cardiol
(2005) - et al.
Upper reference limits of transient ischemic dilation ratio for different protocols on new-generation cadmium zinc telluride cameras: A report from REFINE SPECT Registry
J Nucl Cardiol
(2020)
Cited by (4)
Artificial intelligence-driven cardiac amyloidosis screening
2024, The Lancet Digital HealthDeep learning to automate SPECT MPI myocardial reorientation
2023, Journal of Nuclear CardiologyDeveloping a framework for evaluating and comparing risk models
2023, Journal of Nuclear CardiologyMachine learning to predict abnormal myocardial perfusion from pre-test features
2022, Journal of Nuclear Cardiology
The authors of this article have provided a PowerPoint file, available for download at SpringerLink, which summarizes the contents of the paper and is free for re-use at meetings and presentations. Search for the article DOI on SpringerLink.com.
The authors have also provided an audio summary of the article, which is available to download as ESM, or to listen to via the JNC/ASNC Podcast.