Pre-test probability for coronary artery disease in patients with chest pain based on machine learning techniques

https://doi.org/10.1016/j.ijcard.2023.05.041Get rights and content

Highlights

  • We derived a high-performance model for the ML-PTP that can present the CAD risk according to the individual's risk assessment.

  • Compared to CAG results, the performance range of ML-PTPs had C-statistics of 0.795 to 0.984 before CAG in patients with chest pain.

  • ML-PTPs were adjusted to have 99% sensitivity for true CAD not to miss CAD patients. In the testing dataset, the best accuracy of the ML-PTPs was 92.8%.

Abstracts

Background

A correct and prompt diagnosis of coronary artery disease (CAD) is a crucial component of disease management to reduce the risk of death and improve the quality of life in patients with CAD. Currently, the American College of Cardiology (ACC)/American Heart Association (AHA) and the European Society of Cardiology (ESC) guidelines recommend selecting an appropriate pre-diagnosis test for an individual patient according to the CAD probability. The purpose of this study was to develop a practical pre-test probability (PTP) for obstructive CAD in patients with chest pain using machine learning (ML); also, the performance of ML-PTP for CAD is compared to the final result of coronary angiography (CAG).

Methods

We used a database from a single-center, prospective, all-comer registry designed to reflect real-world practice since 2004. All subjects underwent invasive CAG at Korea University Guro Hospital in Seoul, South Korea. We used logistic regression algorithms, random forest (RF), supporting vector machine, and K-nearest neighbor classification for the ML models. The dataset was divided into two consecutive sets according to the registration period to validate the ML models. ML training for PTP and internal validation used the first dataset registered between 2004 and 2012 (8631 patients). The second dataset registered between 2013 and 2014 (1546 patients) was used for external validation. The primary endpoint was obstructive CAD. Obstructive CAD was defined as having a stenosis diameter of >70% on the quantitative CAG of the main epicardial coronary artery.

Results

We derived an ML-based model consisting of three different models according to the subject used to obtain the information, such as the patient himself (dataset 1), the community's first medical center (dataset 2), and doctors (dataset 3). The performance range of the ML-PTP models as the non-invasive test had C-statistics of 0.795 to 0.984 compared to the result of invasive testing via CAG in patients with chest pain. The training ML-PTP models were adjusted to have 99% sensitivity for CAD so as not to miss actual CAD patients. In the testing dataset, the best accuracy of the ML-PTP model was 45.7% using dataset 1, 47.2% using dataset 2, and 92.8% using dataset 3 and the RF algorithm. The CAD prediction sensitivity was 99.0%, 99.0%, and 98.0%, respectively.

Conclusion

We successfully developed a high-performance model of ML-PTP for CAD which is expected to reduce the need for non-invasive tests in chest pain. However, since this PTP model is derived from data of a single medical center, multicenter verification is required to use it as a PTP recommended by the major American societies and the ESC.

Introduction

Obstructive coronary artery disease (CAD) is one of the representative cardiovascular diseases that could lead to sudden death. [[1], [2], [3], [4], [5]] Therefore, a correct and prompt diagnosis of CAD is a crucial component of disease management to reduce the risk of death and improve the quality of life in CAD patients. [2,[5], [6], [7]] Coronary angiography (CAG) and cardiac computed tomography (CT) are the standard tests for the diagnosis of CAD. CAG, an invasive test for CAD diagnosis, is the gold-standard method for CAD diagnosis but has a risk of complications such as access site bleeding. [8] Also, cardiac CT, a non-invasive imaging test, cannot be performed in an emergency. It can also incur high medical costs for patients who do not have CAD. [8] So, the American College of Cardiology (ACC)/American Heart Association (AHA) and the European Society of Cardiology (ESC) guidelines recommend selecting an appropriate pre-diagnosis test for an individual patient according to the CAD probability. [2] Using the pre-diagnosis test to stratify the obstructive CAD risk could be potentially beneficial. Evaluating an individual's pretest probability (PTP) for CAD makes it possible to select patients with chest pain and a high risk of CAD who might benefit the most from a diagnostic procedure such as CAG or cardiac CT. [2,8] Also, it can be used to stratify the risk of CAD among an unselected cohort of patients hospitalized for chest pain. In the present study, we developed and evaluated a model of an individual's PTP for CAD using machine learning (ML) techniques and limited medical information for diagnosing CAD. ML is a collection of technologies that allow artificial intelligence to learn complex rules and identify patterns in multi-dimensional datasets without explicit programming or applying prior assumptions. [7,[9], [10], [11]] Medical big data used for learning can merge many doctors' experiences and ML as the pattern analyzer can efficiently classify high-risk CAD patients requiring percutaneous coronary intervention. [10,11] This study aimed to develop a practical PTP for obstructive CAD in patients with chest pain using ML. The performance of ML-PTP for CAD is compared to the final result of CAG. We used trained physician diagnoses and CAG results obtained from patients with chest pain. Therefore, we assume that these high-performance ML-PTP models for CAD could promptly and correctly classify CAD in a limited diagnostic situation, raising awareness of CAD in patients with chest pain and assisting the decision-making process for doctors.

Section snippets

Study population

The design of the dataset used in this study has been described in previous studies. [12,13] We have used a single-center, prospective, all-comer registry designed to reflect real-world practice since 2004. [12,13] All patients who underwent invasive CAG at Korea University Guro Hospital (KUGH) in Seoul, South Korea, were prospectively enrolled in the KUGH-percutaneous coronary intervention and vasospastic angina registry. A total 10,177 patients with typical or atypical chest pain underwent

Results

The first ML-PTP training and internal validation dataset allocated a total of 8631 subjects [male = 4745 (54.9%), 58.6 ± 12.5 years] who all underwent CAG between 2004 and 2012. The second dataset for external validation for the performance of ML-PTP allocated 1546 subjects [male = 837 (54.1%), 59.6 ± 12.2 years] who underwent CAG between 2013 and 2014 (Fig. 1, Supplement 1).

Obstructive CAD, the primary endpoint of the study, was diagnosed after CAG in a total of 706 (42.9%) subjects in the

Discussion

In this study, we derived a high-performance model for the ML-PTP that can present the CAD risk according to the individual's risk assessment. An ML-PTP with stable performance was derived through internal and external verification, and the performance of ML-PTP was verified using an independent dataset. 1) We derived an ML-based model consisting of three different models according to the subject used to obtain the information, such as chest pain patients (dataset 1), community first medical

Disclosure

The authors have no financial conflicts of interest relevant to the manuscript to disclose.

Sources of funding

This research was supported by a Hanyang University Research Grant.

Acknowledgment

Yung-Kyun Noh and Byoung Geol Choi is partly supported by the National Research Foundation of Korea Grant (NRF/MSIT2017R1E1A1A03070945) and Hanyang University (HY-2019). Seung-Woon Rha is partly supported by Korea University Guro Hospital ‘KOREA RESEARCH-DRIVEN HOSPITALS’ Grant (O1905411).

References (24)

  • Organization WH

    Cardiovascular Diseases (CVDs)

    (2017)
  • M.S. Bittencourt et al.

    European society of cardiology-recommended coronary artery disease consortium pretest probability scores more accurately predict obstructive coronary disease and cardiovascular events than the diamond and forrester score: the partners registry

    Circulation.

    (2016)
  • 1

    First authors: Byoung Geol Choi and Ji Young Park.

    View full text