Development and validation of a model to predict cardiovascular death, nonfatal myocardial infarction, or nonfatal stroke in patients with type 2 diabetes mellitus and established atherosclerotic cardiovascular disease

Abstract

Background

Among individuals with atherosclerotic cardiovascular disease (ASCVD), type 2 diabetes mellitus (T2DM) is common and confers increased risk for morbidity and mortality. Differentiating risk is key to optimize efficiency of treatment selection. Our objective was to develop and validate a model to predict risk of major adverse cardiovascular events (MACE) comprising the first event of cardiovascular death, myocardial infarction (MI), or stroke for individuals with both T2DM and ASCVD.

Methods

Using data from the Trial Evaluating Cardiovascular Outcomes with Sitagliptin (TECOS), we used Cox proportional hazards models to predict MACE among participants with T2DM and ASCVD. All baseline covariates collected in the trial were considered for inclusion, although some were excluded immediately because of large missingness or collinearity. A full model was developed using stepwise selection in each of 25 imputed datasets, and comprised candidate variables selected in 20 of the 25 datasets. A parsimonious model with a maximum of 10 degrees of freedom was created using Cox models with least absolute shrinkage and selection operator (LASSO), where the adjusted R-square was used as criterion for selection. The model was then externally validated among a cohort of participants with similar criteria in the ACCORD (Action to Control Cardiovascular Risk in Diabetes) trial. Discrimination of both models was assessed using Harrell’s C-index and model calibration by the Greenwood-Nam-D’Agostino statistic based on 4-year event rates.

Results

Overall, 1491 (10.2%) of 14,671 participants in TECOS and 130 (9.3%) in the ACCORD validation cohort (n = 1404) had MACE over 3 years’ median follow-up. The final model included 9 characteristics (prior stroke, age, chronic kidney disease, prior MI, sex, heart failure, insulin use, atrial fibrillation, and microvascular complications). The model had moderate discrimination in both the internal and external validation samples (C-index = 0.65 and 0.61, respectively). The model was well calibrated across the risk spectrum—from a cumulative MACE rate of 6% at 4 years in the lowest risk quintile to 26% in the highest risk quintile.

Conclusion

Among patients with T2DM and prevalent ASCVD, this 9-factor risk model can quantify the risk of future ASCVD complications and inform decision making for treatments and intensity.

Introduction

Among individuals with atherosclerotic cardiovascular disease (ASCVD), type 2 diabetes mellitus (T2DM) is common and confers increased risk of morbidity and mortality [1, 2]. Similarly, patients with T2DM are nearly three times as likely to develop major adverse cardiovascular events (MACE), defined as cardiovascular (CV) death, myocardial infarction (MI), and stroke, compared with those without diabetes [3, 4]. Consequently, prevention of MACE has been a major focus of clinical trials, therapeutic strategies and clinical practice guidelines for patients with T2DM.

Therapeutic decisions among these patients are ideally driven by the underlying risk of patients for CV complications. Yet, to date, models for this risk prediction are lacking, and those available have suboptimal performance and are outdated. The UKPDS Risk Engine has demonstrated reasonable accuracy to predict the risk of ASCVD events in patients with T2DM, yet this was developed among those free of baseline heart disease or stroke and at the time of T2DM diagnosis [5]. In contrast, there has been comparatively less attention directed toward predicting CV risk in individuals with both T2DM and established ASCVD [6]. An updated UKPDS outcomes model was developed to predict secondary MI and stroke events over a median duration of 17 years [7], but the risk model was developed using simulation studies, was not externally validated, and aimed to predict risk over a long-term, near 20-year follow-up. Given that individuals with ASCVD are more than twice as likely to have MACE compared with primary prevention subgroups within 4 years, risk stratification of patients is important to inform treatment decisions [6, 8].

In this study, our aim was to develop a model to predict intermediate-term ASCVD events among individuals with both T2DM and ASCVD enrolled in the Trial Evaluating Cardiovascular Outcomes with Sitagliptin (TECOS) and externally validate the findings in participants in the ACCORD (Action to Control Cardiovascular Risk in Diabetes) trial.

Methods

Because of the sensitive nature of the data collected for this study, requests to access the dataset from qualified researchers trained in human subject confidentiality protocols may be submitted at dcri.org/data-sharing.

Study design and participants

TECOS (NCT00790205) was a double-blind, multinational, placebo-controlled trial evaluating the CV safety of adding sitagliptin to usual care in patients with T2DM and established ASCVD. Details of the trial design and primary results, as well as the study protocol and other supplementary trial resources, have been published [9, 10]. Briefly, the intention-to-treat population comprised 14,671 participants from 38 countries (662 sites) who were enrolled between December 2008 and July 2012 and followed for a median of 3.0 years. For inclusion, patients were ≥ 50 years old with T2DM, ASCVD, and HbA_1c values of 6.5–8.0%. Participants were taking insulin or oral antihyperglycemic agents at baseline with the exception of dipeptidyl peptidase-4 inhibitors, glucagon-like peptide 1 (GLP-1) receptor agonists, and rosiglitazone. Participants were excluded if they had two or more severe hypoglycemic events in the last 12 months or their baseline estimated glomerular filtration rate (eGFR) was < 30 mL/min/1.73 m². Participants were randomized 1:1 to treatment with sitagliptin or placebo with doses based on eGFR values. Concomitant medications could be adjusted during the study according to local standards of care. The trial was run jointly by the Duke Clinical Research Institute and the University of Oxford Diabetes Trials Unit in an academically independent collaboration with the sponsor, Merck Sharp and Dohme. All participants provided written informed consent, and the protocol was approved by the ethics committees at each site.

Outcome of interest

The clinical outcome for the present analyses is three-point composite MACE, defined as the time to first nonfatal MI, nonfatal stroke or CV death. Events were adjudicated by an independent clinical events committee without knowledge of treatment assignment [9].

Candidate variables

In total, 59 baseline variables were considered as candidates for model development (see Additional file 1: Supplemental Methods). Summary statistics for these variables are shown in Table 1, with the amount of nonmissing data shown in Additional file 1: Table S1. After evaluating collinearity and missingness, models were developed using 33 baseline variables. A detailed description of how the variables that were ultimately considered as candidates were chosen is provided in the Supplemental Material. In brief, variables encompassed a range of domains including demographic, anthropometric measures, medical history, laboratory values, current medication use, and diabetes complications. Variables with > 25% missingness were excluded, and the remaining variables with missing data were imputed. A single microvascular diabetic complication variable was used as a composite of blindness, amputation, foot ulcer, diabetic neuropathy or retinopathy.

Table 1 Baseline patient characteristics in the overall cohort and stratified by major adverse cardiovascular event (MACE)

Model development

Details on the model development are provided in the Additional file 1: Supplemental Materials. In sum, two Cox proportional hazards regression models for MACE were developed: (1) a “full” model that was not restricted in the number of variables; and (2) a parsimonious model intended to be easier to use as a risk prediction tool in clinical settings.

Variable selection and risk prediction

Twenty-five imputed datasets were created using SAS PROC MI with the method of fully conditional specification [11]. Variables with more than 25% missing were imputed but were not candidates for the final model. The linearity assumption of Cox regression was tested for each continuous variable, and the estimated association with the hazard of MACE was plotted using the model with restricted cubic splines. In the case of non-linearity, a cut-point was chosen based on visual inspection of the plots, and two linear splines were defined as candidates for model selection [12]. Variables selected in at least 20 of the 25 datasets were used in the final models [13]. The full model was developed using stepwise variable selection with alpha = 0.05 to enter and to stay in the model. The resulting model included all levels of categorical variables and both linear splines even when only some levels of the covariate were selected. The parsimonious model was fitted using Cox proportional hazards regression with least absolute shrinkage and selection operator (LASSO) methods [14]. A maximum of 10 degrees of freedom of only demographic and medical history variables was prespecified. The adjusted R-square statistic was used as criterion for selection.

Model evaluation

Cox proportional hazard ratios and p-values were calculated using Rubin’s method to combine the estimates from the 25 imputed datasets [13]. Discrimination of both models was assessed using Harrell’s C-index [15]. Finally, model calibration was assessed by the Greenwood-Nam-D’Agostino (GND) statistic [16] based on 4-year event rates. For model calibration and testing of modeling assumptions, only the first imputed dataset was used.

Statistical analysis

All variables that were considered for the predictive model were summarized overall and by presence of a MACE event. Continuous variables were summarized with median (25th, 75th percentile) and categorical variables with number and percentage. The numbers of nonmissing values before imputation are presented for each potential predictor. When relationships were found to be non-linear, the p-value corresponds to the model containing two piecewise linear splines. The cumulative incidence of MACE was displayed across quintiles of risk with differences assessed using the log-rank test. All analyses were performed using SAS version 9.4 and R version 3.6.3, with a p-value < 0.05 indicating significance.

External validation of the risk score

The parsimonious model risk score was externally validated in a separate cohort of individuals with T2DM and prevalent ASCVD from the ACCORD trial [17, 18]. Briefly, ACCORD was a randomized, multicenter trial that evaluated whether intensive glycemic control (target HbA1c < 6%) versus standard control (target HbA1c 7–7.9%) reduced the risk of MACE. Recruitment occurred among 77 clinical centers in the United States and Canada in two phases, between January and June 2001 and from February 2003 through October 2005. The mean follow-up was 5.0 years. Participants were aged 40–79 years with T2DM and inadequate glycemic control (HbA1c ≥ 7.5%) and either established ASCVD or aged 55–79 years with left ventricular hypertrophy, albuminuria, atherosclerosis, or two or more other ASCVD risk factors (hyperlipidemia, hypertension, obesity or current smoking). Given that the results of the primary trial were null, patients randomized to both treatment arms were included. Similar to the derivation (TECOS) cohort, participants without ASCVD at baseline, age < 50 years, HbA1c > 8%, or eGFR < 30 were excluded. The primary outcome of interest was three-point composite MACE. Each of the components (nonfatal MI, nonfatal stroke, or CV death) were adjudicated outcomes of the ACCORD trial [17]. Model evaluation was assessed similar to the primary analysis. There were no missing data in the ACCORD cohort.

Results

Baseline characteristics for the 14,671 TECOS participants in the overall cohort and stratified by first event of the MACE composite outcome are provided in Table 1. Overall, 1491 (10.2%) of the participants had a primary outcome event (MACE) at a rate of 3.6 events per 100 patient years of follow-up. The first event of the composite was CV death for 607, non-fatal MI for 578, non-fatal ischemic stroke for 283, nonfatal hemorrhagic stroke for 17, and nonfatal stroke of unknown type for 6. Participants who developed MACE were more commonly male, older, and had a higher frequency of prior MI, heart failure, atrial fibrillation, and chronic obstructive pulmonary disease (COPD). Participants who experienced MACE events also had lower eGFR and higher urine albumin-to-creatinine ratio levels at baseline.

Development and internal validation of the full model

In multivariable Cox analysis of the full model, the identified variables included age, prior stroke, prior MI, eGFR, male sex, atrial flutter or fibrillation, insulin use, microvascular diabetic complications, heart failure, New York Heart Association (NYHA) class, non-high-density lipoprotein cholesterol, diastolic blood pressure, heart rate, albuminuria, coronary artery disease, body mass index < 25 kg/m², current smoking, ≥ 50% carotid artery stenosis, dyslipidemia, and COPD (Table 2). In the internal validation cohort, the C-index for discrimination was 0.68 (95% CI 0.66–0.69). If hemorrhagic stroke were excluded as part of the composite endpoint, selected variables and the C-index would be unchanged (Additional file 1: Table S2). Calibration across deciles of predicted and observed risk is shown in Fig. 1 (GND chi-square = 7.6, p = 0.570).

Table 2 Multivariable adjusted model output parameters using variables in the extended risk score

Fig. 1

Predicted versus observed major adverse cardiovascular events (MACE) 4-year rates of the simplified and extended risk scores based on deciles of predicted risk. MACE is a composite of first nonfatal myocardial infarction, nonfatal stroke, or cardiovascular death

Development and internal validation of the simplified model

In multivariable Cox analysis of the parsimonious model, the identified variables were age, male sex, prior stroke, prior MI, chronic kidney disease (eGFR < 60), heart failure, atrial flutter or fibrillation, insulin use, and microvascular diabetic complications (Table 3). From the 9 identified covariates, we created a risk score for predicting MACE. The risk score demonstrated moderate discrimination with a C-index of 0.65 (95% CI 0.64–0.67) and acceptable calibration (GND chi-square = 11.4, p = 0.25) (Fig. 1). The cumulative incidence of MACE at 4 years increased in a graded fashion across quintiles of increasing modeled risk, from 6.0% in the lowest risk quintile to 25.8% in the highest risk quintile (log-rank p < 0.001; Fig. 2A). If the stroke portion of the composite were limited to ischemic and unknown type, model calibration and the variables selected for the simplified model would be similar (Additional file 1: Table S3).

Table 3 Multivariable adjusted model output parameters using variables in the parsimonious risk score

Fig. 2

Cumulative incidence of MACE across quintiles of the risk score in the A derivation (TECOS) and B validation (ACCORD) cohorts. MACE is a composite of first nonfatal myocardial infarction, nonfatal stroke, or cardiovascular death

External validation of the simplified model

Among 10,251 participants in the ACCORD trial, 6642 were excluded from the present analyses for non-ASCVD at baseline, 2119 for HbA1c > 8%, 72 for age < 50 years, and 14 for missing follow-up. The coefficients in Table 3 were used to calculate the risk score for each of the 1404 participants in the external validation cohort; 130 of the participants (9.3%) had MACE within 4 years of follow-up. The baseline characteristics of ACCORD participants are shown in Additional file 1: Table S4. The external validation cohort was of similar age and sex to TECOS but had a somewhat different disease profile. Both had T2DM and established ASCVD, but those in ACCORD had more prior MI, heart failure, coronary artery bypass grafting, carotid stenosis, higher HbA1c, more insulin use, and higher eGFR, whereas TECOS had higher atrial fibrillation, high-density lipoprotein cholesterol, diastolic blood pressure, heart rate, and chronic kidney disease. In discrimination analysis, the C-index of the model containing only the 9-variable risk score was 0.61 (95% CI 0.56–0.66). Further, no evidence of miscalibration was observed (GND chi-square = 9.31, p = 0.32). The cumulative incidence of MACE at 4 years increased in a graded fashion across risk-estimated quintiles, from 6.3% in quintile 1 to 15.2% in quintile 5 (log-rank p < 0.001; Fig. 2B).

Discussion

In this study, a simple novel risk score was developed and validated, and shown to have reasonable accuracy in predicting MACE events (nonfatal MI, stroke, or CV death) among patients with T2DM and ASCVD. Several important findings were observed. First, the developed risk score incorporated commonly used clinical and laboratory variables, and could identify individuals with 4-year risk of MACE ranging from 6 to 25%. Second, while the more complex model of 19 variables resulted in better discrimination, a simplified risk score of only 9 patient characteristics retained much of the operating characteristics of the larger, more complex model. Finally, the score performed well in an external cohort of patients with T2DM and ASCVD. In all, the findings suggest a novel method to identify the highest risk patients and inform targeting of ASCVD secondary prevention strategies in patients with T2DM and prevalent ASCVD.

Management of patients with T2DM has historically been focused on the achievement of glycemic targets. However, newer classes of medications—such as sodium-glucose cotransporter 2 inhibitors (SGLT2i) and GLP-1 receptor agonists—have been shown to reduce the risk of CV events in patients with T2DM and established ASCVD [19,20,21]. The mechanisms by which these drugs achieve CV benefit and whether these benefits exists across the entire spectrum of ASCVD risk, though, are unclear. Additionally, these newer agents are associated with considerable healthcare costs—especially among low-income countries [22, 23]. Thus, tools to stratify patients with T2DM and ASCVD are needed to most efficiently use these medications, especially in resource-limited settings, and guide further therapy [24]. Similarly, tools to identify individuals most at risk for MACE may help healthcare providers meet quality care metrics and allocate resources to the highest risk patients.

Current American College of Cardiology Foundation/American Heart Association guidelines recommend ASCVD risk stratification in patients with T2DM using available global risk calculators [25]. Unfortunately, existing risk calculators have not been evaluated in individuals with T2DM and above-average ASCVD risk, or specifically with prevalent ASCVD as in the present analyses. Thus, risk stratification models to identify which patients may benefit from pharmacologic interventions, such as GLP-1 receptor agonists or SGLT-2 inhibitor therapy, could be beneficial to allocate resources and guide therapeutic decision making, especially relevant for proprietary therapies for which access and cost effectiveness are key considerations. Similarly, prediction models can help inform patients about their individual risk and prognosis [26]. Results from such risk scores can be used to educate and motivate patients, promoting physical activity and lifestyle modifications. The 9-variable risk prediction tool was developed to address this knowledge gap and predict MACE among individuals with prevalent ASCVD and T2DM.

The present findings also build on previous MACE risk prediction tools from individual cohorts, including the Framingham Heart Study, Swedish National Diabetes Register (NDR), and U.K. Prospective Diabetes Study (UKPDS) cohorts [27,28,29,30]. While the Framingham risk score includes participants with prevalent ASCVD, the risk score was not designed for individuals with T2DM and has been shown to have decreased accuracy in this population [31]. Conversely, the NDR risk score is designed for individuals free of baseline ASCVD and is not generalizable to individuals with a history of ASCVD. Finally, the UKPDS Risk Engine was developed using data collected between 1977 and 1991 from individuals with newly diagnosed T2DM aged 25–65, and may not reflect the current landscape of medical practices and burden of disease seen in individuals with long-standing T2DM, especially those with established ASCVD [5]. For example, the UKPDS score does not include current medications, such as insulin use, which is incorporated in our risk score and reflects current standard-of-care treatment regimens for those with advanced T2DM [32]. Similarly, the UKPDS Risk Engine was designed to predict long-term MACE risk (up to 20+ years). By contrast, our risk score derivation included a diverse cohort of Black and White men and women with above-average short-term ASCVD risk from a recent, clinical trial-based cohort to allow for robust and personalized 4-year estimates of MACE in patients with T2DM and prevalent ASCVD.

Study strengths and limitations

The present study has several strengths, including derivation of the models in a large cohort of participants from the TECOS trial, analyses of data from patients with contemporary clinical care, a large number of events to analyze, use of advanced statistical techniques to identify and analyze variables, and validation of the models in an external cohort of patients with T2DM and ASCVD.

This study also has notable limitations. The TECOS trial was conducted between 2008 and 2012, and certain biomarkers associated with worsening CV outcomes, such as high-sensitivity troponin, natriuretic peptide, C-reactive peptide, and coronary calcium levels, were not available to assess their potential contributions to the models. However, these data are not routinely collected in individuals with stable T2DM and ASCVD. Similarly, the TECOS trial excluded individuals with severe kidney dysfunction—a known risk factor of worsening CV events. Time since prior ASCVD at baseline was not collected but could be potentially informative; however this could be variable in relation to the baseline examination of the trial. Although both populations had T2DM and ASCVD, the participant baseline comorbidities in TECOS and ACCORD occurred with different frequencies. Despite these differences, the model performed well in the validation cohort. Although defined similarly to other studies, in hindsight, the prespecified MACE endpoint for both TECOS and ACCORD studies would have been better if it did not include hemorrhagic stroke, since hemorrhagic stroke is thought to have a different risk profile than other components of the composite. However, if hemorrhagic stroke were removed from the endpoint, only 14 events (< 1%) would be lost and all predictors in the models would remain significant. Details for the models of CV death, nonfatal MI, or nonfatal ischemic stroke (without hemorrhagic stroke) are presented in Additional file 1: Tables S2 and S3. No measures of socioeconomic status (SES) were available to be considered for the model. Lower SES has been shown to be associated with poor prognosis among those with ASCVD, and a clinical trial population is likely under-representative of these higher risk, lower SES patients. Future studies are needed to test the simplified and extended risk scores in a more diverse cohort beyond clinical trial settings. Finally, validation of the extended model in the ACCORD trial was not possible as data on carotid artery stenosis, micro- versus macroalbuminuria, and COPD were not included in the trial report.

Conclusion

In analyses from 14,671 individuals with T2DM and established ASCVD, a novel 4-year risk score was developed and validated to predict MACE. Using variables commonly available in the clinical setting, the risk score helps quantify the risk of developing future adverse CV events. The 9-factor risk score may identify individuals with the highest likelihood of developing MACE who may benefit from the most aggressive targeted secondary prevention therapies and warrant treatment with more expensive therapies given an increased cost efficiency of risk reduction.