A New Risk Stratification Score for Patients with Suspected Cardiac Chest Pain in Emergency Departments, Based on Machine Learning
Chest pain is one of the most frequent complaints among patients presenting to emergency departments (EDs) worldwide. Accurately stratifying the risk of acute coronary syndrome (ACS) in these patients is critical for effective clinical decision-making and resource allocation. Traditional risk stratification tools such as the Thrombolysis in Myocardial Infarction (TIMI) score, the Global Registry for Acute Coronary Events (GRACE) score, the Banach score, and the HEART score have been widely used in clinical practice. Among these, the HEART score has demonstrated superior performance in predicting 7-day major adverse cardiac events (MACE) in previous studies, with a C-statistic of 0.731. However, the advent of machine learning (ML) algorithms offers the potential to develop more accurate and efficient risk stratification models. This study aimed to develop and evaluate ML-based models for predicting 7-day MACE in patients with suspected cardiac chest pain, comparing their performance to the HEART score.
Study Design and Methodology
This study was a retrospective observational cohort analysis based on data from a prospective observational study. Patients were recruited from two hospitals: the Prince of Wales Hospital (PWH) in Hong Kong and the Second Affiliated Hospital of Guangzhou Medical University (AHGZMU) in Guangzhou, China. The recruitment periods spanned from May 2012 to March 2013 at PWH and from March 2012 to August 2013 at AHGZMU. The study included patients aged 18 years or older who presented to the ED with chest pain or discomfort of possible cardiac origin. Exclusion criteria included non-Chinese patients, those with a clear non-cardiac cause of chest pain, and those with confirmed ST-segment elevation myocardial infarction (STEMI), as these patients did not have undifferentiated chest pain.
A total of 1,274 eligible patients were identified, of which 418 were excluded due to unwillingness to participate, missing onset time, inability to provide consent, or non-cardiac chest pain. This left 856 patients for inclusion in the study. Of these, 833 completed the 7-day follow-up. Data on subsequent ED visits, hospital readmissions for chest pain evaluation, and cardiac procedures were obtained from the Clinical Management System (CMS) at PWH and the Health Insurance Information Management System (HIIMS) at AHGZMU. These data were further confirmed through phone interviews conducted 7 days after the initial presentation.
The dataset was divided into training and testing sets. Data from 583 patients (70%) were used to develop the classification models, while data from 250 patients (30%) were reserved for evaluating the models’ prognostic performance. Three ML algorithms were employed: eXtreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), and Logistic Regression (LR). The performance of these models was compared to the HEART score using receiver operating characteristic (ROC) curve analysis.
Key Findings
The study found that the XGBoost model outperformed the other ML algorithms and the HEART score in predicting 7-day MACE. The area under the ROC curve (AUC) for XGBoost was 0.822 (95% confidence interval [CI]: 0.769 to 0.868), which was significantly higher than the AUCs for SVM (0.649, 95% CI: 0.586 to 0.708), LR (0.667, 95% CI: 0.605 to 0.725), and the HEART score (0.702, 95% CI: 0.641 to 0.758). The differences in AUCs between XGBoost and the other models were statistically significant, with p-values of 0.002, 0.001, and 0.098 for SVM, LR, and HEART, respectively.
The XGBoost algorithm identified troponin, gender, and creatinine as the three most important features for predicting 7-day MACE. This finding aligns with the known clinical significance of these variables in cardiac risk assessment. Troponin, a biomarker of myocardial injury, is a cornerstone of ACS diagnosis. Gender differences in cardiovascular disease presentation and outcomes are well-documented, while creatinine levels reflect renal function, which is closely linked to cardiovascular health.
Strengths and Clinical Implications
The primary strength of this study lies in the development of a novel risk stratification model using the XGBoost algorithm, which demonstrated superior prognostic performance compared to traditional ML models and the HEART score. XGBoost’s ability to automatically and efficiently summarize rules from medical data allows for a more comprehensive analysis, incorporating all variables from raw data. This capability is particularly advantageous in the ED setting, where rapid and accurate risk stratification is essential for triage and clinical decision-making.
Another significant advantage of the XGBoost model is its feasibility for implementation in the ED. All key variables required for the model, including symptoms, signs, and blood biomarkers, can be obtained within two hours of patient presentation. This makes the model a practical tool for emergency staff, enabling them to predict clinical outcomes and make informed decisions about triage classification and patient management.
Limitations and Future Directions
While the study’s findings are promising, several limitations should be acknowledged. First, the study was conducted in a specific population of Chinese patients, which may limit the generalizability of the results to other ethnic groups. Second, the retrospective nature of the study introduces the potential for bias, particularly in data collection and patient selection. Third, the study focused on short-term outcomes (7-day MACE), and further research is needed to evaluate the model’s performance in predicting long-term outcomes.
Future studies should aim to validate the XGBoost model in larger, more diverse populations and explore its integration into clinical decision support systems. Additionally, the model could be enhanced by incorporating other relevant variables, such as imaging data and patient comorbidities, to further improve its predictive accuracy.
Conclusion
In conclusion, this study demonstrates that the XGBoost algorithm is a powerful tool for predicting 7-day MACE in patients presenting to the ED with chest pain. Its superior performance compared to SVM, LR, and the HEART score highlights the potential of ML algorithms to revolutionize risk stratification in emergency medicine. By enabling rapid and accurate risk assessment, the XGBoost model can assist emergency staff in making timely and informed decisions, ultimately improving patient outcomes.
doi.org/10.1097/CM9.0000000000000725
Was this helpful?
0 / 0