(A-5) Predicting ICU death and readmission risks using publicly available dataset

Introduction:: Introduction: In intensive care units (ICUs), critically ill patients are treated and kept under control. Thanks to the developments in artificial clinical decision-making mechanisms, determining the probability of patients dying in the intensive care unit, giving priority to the most risky patients, are all within reach. In this study, using a popular, public ICU data collection, we experiment with two main tasks: how can well we perform the binary tasks conducted in the literature (i.e., predict ICU death, predict ICU readmission), and compare the binary results to its multiclass classification version to find what makes this a difficult task.


Materials and Methods:: Materials and Methods: We chose the public MIMIC-III dataset collected from the hospital Beth Israel Deaconess Medical Center in Boston, Massachusetts. To model the patients’ status in ICU, we used the APACHE scoring method used by healthcare professionals. We removed the patients with less than an acceptable threshold of APACHE. We conducted feature importance evaluations using logistic regression on 5-fold cross-validation framework and determined the most impactful set of features. Next, over the selected top one hundred features, we compared logistic regression (LR), support vector machines, XGBoost and various multilayer perceptrons.


Results, Conclusions, and Discussions:: Results: To determine the performance of the experimented machine learning (ML) methods, we used weighted F1 score as the basis, as it takes the class imbalance into account, and performed 5-fold cross-validation experiments. With that, predicting ICU deaths within 48 hours of initial admission is possible with F1 score of 86.2% with 0.65% standard deviation (SD) using LR, while the other ML methods returned similar results. Separately, when we experiment with patients re-admitted to ICU, we find that we can predict ICU re-admission within 2 days of discharge with 86.2% F1 score and 0.65% SD. When we increase these number of days to 30 days, we find the same results from 5-fold cross-validation. When we convert this task into a multiclass classification problem where we try to classify if a patient will die within 48 hours of ICU admission or get readmitted within 30 days after discharge, or will recover and not return within 3 months of discharge, the task becomes more difficult and the results change abruptly. F1 score when we use the same method and cross-validation framework becomes 66.7% F1 score and 0.47% SD. When we study the confusion between classifications, we find that most confusion is between the readmission and recovery classes, while death classification is alone always classified above 70%.



Conclusions: Our experiments showed on MIMIC-III dataset that it is possible to predict ICU readmission or ICU death separately while solely using some APACHE scores as features within a machine learning framework. These results alone show that such models can help healthcare workers to partially automate the patient status evaluation procedure, saving time and cost. Meanwhile, the initial results of the multiclass classification shows that more feature engineering and different modeling techniques are required to improve the results and to better attend to ICU patients. To overcome that, we plan to develop different frameworks and consider including other features as well, so we can handle the confusions between classes.




Acknowledgements (Optional): :

References (Optional): : Hsieh, M.H., Hsieh, M.J., Chen, CM. et al. Comparison of machine learning models for the prediction of mortality of patients with unplanned extubation in intensive care units. Sci Rep 8, 17116 (2018)