본문 바로가기
KERD
논문KCI 등재

Machine Learning for Benchmarking Critical Care Outcomes

1

조회

0

다운로드

0

인용

0

참고문헌

키워드

초록

Objectives: Enhancing critical care efficacy involves evaluating and improving system functioning. Benchmarking, a retrospectivecomparison of results against standards, aids risk-adjusted assessment and helps healthcare providers identify areasfor improvement based on observed and predicted outcomes. The last two decades have seen the development of severalmodels using machine learning (ML) for clinical outcome prediction. ML is a field of artificial intelligence focused on creatingalgorithms that enable computers to learn from and make predictions or decisions based on data. This narrative reviewcenters on key discoveries and outcomes to aid clinicians and researchers in selecting the optimal methodology for criticalcare benchmarking using ML. Methods: We used PubMed to search the literature from 2003 to 2023 regarding predictivemodels utilizing ML for mortality (592 articles), length of stay (143 articles), or mechanical ventilation (195 articles). Wesupplemented the PubMed search with Google Scholar, making sure relevant articles were included. Given the narrative style,papers in the cohort were manually curated for a comprehensive reader perspective. Results: Our report presents comparativeresults for benchmarked outcomes and emphasizes advancements in feature types, preprocessing, model selection, andvalidation. It showcases instances where ML effectively tackled critical care outcome-prediction challenges, including nonlinearrelationships, class imbalances, missing data, and documentation variability, leading to enhanced results. Conclusions:Although ML has provided novel tools to improve the benchmarking of critical care outcomes, areas that require further researchinclude class imbalance, fairness, improved calibration, generalizability, and long-term validation of published models.

Abstract

Objectives: Enhancing critical care efficacy involves evaluating and improving system functioning. Benchmarking, a retrospectivecomparison of results against standards, aids risk-adjusted assessment and helps healthcare providers identify areasfor improvement based on observed and predicted outcomes. The last two decades have seen the development of severalmodels using machine learning (ML) for clinical outcome prediction. ML is a field of artificial intelligence focused on creatingalgorithms that enable computers to learn from and make predictions or decisions based on data. This narrative reviewcenters on key discoveries and outcomes to aid clinicians and researchers in selecting the optimal methodology for criticalcare benchmarking using ML. Methods: We used PubMed to search the literature from 2003 to 2023 regarding predictivemodels utilizing ML for mortality (592 articles), length of stay (143 articles), or mechanical ventilation (195 articles). Wesupplemented the PubMed search with Google Scholar, making sure relevant articles were included. Given the narrative style,papers in the cohort were manually curated for a comprehensive reader perspective. Results: Our report presents comparativeresults for benchmarked outcomes and emphasizes advancements in feature types, preprocessing, model selection, andvalidation. It showcases instances where ML effectively tackled critical care outcome-prediction challenges, including nonlinearrelationships, class imbalances, missing data, and documentation variability, leading to enhanced results. Conclusions:Although ML has provided novel tools to improve the benchmarking of critical care outcomes, areas that require further researchinclude class imbalance, fairness, improved calibration, generalizability, and long-term validation of published models.

참고문헌 (80)

  1. Rhodes A, "Prospectively defined indicators to improve the safety and quality of care for critically ill patients : a report from the Task Force on Safety and Quality of the European Society of Intensive Care Medicine(ESICM)", Intensive Care Med, 38(4), pp.598~605 (2012)[DOI]
  2. Vincent JL, "Assessment of the worldwide burden of critical illness : the intensive care over nations(ICON)audit", Lancet Respir Med, 2(5), pp.380~386 (2014)[DOI]
  3. Higgins TL, "Quantifying risk and benchmarking performance in the adult intensive care unit", J Intensive Care Med, 22(3), pp.141~156 (2007)[DOI]
  4. Braun JP, "The German quality indicators in intensive care medicine 2013: second edition", Ger Med Sci, 11, pp.Doc09~ (2013)[DOI]
  5. Kumpf O, "Quality indicators in intensive care medicine for Germany: third edition 2017", Ger Med Sci, 15, pp.Doc10~ (2017)[DOI]
  6. Brown SE, "An empirical comparison of key statistical attributes among potential ICU quality indicators", Crit Care Med, 42(8), pp.1821~1831 (2014)[DOI]
  7. Salluh JIF, "Understanding intensive care unit benchmarking", Intensive Care Med, 43(11), pp.1703~1707 (2017)[DOI]
  8. Higgins TL, "Coronavirus disease 2019 ICU patients have higher-than-expected Acute Physiology and Chronic Health Evaluation-adjusted mortality and length of stay than viral pneumonia ICU patients", Crit Care Med, 49(7), pp.e701~e706 (2021)[DOI]
  9. Verburg IW, "Which models can I use to predict adult ICU length of stay? A systematic review", Crit Care Med, 45(2), pp.e222~e231 (2017)[DOI]
  10. van Sluisveld N, "Variation in rates of ICU readmissions and post-ICU in-hospital mortality and their association with ICU discharge practices", BMC Health Serv Res, 17(1), pp.281~ (2017)[DOI]
  11. Seneff MG, "Predicting the duration of mechanical ventilation. The importance of disease and patient characteristics", Chest, 110(2), pp.469~479 (1996)[DOI]
  12. Shashikumar SP, "Development and prospective validation of a deep learning algorithm for predicting need for mechanical ventilation", Chest, 159(6), pp.2264~2273 (2021)[DOI]
  13. Malmgren J, "Long-term health-related quality of life and burden of disease after intensive care : development of a patient-reported outcome measure", Crit Care, 25(1), pp.82~ (2021)[DOI]
  14. Higgins TL, "Outcome prediction in critical care: the Mortality Probability Models", Curr Opin Crit Care, 14(5), pp.498~505 (2008)[DOI]
  15. Ferrando-Vivas P, "Development and validation of the new ICNARC model for prediction of acute hospital mortality in adult critical care", J Crit Care, 38, pp.335~339 (2017)[DOI]
  16. Keuning BE, "Mortality prediction models in the adult critically ill: a scoping review", Acta Anaesthesiol Scand, 64(4), pp.424~442 (2020)[DOI]
  17. Strand K, "Severity scoring in the ICU : a review", Acta Anaesthesiol Scand, 52(4), pp.467~478 (2008)[DOI]
  18. Siontis GC, "Predicting death:an empirical evaluation of predictive tools for mortality", Arch Intern Med, 171(19), pp.1721~1726 (2011)[DOI]
  19. Raj R, "Machine learning-based dynamic mortality prediction after traumatic brain injury", Sci Rep, 9(1), pp.17672~ (2019)[DOI]
  20. Subudhi S, "Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19", NPJ Digit Med, 4(1), pp.87~ (2021)[DOI]
  21. Silva I, "Predicting in-hospital mortality of ICU patients : the PhysioNet", computing in cardiology challenge 2012, 2012, pp.39, 245~248
  22. Barboi C, "Comparison of severity of illness scores and artificial intelligence models that are predictive of intensive care unit mortality : meta-analysis and review of the literature", JMIR Med Inform, 10(5), pp.e35293~ (2022)[DOI]
  23. Thorsen-Meyer HC, "Dynamic and explainable machine learning prediction of mortality in patients in the intensive care unit: a retrospective study of high-frequency data in electronic patient records", Lancet Digit Health, 2(4), pp.e179~e191 (2020)[DOI]
  24. Caicedo-Torres W, "ISeeU: visually interpretable deep learning for mortality prediction inside the ICU", J Biomed Inform, 98, pp.103269~ (2019)[DOI]
  25. Aczon MD, "Continuous prediction of mortality in the PICU : a recurrent neural network model in a single-center dataset", Pediatr Crit Care Med, 22(6), pp.519~529 (2021)[DOI]
  26. "Neural document embeddings for intensive care patient mortality prediction" (2016)[URL]
  27. Ghassemi M, "A multivariate timeseries modeling approach to severity of illness assessment and forecasting in ICU with sparse, heterogeneous clinical data", Proc AAAI Conf Artif Intell, 2015, pp.446~453 (2015)
  28. Badawi O, "Readmissions and death after ICU discharge : development and validation of two predictive models", PLoS One, 7(11), pp.e48758~ (2012)[DOI]
  29. Bhattacharya S, "ICU mortality prediction : a classification algorithm for imbalanced datasets", Proc AAAI Conf Artif Intell, 31(1), pp.1288~1294 (2017)[DOI]
  30. Guo C, "A dynamic ensemble learning algorithm based on K-means for ICU mortality prediction", Appl Soft Comput, 103, pp.107166~ (2021)[DOI]
  31. Sujin Kim, "A Comparison of Intensive Care Unit Mortality Prediction Models through the Use of Data Mining Techniques", Healthcare Informatics Research, 17(4), pp.232~243 (2011), 대한의료정보학회
  32. Awad A, "Early hospital mortality prediction of intensive care unit patients using an ensemble learning approach", Int J Med Inform, 108, pp.185~195 (2017)[DOI]
  33. Pirracchio R, "Mortality prediction in intensive care units with the Super ICU Learner Algorithm(SICULA) : a population-based study", Lancet Respir Med, 3(1), pp.42~52 (2015)[DOI]
  34. Moser A, "Mortality prediction in intensive care units including premorbid functional status improved performance and internal validity", J Clin Epidemiol, 142, pp.230~241 (2022)[DOI]
  35. El-Rashidy N, "Intensive care unit mortality prediction : an improved patient-specific stacking ensemble model", IEEE Access, 8, pp.133541~133564 (2020)[DOI]
  36. Badawi O, "Evaluation of ICU risk models adapted for use as continuous markers of severity of illness throughout the ICU stay", Crit Care Med, 46(3), pp.361~367 (2018)[DOI]
  37. Chiu CC, "Predicting the mortality of ICU patients by topic model with machine-learning techniques", Healthcare(Basel), 10(6), pp.1087~ (2022)[DOI]
  38. Iwase S, "Prediction algorithm for ICU mortality and length of stay using machine learning", Sci Rep, 12(1), pp.12912~ (2022)[DOI]
  39. Pang K, "Establishment of ICU mortality risk prediction models with machine learning algorithm using MIMIC-IV database", Diagnostics(Basel), 12(5), pp.1068~ (2022)[DOI]
  40. Safaei N, "E-CatBoost : an efficient machine learning framework for predicting ICU mortality using the eICU Collaborative Research Database", PLoS One, 17(5), pp.e0262895~ (2022)[DOI]
  41. Stenwig E, "Comparative analysis of explainable machine learning prediction models for hospital mortality", BMC Med Res Methodol, 22(1), pp.53~ (2022)[DOI]
  42. Zhao S, "Improving mortality risk prediction with routine clinical data:a practical machine learning model based on eICU patients", Int J Gen Med, 16, pp.3151~3161 (2023)[DOI]
  43. Meiring C, "Optimal intensive care outcome prediction over time using machine learning", PLoS One, 13(11), pp.e0206862~ (2018)[DOI]
  44. Davoodi R, "Mortality prediction in intensive care units (ICUs) using a deep rule-based fuzzy classifier", J Biomed Inform, 79, pp.48~59 (2018)[DOI]
  45. Marafino BJ, "Validation of prediction models for critical care outcomes using natural language processing of electronic health record data", JAMA Netw Open, 1(8), pp.e185097~ (2018)[DOI]
  46. Purushotham S, "Benchmarking deep learning models on large healthcare datasets", J Biomed Inform, 83, pp.112~134 (2018)[DOI]
  47. Shapley LS, "Contributions to the theory of games (Volume II)", pp.307~318 (1953), Princeton University Press
  48. Li L, "In-hospital mortality prediction for ICU patients on large healthcare MIMIC datasets using class imbalance learning", Proceedings of 2020 5th IEEE International Conference on Big Data Analytics (ICBDA), pp.90~93 (2020)
  49. Zimmerman JE, "Intensive care unit length of stay : benchmarking based on Acute Physiology and Chronic Health Evaluation(APACHE)IV", Crit Care Med, 34(10), pp.2517~2529 (2006)[DOI]
  50. Vasilevskis EE, "Mortality probability model III and simplified acute physiology score II: assessing their value in predicting length of stay and comparison to APACHE IV", Chest, 136(1), pp.89~101 (2009)[DOI]
  51. Peres IT, "Prediction of intensive care units length of stay : a concise review", Rev Bras Ter Intensiva, 33(2), pp.183~187 (2021)[DOI]
  52. Bacchi S, "Machine learning in the prediction of medical inpatient length of stay", Intern Med J, 52(2), pp.176~185 (2022)[DOI]
  53. Harutyunyan H, "Multitask learning and benchmarking with clinical time series data", Sci Data, 6(1), pp.96~ (2019)[DOI]
  54. Houthooft R, "Predictive modelling of survival and length of stay in critically ill patients using sequential organ failure scores", Artif Intell Med, 63(3), pp.191~207 (2015)[DOI]
  55. Li C, "Prediction of length of stay on the intensive care unit based on least absolute shrinkage and selection operator", IEEE Access, 7, pp.110710~110721 (2019)[DOI]
  56. Sotoodeh M, "Improving length of stay prediction using a hidden Markov model", AMIA Jt Summits Transl Sci Proc, 2019, pp.425~434 (2019)
  57. Gentimis T, "Predicting hospital length of stay using neural networks on MIMIC III data", Proceedings of 2017 IEEE 15th International Conference on Dependable, Autonomic and Secure Computing, 15th International Conference on Pervasive Intelligence and Computing, 3rd International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC, DataCom, pp.CyberSciTech), 2017
  58. Ma X, "Length of stay prediction for ICU patients using individualized single classification algorithm", Comput Methods Programs Biomed, 186, pp.105224~ (2020)[DOI]
  59. Muhlestein WE, "Predicting inpatient length of stay after brain tumor surgery : developing machine learning ensembles to improve predictive performance", Neurosurgery, 85(3), pp.384~393 (2019)[DOI]
  60. Wu J, "Predicting prolonged length of ICU stay through machine learning", Diagnostics (Basel), 11(12), pp.2242~ (2021)[DOI]
  61. Alghatani K, "Predicting intensive care unit length of stay and mortality using patient vital signs : machine learning model development and validation", JMIR Med Inform, 9(5), pp.e21347~ (2021)[DOI]
  62. Peres IT, "Data-driven methodology to predict the ICU length of stay : a multicentre study of 99, 492 admissions in 109 Brazilian units", Anaesth Crit Care Pain Med, 41(6), pp.101142~ (2022)[DOI]
  63. Weissman GE, "Inclusion of unstructured clinical text improves early prediction of death or prolonged ICU stay", Crit Care Med, 46(7), pp.1125~1132 (2018)[DOI]
  64. Williford E, "Dealing with highly skewed hospital length of stay distributions : the use of Gamma mixture models to study delivery hospitalizations", PLoS One, 15(4), pp.e0231825~ (2020)[DOI]
  65. Peres IT, "What factors predict length of stay in the intensive care unit? Systematic review and meta-analysis", J Crit Care, 60, pp.183~194 (2020)[DOI]
  66. Sayed M, "Predicting duration of mechanical ventilation in acute respiratory distress syndrome using supervised machine learning", J Clin Med, 10(17), pp.3824~ (2021)[DOI]
  67. Kramer AA, "Variations in case-mix-adjusted duration of mechanical ventilation among ICUs", Crit Care Med, 44(6), pp.1042~1048 (2016)[DOI]
  68. Kulkarni AR, "Deep learning model to predict the need for mechanical ventilation using chest X-ray images in hospitalised patients with COVID-19", BMJ Innov, 7(2), pp.261~270 (2021)[DOI]
  69. Yu L, "Machine learning methods to predict mechanical ventilation and mortality in patients with COVID-19", PLoS One, 16(4), pp.e0249285~ (2021)[DOI]
  70. Douville NJ, "Clinically applicable approach for predicting mechanical ventilation in patients with COVID-19", Br J Anaesth, 126(3), pp.578~589 (2021)[DOI]
  71. Karri R, "Machine learning predicts the short-term requirement for invasive ventilation among Australian critically ill COVID-19 patients", Plops One, 17(10), pp.e0276509~ (2022)[DOI]
  72. Parreco J, "Using artificial intelligence to predict prolonged mechanical ventilation and tracheostomy placement", J Surg Res, 228, pp.179~187 (2018)[DOI]
  73. Sauer CM, "Systematic review and comparison of publicly available ICU data sets : a decision guide for clinicians and data scientists", Crit Care Med, 50(6), pp.e581~e588 (2022)[DOI]
  74. Liu X, "369: ICU length-of-stay models should account for the interaction between survival and patient severity", Crit Care Med, 48(1), pp.166~ (2020)[DOI]
  75. Leevy JL, "A survey on addressing high-class imbalance in big data", J Big Data, 5(1), pp.42~ (2018)[DOI]
  76. Johnson JM, "Survey on deep learning with class imbalance", J Big Data, 6(1), pp.27~ (2019)[DOI]
  77. Zimmerman JE, "Acute Physiology and Chronic Health Evaluation(APACHE)IV : hospital mortality assessment for today's critically ill patients", Crit Care Med, 34(5), pp.1297~1310 (2006)[DOI]
  78. Roosli E, "Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model", Sci Data, 9(1), pp.24~ (2022)[DOI]
  79. Xu J, "Federated learning for healthcare informatics", J Healthc Inform Res, 5(1), pp.1~19 (2021)[DOI]
  80. Lane-Fall MB, "Outcomes measures and risk adjustment", Int Anesthesiol Clin, 51(4), pp.10~21 (2013)[DOI]

관련 논문AI추천

Validation of an Automatic Tagging System for Identifying Respiratory and Hemodynamic Deterioration Events in the Intensive Care UnitAndrea Forgacs, Ari M. Lipsky, Craig M. Lilly, Danielle Jeddah, Gershon Celniker, Itai M. Pessach, Ofer Chen대한의료정보학회 · Healthcare Informatics Research2021 · 제27권 제3호Machine Learning Model for the Prediction of Hemorrhage in Intensive Care Units박철, 강소라, 윤덕용, 이진석대한의료정보학회 · Healthcare Informatics Research2022 · 제28권 제4호Advances and utility of digital twins in critical care and acute care medicine: a narrative reviewAmos Lal, Diksha M. Gowda, Gabriele A. Halpern, Marko Nemet, Oguz Kilickaya영남대학교 의과대학 · Journal of Yeungnam Medical Science2025 · 제42권 A Comparison of Intensive Care Unit Mortality Prediction Models through the Use of Data Mining Techniques김우재, 박래웅, Sujin Kim대한의료정보학회 · Healthcare Informatics Research2011 · 제17권 제4호Development of a deep learning model for predicting critical events in a pediatric intensive care unit박준동, 이봉진, 이인경대한중환자의학회 · Acute and Critical Care2024 · 제39권 제1호ChatGPT Predicts In-Hospital All-Cause Mortality for Sepsis: In-Context Learning with the Korean Sepsis Alliance Database고령은, 김종만, 박미현, 서준혁, 서지영, 오남기, 오동규, 이수연, 임채만, 정치량, 차원철, 최성규대한의료정보학회 · Healthcare Informatics Research2024 · 제30권 제3호

이 간행물의 다른 논문 (11건)