Multicenter validation of a deep-learning-based pediatric early-warning system for prediction of deterioration events
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Background: Early recognition of deterioration events is crucial to improve clinical outcomes. For this purpose, we developed a deep-learning-based pediatric early-warning system (pDEWS) and aimed to validate its clinical performance. Methods: This is a retrospective multicenter cohort study including five tertiary-care academic children’s hospitals. All pediatric patients younger than 19 years admitted to the general ward from January 2019 to December 2019 were included. Using patient electronic medical records, we evaluated the clinical performance of the pDEWS for identifying deterioration events defined as in-hospital cardiac arrest (IHCA) and unexpected general ward-to-pediatric intensive care unit transfer (UIT) within 24 hours before event occurrence. We also compared pDEWS performance to those of the modified pediatric early-warning score (PEWS) and prediction models using logistic regression (LR) and random forest (RF). Results: The study population consisted of 28,758 patients with 34 cases of IHCA and 291 cases of UIT. pDEWS showed better performance for predicting deterioration events with a larger area under the receiver operating characteristic curve, fewer false alarms, a lower mean alarm count per day, and a smaller number of cases needed to examine than the modified PEWS, LR, or RF models regardless of site, event occurrence time, age group, or sex. Conclusions: The pDEWS outperformed modified PEWS, LR, and RF models for early and accurate prediction of deterioration events regardless of clinical situation. This study demonstrated the potential of pDEWS as an efficient screening tool for efferent operation of rapid response teams.
Abstract
Background: Early recognition of deterioration events is crucial to improve clinical outcomes. For this purpose, we developed a deep-learning-based pediatric early-warning system (pDEWS) and aimed to validate its clinical performance. Methods: This is a retrospective multicenter cohort study including five tertiary-care academic children’s hospitals. All pediatric patients younger than 19 years admitted to the general ward from January 2019 to December 2019 were included. Using patient electronic medical records, we evaluated the clinical performance of the pDEWS for identifying deterioration events defined as in-hospital cardiac arrest (IHCA) and unexpected general ward-to-pediatric intensive care unit transfer (UIT) within 24 hours before event occurrence. We also compared pDEWS performance to those of the modified pediatric early-warning score (PEWS) and prediction models using logistic regression (LR) and random forest (RF). Results: The study population consisted of 28,758 patients with 34 cases of IHCA and 291 cases of UIT. pDEWS showed better performance for predicting deterioration events with a larger area under the receiver operating characteristic curve, fewer false alarms, a lower mean alarm count per day, and a smaller number of cases needed to examine than the modified PEWS, LR, or RF models regardless of site, event occurrence time, age group, or sex. Conclusions: The pDEWS outperformed modified PEWS, LR, and RF models for early and accurate prediction of deterioration events regardless of clinical situation. This study demonstrated the potential of pDEWS as an efficient screening tool for efferent operation of rapid response teams.
참고문헌 (39)
- Agulnik A, "Cost-benefit analysis of implementing a pediatric early warning system at a pediatric oncology hospital in a low-middle income country", Cancer, 125, pp.4052~4058 (2019)
- Bonafide CP, "Cost-benefit analysis of a medical emergency team in a children’s hospital", Pediatrics, 134, pp.235~241 (2014)
- de Groot JF, "Implementing paediatric early warning scores systems in the Netherlands : future implications", BMC Pediatr, 18, pp.128~ (2018)
- Sambeeck SJ, "Pediatric Early Warning System Scores : lessons to be Learned", J Pediatr Intensive Care, 7, pp.27~32 (2018)
- Lambert V, "Paediatric early warning systems for detecting and responding to clinical deterioration in children : a systematic review", BMJ Open, 7, pp.e014497~ (2017)
- Chapman SM, "‘The Score Matters’ : wide variations in predictive performance of 18 paediatric track and trigger systems", Arch Dis Child, 102, pp.487~495 (2017)
- Chapman SM, "Early warning scores in paediatrics : an overview", Arch Dis Child, 104, pp.395~399 (2019)
- Lockwood JM, "AutoPEWS : automating pediatric early warning score calculation improves accuracy without sacrificing predictive ability", Pediatr Qual Saf, 5, pp.e274~ (2020)
- Gorham TJ, "The vitals risk index-retrospective performance analysis of an automated and objective pediatric early warning system", Pediatr Qual Saf, 5, pp.e271~ (2020)
- Zhai H, "Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children", Resuscitation, 85, pp.1065~1071 (2014)
- Pimentel MA, "Detecting deteriorating patients in the hospital : development and validation of a novel scoring system", Am J Respir Crit Care Med, 204, pp.44~52 (2021)
- Rubin J, "An ensemble boosting model for predicting transfer to the pediatric intensive care unit", Int J Med Inform, 112, pp.15~20 (2018)
- Kang DY, "Artificial intelligence algorithm to predict the need for critical care in prehospital emergency medical services", Scand J Trauma Resusc Emerg Med, 28, pp.17~ (2020)
- Kwon JM, "An algorithm based on deep learning for predicting in-hospital cardiac arrest", J Am Heart Assoc, 7, pp.e008678~ (2018)
- Cho KJ, "Detecting patient deterioration using artificial intelligence in a rapid response system", Crit Care Med, 48, pp.e285~e289 (2020)
- Lee YJ, "A multicentre validation study of the deep learning-based early warning score for predicting in-hospital cardiac arrest in patients admitted to general wards", Resuscitation, 163, pp.78~85 (2021)
- Park SJ, "Development and validation of a deep-learning-based pediatric early warning system : a single-center study", Biomed J, 45, pp.155~168 (2022)
- Hochreiter S, "Long short-term memory", Neural Comput, 9, pp.1735~1780 (1997)
- Srivastava N, "Dropout : a simple way to prevent neural networks from overfitting", J Mach Learn Res, 15, pp.1929~1958 (2014)
- "Adam: a method for stochastic optimization" (2017)[URL]
- Torrey L, "Handbook of research on machine learning applications and trends: algorithms, methods, and techniques", pp.242~264 (2009), IGI Global
- Ozenne B, "The precision : recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases", J Clin Epidemiol, 68, pp.855~859 (2015)
- Weng CG, "A new evaluation measure for imbalanced datasets", AusDM ‘08: proceedings of the 7th Australasian Data Mining Conference, pp.27~32 (2008), Australian Computer Society, Inc
- Leening MJ, "Net reclassification improvement : computation, interpretation, and controversies : a literature review and clinician’s guide", Ann Intern Med, 160, pp.122~131 (2014)
- Lundberg SM, "A unified approach to interpreting model predictions", Advances in neural information processing systems 30 (NIPS 2017): NeurIPS Proceedings (2017), Curran Associates Inc
- Trubey R, "Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalised children : a systematic review", BMJ Open, 9, pp.e022105~ (2019)
- Kowalski RL, "Accuracy and Monitoring of Pediatric Early Warning Score(PEWS)scores prior to emergent pediatric intensive care unit(ICU)transfer : retrospective analysis", JMIR Pediatr Parent, 4, pp.e25991~ (2021)
- Jensen CS, "Inter-rater reliability of two paediatric early warning score tools", Eur J Emerg Med, 26, pp.34~40 (2019)
- Kotsakis A, "Implementation of a multicenter rapid response system in pediatric academic hospitals is effective", Pediatrics, 128, pp.72~78 (2011)
- McLellan MC, "Validation of the Children’s Hospital Early Warning System for critical deterioration recognition", J Pediatr Nurs, 32, pp.52~58 (2017)
- Brown SR, "Scoping Review of Pediatric Early Warning Systems(PEWS)in resource-limited and humanitarian settings", Front Pediatr, 6, pp.410~ (2019)
- Dean NP, "Improving accuracy and timeliness of nursing documentation of Pediatric Early Warning Scores", Pediatr Qual Saf, 5, pp.e278~ (2020)
- Hyland SL, "Early prediction of circulatory failure in the intensive care unit using machine learning", Nat Med, 26, pp.364~373 (2020)
- Nguyen J, "Vignettes in patient safety", pp.93~113 (2019), IntechOpen Limited
- Lyons PG, "Characteristics of rapid response calls in the United States : an analysis of the first 402, 023 adult cases from the get with the guidelines resuscitation-medical emergency team registry", Crit Care Med, 47, pp.1283~1289 (2019)
- Ruskin KJ, "Alarm fatigue : impacts on patient safety", Curr Opin Anaesthesiol, 28, pp.685~690 (2015)
- Cvach M, "Monitor alarm fatigue : an integrative review", Biomed Instrum Technol, 46, pp.268~277 (2012)
- The Lancet Respiratory Medicine, "Opening the black box of machine learning", Lancet Respir Med, 6, pp.801~ (2018)
- Azodi CB, "Opening the black box : interpretable machine learning for geneticists", Trends Genet, 36, pp.442~455 (2020)