Validation of Machine Learning Models to Predict Adverse Outcomes in Patients with COVID-19: A Prospective Pilot Study
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Purpose: We previously developed learning models for predicting the need for intensive care and oxygen among patients withcoronavirus disease (COVID-19). Here, we aimed to prospectively validate the accuracy of these models. Materials and Methods: Probabilities of the need for intensive care [intensive care unit (ICU) score] and oxygen (oxygen score)were calculated from information provided by hospitalized COVID-19 patients (n=44) via a web-based application. The performanceof baseline scores to predict 30-day outcomes was assessed. Results: Among 44 patients, 5 and 15 patients needed intensive care and oxygen, respectively. The area under the curve of ICUscore and oxygen score to predict 30-day outcomes were 0.774 [95% confidence interval (CI): 0.614–0.934] and 0.728 (95% CI:0.559–0.898), respectively. The ICU scores of patients needing intensive care increased daily by 0.71 points (95% CI: 0.20–1.22) afterhospitalization and by 0.85 points (95% CI: 0.36–1.35) after symptom onset, which were significantly different from those in individualsnot needing intensive care (p=0.002 and <0.001, respectively). Trends in daily oxygen scores overall were not markedlydifferent; however, when the scores were evaluated within <7 days after symptom onset, the patients needing oxygen showed ahigher daily increase in oxygen scores [1.81 (95% CI: 0.48–3.14) vs. -0.28 (95% CI: 1.00–0.43), p=0.007]. Conclusion: Our machine learning models showed good performance for predicting the outcomes of COVID-19 patients andcould thus be useful for patient triage and monitoring.
Abstract
Purpose: We previously developed learning models for predicting the need for intensive care and oxygen among patients withcoronavirus disease (COVID-19). Here, we aimed to prospectively validate the accuracy of these models. Materials and Methods: Probabilities of the need for intensive care [intensive care unit (ICU) score] and oxygen (oxygen score)were calculated from information provided by hospitalized COVID-19 patients (n=44) via a web-based application. The performanceof baseline scores to predict 30-day outcomes was assessed. Results: Among 44 patients, 5 and 15 patients needed intensive care and oxygen, respectively. The area under the curve of ICUscore and oxygen score to predict 30-day outcomes were 0.774 [95% confidence interval (CI): 0.614–0.934] and 0.728 (95% CI:0.559–0.898), respectively. The ICU scores of patients needing intensive care increased daily by 0.71 points (95% CI: 0.20–1.22) afterhospitalization and by 0.85 points (95% CI: 0.36–1.35) after symptom onset, which were significantly different from those in individualsnot needing intensive care (p=0.002 and <0.001, respectively). Trends in daily oxygen scores overall were not markedlydifferent; however, when the scores were evaluated within <7 days after symptom onset, the patients needing oxygen showed ahigher daily increase in oxygen scores [1.81 (95% CI: 0.48–3.14) vs. -0.28 (95% CI: 1.00–0.43), p=0.007]. Conclusion: Our machine learning models showed good performance for predicting the outcomes of COVID-19 patients andcould thus be useful for patient triage and monitoring.
참고문헌 (37)
- Zhu N, "A novel coronavirus from patients with pneumonia in China, 2019", N Engl J Med, 382, pp.727~733 (2020)
- Beigel JH, "Remdesivir for the treatment of COVID-19-final report", N Engl J Med, 383, pp.1813~1826 (2020)
- RECOVERY Collaborative Group, "Dexamethasone in hospitalized patients with COVID-19", N Engl J Med, 384, pp.693~704 (2021)
- Shen C, "Treatment of 5critically ill patients with COVID-19 with convalescent plasma", JAMA, 323, pp.1582~1589 (2020)
- Stone JH, "Efficacy of tocilizumab in patients hospitalized with COVID-19", N Engl J Med, 383, pp.2333~2344 (2020)
- Cao B, "A trial of lopinavir-ritonavir in adults hospitalized with severe COVID-19", N Engl J Med, 382, pp.1787~1799 (2020)
- RECOVERY Collaborative Group, "Effect of hydroxychloroquine in hospitalized patients with COVID-19", N Engl J Med, 383, pp.2030~2040 (2020)
- Wiersinga WJ, "Pathophysiology, transmission, diagnosis, and treatment of coronavirus disease 2019(COVID-19) : a review", JAMA, 324, pp.782~793 (2020)
- Ranney ML, "Critical supply shortages—the need for ventilators and personal protective equipment during the COVID-19 pandemic", N Engl J Med, 382, pp.e41~ (2020)
- Wynants L, "Prediction models for diagnosis and prognosis of COVID-19 : systematic review and critical appraisal", BMJ, 369, pp.m1328~ (2020)
- Ji M, "A predictive model for disease progression in non-severely ill patients with coronavirus disease 2019", Eur Respir J, 56, pp.2001234~ (2020)
- Yan L, "An interpretable mortality prediction model for COVID-19 patients", Nat Mach Intell, 2, pp.283~288 (2020)
- Cheng FY, "Using machine learning to predict ICU transfer in hospitalized COVID-19 patients", J Clin Med, 9, pp.1668~ (2020)
- Kim HJ, "An easy-to-use machine learning model to predict the prognosis of patients with COVID-19 : retrospective cohort study", J Med Internet Res, 22, pp.e24225~ (2020)
- Heo J, "COVID-19 outcome prediction and monitoring solution for military hospitals in South Korea : development and evaluation of an application", J Med Internet Res, 22, pp.e22131~ (2020)
- Heo J, "A patient selfcheckup app for COVID-19 : development and usage pattern analysis", J Med Internet Res, 22, pp.e19665~ (2020)
- Liu Y, "The reproductive number of COVID-19 is higher compared to SARS coronavirus", J Travel Med, 27, pp.taaa021~ (2020)
- Majumdar SR, "Oxygen saturations less than 92% are associated with major adverse events in outpatients with pneumonia : a population-based cohort study", Clin Infect Dis, 52, pp.325~331 (2011)
- Waring J, "Automated machine learning : review of the state-of-the-art and opportunities for healthcare", Artif Intell Med, 104, pp.101822~ (2020)
- Lindstrom MJ, "Newton—Raphson and EM algorithms for linear mixed-effects models for repeated-measures data", J Am Stat Assoc, 83, pp.1014~1022 (1988)
- Cnaan A, "Using the general linear mixed model to analyse unbalanced repeated measures and longitudinal data", Stat Med, 16, pp.2349~2380 (1997)
- Argenziano MG, "Characterization and clinical course of 1000 patients with coronavirus disease 2019 in New York : retrospective case series", BMJ, 369, pp.m1996~ (2020)
- Yang X, "Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China : a single-centered, retrospective, observational study", Lancet Respir Med, 8, pp.475~481 (2020)
- Gong J, "A tool for early prediction of severe coronavirus disease 2019(COVID-19) : a multicenter study using the risk nomogram in Wuhan and Guangdong, China", Clin Infect Dis, 71, pp.833~840 (2020)
- Ji D, "Prediction for progression risk in patients with COVID-19 pneumonia : the CALL score", Clin Infect Dis, 71, pp.1393~1399 (2020)
- Knight SR, "Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO clinical characterisation protocol : development and validation of the 4C Mortality Score", BMJ, 370, pp.m3339~ (2020)
- Oh B, "Prediction models for the clinical severity of patients with COVID-19 in Korea : retrospective multicenter cohort study", J Med Internet Res, 23, pp.e25852~ (2021)
- Subudhi S, "Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19", NPJ Digit Med, 4, pp.87~ (2021)
- Chia ML, "Managing COVID-19 in a novel, rapidly deployable community isolation quarantine facility", Ann Intern Med, 174, pp.247~251 (2021)
- Zhang Z, "Wuhan and Hubei COVID-19 mortality analysis reveals the critical role of timely supply of medical resources", J Infect, 81, pp.147~178 (2020)
- Shim E, "Estimating the risk of COVID-19 death during the course of the outbreak in Korea, February–May 2020", J Clin Med, 9, pp.1641~ (2020)
- Hollander JE, "Virtually perfect? Telemedicine for COVID-19", N Engl J Med, 382, pp.1679~1681 (2020)
- Petrilli CM, "Factors associated with hospital admission and critical illness among 5279 people with coronavirus disease 2019 in New York city : prospective cohort study", BMJ, 369, pp.m1966~ (2020)
- Faes C, "Time between symptom onset, hospitalisation and recovery or death : statistical analysis of Belgian COVID-19 patients", Int J Environ Res Public Health, 17, pp.7560~ (2020)
- Linton NM, "Incubation period and other epidemiological characteristics of 2019 novel coronavirus infections with right truncation : a statistical analysis of publicly available case data", J Clin Med, 9, pp.538~ (2020)
- Zhou F, "Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China : a retrospective cohort study", Lancet, 395, pp.1054~1062 (2020)
- Young BE, "Epidemiologic features and clinical course of patients infected with SARS-CoV-2 in Singapore", JAMA, 323, pp.1488~1494 (2020)