Need for Transparency and Clinical Interpretability in Hemorrhagic Stroke Artificial Intelligence Research: Promoting Effective Clinical Application
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Purpose: This study aimed to evaluate the quality of artificial intelligence (AI)/machine learning (ML) studies on hemorrhagic stroke using the Minimum Information for Medical AI Reporting (MINIMAR) and Minimum Information About Clinical Artificial Intelligence Modeling (MI-CLAIM) frameworks to promote clinical application. Materials and Methods: PubMed, MEDLINE, and Embase were searched for AI/ML studies on hemorrhagic stroke. Out of the 531 articles found, 29 relevant original research articles were included. MINIMAR and MI-CLAIM scores were assigned by two experienced radiologists to assess the quality of the studies. Results: We analyzed 29 investigations that utilized AI/ML in the field of hemorrhagic stroke, involving a median of 224.5 patients. The majority of studies focused on diagnostic outcomes using computed tomography scans (89.7%) and were published in computer science journals (48.3%). The overall adherence rates to reporting guidelines, as assessed through the MINIMAR and MI-CLAIM frameworks, were 47.6% and 46.0%, respectively. In MINIMAR, none of the studies reported the socioeconomic status of the patients or how missing values had been addressed. In MI-CLAIM, only two studies applied model-examination techniques to improve model interpretability. Transparency and reproducibility were limited, as only 10.3% of the studies had publicly shared their code. Cohen’s kappa between the two radiologists was 0.811 and 0.779 for MINIMAR and MI-CLAIM, respectively. Conclusion: The overall reporting quality of published AI/ML studies on hemorrhagic stroke is suboptimal. It is necessary to incorporate model examination techniques for interpretability and promote code openness to enhance transparency and increase the clinical applicability of AI/ML studies.
Abstract
Purpose: This study aimed to evaluate the quality of artificial intelligence (AI)/machine learning (ML) studies on hemorrhagic stroke using the Minimum Information for Medical AI Reporting (MINIMAR) and Minimum Information About Clinical Artificial Intelligence Modeling (MI-CLAIM) frameworks to promote clinical application. Materials and Methods: PubMed, MEDLINE, and Embase were searched for AI/ML studies on hemorrhagic stroke. Out of the 531 articles found, 29 relevant original research articles were included. MINIMAR and MI-CLAIM scores were assigned by two experienced radiologists to assess the quality of the studies. Results: We analyzed 29 investigations that utilized AI/ML in the field of hemorrhagic stroke, involving a median of 224.5 patients. The majority of studies focused on diagnostic outcomes using computed tomography scans (89.7%) and were published in computer science journals (48.3%). The overall adherence rates to reporting guidelines, as assessed through the MINIMAR and MI-CLAIM frameworks, were 47.6% and 46.0%, respectively. In MINIMAR, none of the studies reported the socioeconomic status of the patients or how missing values had been addressed. In MI-CLAIM, only two studies applied model-examination techniques to improve model interpretability. Transparency and reproducibility were limited, as only 10.3% of the studies had publicly shared their code. Cohen’s kappa between the two radiologists was 0.811 and 0.779 for MINIMAR and MI-CLAIM, respectively. Conclusion: The overall reporting quality of published AI/ML studies on hemorrhagic stroke is suboptimal. It is necessary to incorporate model examination techniques for interpretability and promote code openness to enhance transparency and increase the clinical applicability of AI/ML studies.
참고문헌 (37)
- Feigin VL, "Global burden of stroke", Circ Res, 120, pp.439~448 (2017)
- GBD 2016 Stroke Collaborators, "Global, regional, and national burden of stroke, 1990-2016 : a systematic analysis for the global burden of disease study 2016", Lancet Neurol, 18, pp.439~458 (2019)
- van Asch CJ, "Incidence, case fatality, and functional outcome of intracerebral haemorrhage over time, according to age, sex, and ethnic origin : a systematic review and meta-analysis", Lancet Neurol, 9, pp.167~176 (2010)
- Feigin VL, "Update on the global burden of ischemic and hemorrhagic stroke in 1990-2013 : the GBD 2013 study", Neuroepidemiology, 45, pp.161~176 (2015)
- An SJ, "Epidemiology, risk factors, and clinical features of intracerebral hemorrhage : an update", J Stroke, 19, pp.3~10 (2017)
- Powers WJ, "Guidelines for the early management of patients with acute ischemic stroke : 2019 update to the 2018 guidelines for the early management of acute ischemic stroke : a guideline for healthcare professionals from the American Heart Association", American Stroke Association, 2019, pp.50, e344~e418
- Hemphill JC 3rd, "Guidelines for the management of spontaneous intracerebral hemorrhage : a guideline for healthcare professionals from the American Heart Association", American Stroke Association, 2015, pp.46, 2032~2060
- Lambin P, "Radiomics : the bridge between medical imaging and personalized medicine", Nat Rev Clin Oncol, 14, pp.749~762 (2017)
- Mongan J, "Checklist for artificial intelligence in medical imaging(CLAIM) : a guide for authors and reviewers", Radiol Artif Intell, 2, pp.e200029~ (2020)
- Bluemke DA, "Assessing radiology research on artificial intelligence : a brief guide for authors, reviewers, and readers—from the Radiology editorial board", Radiology, 294, pp.487~489 (2020)
- Sounderajah V, "Developing a reporting guideline for artificial intelligencecentred diagnostic test accuracy studies : the STARD-AI protocol", BMJ Open, 11, pp.e047709~ (2021)
- Collins GS, "Protocol for development of a reporting guideline(TRIPOD-AI)and risk of bias tool(PROBAST-AI)for diagnostic and prognostic prediction model studies based on artificial intelligence", BMJ Open, 11, pp.e048008~ (2021)
- Liu X, "Reporting guidelines for clinical trial reports for interventions involving artificial intelligence : the CONSORT-AI extension", Nat Med, 26, pp.1364~1374 (2020)
- Norgeot B, "Minimum information about clinical artificial intelligence modeling : the MI-CLAIM checklist", Nat Med, 26, pp.1320~1324 (2020)
- Hernandez-Boussard T, "MINIMAR(minimum information for medical AI reporting) : developing reporting standards for artificial intelligence in health care", J Am Med Inform Assoc, 27, pp.2011~2015 (2020)
- Sharrock MF, "3D deep neural network segmentation of intracerebral hemorrhage : development and validation for clinical trials", Neuroinformatics, 19, pp.403~415 (2021)
- Muschelli J, "PItcHPERFeCT : primary intracranial hemorrhage probability estimation using random forests on CT", Neuroimage Clin, 14, pp.379~390 (2017)
- Jianbo C, "Weakly supervised multitask learning models to identify symptom onset time of unclear-onset intracerebral hemorrhage", Int J Stroke, 17, pp.785~792 (2022)
- Barros RS, "Automated segmentation of subarachnoid hemorrhages with convolutional neural networks", Inform Med Unlocked, 19, pp.100321~ (2020)
- Arab A, "A fast and fully-automated deep-learning approach for accurate hemorrhage segmentation and volume quantification in non-contrast whole-head CT", Sci Rep, 10, pp.19389~ (2020)
- Yin HL, "A magnetic resonance angiography-based study comparing machine learning and clinical evaluation : screening intracranial regions associated with the hemorrhagic stroke of adult moyamoya disease", J Stroke Cerebrovasc Dis, 31, pp.106382~ (2022)
- Sohn B, "Quality assessment of stroke radiomics studies : promoting clinical application", Eur J Radiol, 161, pp.110752~ (2023)
- Kouli O, "Automated brain tumor identification using magnetic resonance imaging : a systematic review and meta-analysis", Neurooncol Adv, 4, pp.vdac081~ (2022)
- Pellegrini E, "Machine learning of neuroimaging for assisted diagnosis of cognitive impairment and dementia : a systematic review", Alzheimers Dement(Amst), 10, pp.519~535 (2018)
- Whiting PF, "QUADAS-2 : a revised tool for the quality assessment of diagnostic accuracy studies", Ann Intern Med, 155, pp.529~536 (2011)
- Zhang L, "Computed tomography angiography-based analysis of high-risk intracerebral haemorrhage patients by employing a mathematical model", BMC Bioinformatics, 20(Suppl 7), pp.193~ (2019)
- Tang Z, "Deep learningbased prediction of hematoma expansion using a single brain computed tomographic slice in patients with spontaneous intracerebral hemorrhages", World Neurosurg, 165, pp.e128~e136 (2022)
- Rangaraj S, "Identifying risk factors of intracerebral hemorrhage stability using explainable attention model", Med Biol Eng Comput, 60, pp.337~348 (2022)
- Li H, "Non-contrast CT-based radiomics score for predicting hematoma enlargement in spontaneous intracerebral hemorrhage", Clin Neuroradiol, 32, pp.517~528 (2022)
- Seymour SE, "Predicting hematoma expansion after spontaneous intracranial hemorrhage through a radiomics based model", Proc SPIE Int Soc Opt Eng, 12033, pp.120332X~ (2022)
- Skoch J, "Predicting symptomatic cerebral vasospasm after aneurysmal subarachnoid hemorrhage with an artificial neural network in a pediatric population", Childs Nerv Syst, 33, pp.2153~2157 (2017)
- Liu J, "Prediction of hematoma expansion in spontaneous intracerebral hemorrhage using support vector machine", EBioMedicine, 43, pp.454~459 (2019)
- "Towards better explanations of class activation mapping"[URL]
- Kelly BS, "Radiology artificial intelligence : a systematic review and evaluation of methods(RAISE)", Eur Radiol, 32, pp.7998~8007 (2022)
- Cohen JF, "TRIPOD+AI : an updated reporting guideline for clinical prediction models", BMJ, 385, pp.q824~ (2024)