Black-box Medicine with AI/ML Technology Regulation: Clinical Roles, Relations, and Risks
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study discusses the issues with black-box medicine, including the challenges or barriers to fostering trust in artificial intelligence systems for their eventual integration, use, and adoption in medical settings. It also explores potential medical liabilities such as legal and ethical issues, which are novel and unresolved in terms of potential errors or harms caused by artificial intelligence diagnoses or treatment recommendations. Finally, these subjects also necessitate a brief examination of changing physician roles, in that black-box algorithms will eventually affect the relationships between patient and physicians when artificial intelligence becomes capable of simulating, assisting, or replacing traditionally human physician tasks. Technological advances in the form of black-box medicine as well as artificial intelligence generally and machine learning continue to affect regulatory frameworks and perceived risks. Regarding the implementation of these tools in medical devices, there is a need for greater clarity to ensure safety for patients while also being responsive to claims of efficacy or utility. Specifically, increasing the explanatory power of these technologies in the course of medical artificial intelligence development also furthers these twin goals. Technology usage for general scientific applications like drug discovery further highlights the potentially pervasive effects of black-box medicine on overall human health. At the same, separate issues arise concerning broader artificial intelligence safety and related areas such as bias far beyond mere medical outcomes. These uncertainties are by no means easily resolved, and they require the careful deliberation of interdisciplinary analysis drawing from legal, medical, and other expert backgrounds. In particular, it is necessary for lawmakers and regulators to be knowledgeable about the clinical impact of these tools as well as their impact on standard industry norms for policy goals . While the rapid development of new artificial intelligence models continues to improve human health, there are ongoing risks involved with regard to the way that decisions are made. Although unsupervised learning from artificial intelligence in medicine presents an opportunity, safety and other issues may mitigate the effectiveness of artificial intelligence-based studies by clinicians in the absence of additional protections. Reassuring patients as well as clinicians about artificial intelligence safety is imperative and promotes debate about the necessity of supporting or replacing clinical judgment in a healthcare context. This tension between human users and the new models as utilized in treatment underscores the idea that these technologies have the potential to either supplant human skills or augment them, depending on the needs of the medical setting. United States and Korean regulations also suggest that at the level of algorithms in black-box medicine, additional scrutiny may be warranted. Concerns about the opaqueness or black-box nature of an algorithm itself also give rise to calls for interpretability and developer awareness. Rapid product improvement with effective safeguards and pre-market approval prompt additional areas of analysis as medicine and other areas of science strive to balance the goal of enhancing medicine with the underlying nature of black-box medicine as algorithms which may not be capable of perfect transparency. In summary, black-box medicine suggests a new challenge in medicine which also begets opportunities. However, it also calls for detailed supervision in policy and practice with sound methodologies both to assure its ethical deployment and to foster trust towards innovative medical technologies.
Abstract
This study discusses the issues with black-box medicine, including the challenges or barriers to fostering trust in artificial intelligence systems for their eventual integration, use, and adoption in medical settings. It also explores potential medical liabilities such as legal and ethical issues, which are novel and unresolved in terms of potential errors or harms caused by artificial intelligence diagnoses or treatment recommendations. Finally, these subjects also necessitate a brief examination of changing physician roles, in that black-box algorithms will eventually affect the relationships between patient and physicians when artificial intelligence becomes capable of simulating, assisting, or replacing traditionally human physician tasks. Technological advances in the form of black-box medicine as well as artificial intelligence generally and machine learning continue to affect regulatory frameworks and perceived risks. Regarding the implementation of these tools in medical devices, there is a need for greater clarity to ensure safety for patients while also being responsive to claims of efficacy or utility. Specifically, increasing the explanatory power of these technologies in the course of medical artificial intelligence development also furthers these twin goals. Technology usage for general scientific applications like drug discovery further highlights the potentially pervasive effects of black-box medicine on overall human health. At the same, separate issues arise concerning broader artificial intelligence safety and related areas such as bias far beyond mere medical outcomes. These uncertainties are by no means easily resolved, and they require the careful deliberation of interdisciplinary analysis drawing from legal, medical, and other expert backgrounds. In particular, it is necessary for lawmakers and regulators to be knowledgeable about the clinical impact of these tools as well as their impact on standard industry norms for policy goals . While the rapid development of new artificial intelligence models continues to improve human health, there are ongoing risks involved with regard to the way that decisions are made. Although unsupervised learning from artificial intelligence in medicine presents an opportunity, safety and other issues may mitigate the effectiveness of artificial intelligence-based studies by clinicians in the absence of additional protections. Reassuring patients as well as clinicians about artificial intelligence safety is imperative and promotes debate about the necessity of supporting or replacing clinical judgment in a healthcare context. This tension between human users and the new models as utilized in treatment underscores the idea that these technologies have the potential to either supplant human skills or augment them, depending on the needs of the medical setting. United States and Korean regulations also suggest that at the level of algorithms in black-box medicine, additional scrutiny may be warranted. Concerns about the opaqueness or black-box nature of an algorithm itself also give rise to calls for interpretability and developer awareness. Rapid product improvement with effective safeguards and pre-market approval prompt additional areas of analysis as medicine and other areas of science strive to balance the goal of enhancing medicine with the underlying nature of black-box medicine as algorithms which may not be capable of perfect transparency. In summary, black-box medicine suggests a new challenge in medicine which also begets opportunities. However, it also calls for detailed supervision in policy and practice with sound methodologies both to assure its ethical deployment and to foster trust towards innovative medical technologies.
참고문헌 (36)
- Stan Benjamens, "The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database", npj Digital Medicine, 3(1), pp.118~ (2020), Springer Science and Business Media LLC[DOI]
- "Advancing into a Leader in AI Medical Devices" (2020)[URL]
- "A new Rx: AI for operations in healthcare"[URL]
- "AI Technology Assisting Deaf Students" (2018)[URL]
- "FDA" (2020)[URL]
- "Artificial Intelligence and Machine Learning in Software as a Medical Device" (2019)[URL]
- "What Is Artificial Intelligence?" (2007)[URL]
- Bertalan Meskó, "A short guide for medical professionals in the era of artificial intelligence", npj Digital Medicine, 3(1), pp.126~ (2020), Springer Science and Business Media LLC[DOI]
- "Artificial intelligence and Machine learning in Software" (2020)[URL]
- Kun-Hsing Yu, "Artificial Intelligence in Healthcare", Nature Biomedical Engineering, 2, pp.719~ (2018)
- W. Nicholson Price II, "Black-Box Medicine", Harvard J Law & Technology, 28(2) (2015)
- Stephen J. Capuzzi, "Chemotext: A Publicly Available Web Server for Mining Drug–Target–Disease Relationships in PubMed", Journal of Chemical Information and Modeling, 58(2), pp.212~218 (2018), American Chemical Society (ACS)[DOI]
- Gulshan V, "Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs", JAMA, 316(22), pp.2402~2410 (2016)
- Ryan Poplin, "Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning", Nature Biomedical Engineering, 2(3), pp.158~164 (2018), Springer Science and Business Media LLC[DOI]
- Bertalan Meskó, "A short guide for medical professionals in the era of artificial intelligence", npj Digital Medicine, 3(1), pp.126~ (2020), Springer Science and Business Media LLC[DOI]
- Pouyan Esmaeilzadeh, "Use of AI-based tools for healthcare purposes: a survey study from consumers’ perspectives", BMC Medical Informatics and Decision Making, 20(1), pp.170~ (2020), Springer Science and Business Media LLC[DOI]
- "When AI Overrules the Nurses Caring for You" (2023)[URL]
- Pu Wang, "Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study", Gut, 68(10), pp.1813~1819 (2019), BMJ[DOI]
- Ioana A. Cristea, "Ioannidis, Stealth research: Lack of peer-reviewed evidence from healthcare unicorns", Eur J Clin Investigation (2019)
- Liz Szabo, "Artificial intelligence is rushing into patient care – and could raise risks", Kaiser Health News, Scientific American, ~ (2019)
- Jenna Wiens, "Do no harm : a roadmap for responsible machine learning for health care", Nature medicine, 25(9), pp.1337~1340 (2019)
- Matheny, M., "Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril" (2022), National Academy of Medicine
- "Risks of letting AI experts experiment with healthcare" (2022)[URL]
- Stan Benjamens, "The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database", npj Digital Medicine, 3(1), pp.118~ (2020), Springer Science and Business Media LLC[DOI]
- "Clinical Decision Support Software Draft Guidance for Industry and Food and Drug Administration Staff" (2019)[URL]
- "Clinical Decision Support Software--Guidance for Industry and Food and Drug Administration Staff"[URL]
- Matheny, M., "Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril" (2022), National Academy of Medicine
- Theodore T. Lee, "U.S. Food and Drug Administration Precertification Pilot Program for Digital Health Software: Weighing the Benefits and Risks", Annals of Internal Medicine, 168(10), pp.730~732 (2018), American College of Physicians[DOI]
- Ezekiel J. Emanuel, "Artificial Intelligence in Health Care: Will the Value Match the Hype?", JAMA, 321(23), pp.2281~2282 (2019), American Medical Association (AMA)[DOI]
- "FDA How is the FDA Considering Regulation of Artificial Intelligence and Machine Learning Medical Devices" (2020)[URL]
- "FDA Proposed regulatory framework for modifications to artificial intelligence/machine learning (ai/ml)-based Software as a Medical Device (SaMD)" (2019)[URL]
- "Modifications to Devices Subject to Premarket Approval (PMA) - The PMA Supplement Decision-Making Process"[URL]
- "Hip implants (FDA Medical device recalls, 1-10 of 500 results from hip replacement"[URL]
- "Hip replacement recall" (2023)[URL]
- "FDA Class 2 device recall proceed surgical mesh"[URL]
- "Software as a Medical Device : Possible Framework for Risk Categorization and Corresponding Considerations" (2014)[URL]