의료 인공지능과 설명가능성
Explainable Artificial Intelligence in Medicine
0
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
인공지능 기술은 의료의 다양한 분야에서 뛰어난 성능을 보이고 있으나 인공지능알고리즘의 불투명성은 인공지능이 임상에서 널리 활용되는 데에 걸림돌로 작용하고있다. 우선 의사의 관점에서 바라보면, 투명성이 결여되면 의료 인공지능 모델에 대한신뢰성이 약화되고 인공지능 모델의 임상에서의 권고를 검증하거나 오류 또는 편향을 발견해내기 어려워진다. 또한 환자의 관점에서는 환자의 개인정보에 대한 자기결정권 또는 생명, 신체에 대한 자기결정권을 보호하기 위해서도 설명가능한 인공지능이 필요할 수 있다. 의료 인공지능 분야에서 설명가능성에 대한 관심은 최근 몇 년 사이 폭발적으로 증가하고 있는 추세이다. 이 글은 우리나라에서 의료 인공지능의 설명가능성에 대하여 법적인 관점에서 분석한 첫 번째 연구로서, 의료 인공지능에 설명가능성이 갖는 법적 의미를 설명가능한 인공지능 기술의 발전단계와 의료 인공지능의특성에 비추어 살펴보고자 한다. 우선, 설명가능성의 의미와 의료 인공지능에서 빈번하게 쓰이는 설명가능한 인공지능 기술에 대하여 개관하고, 다음으로 현재 설명가능한 인공지능이 의료분야에 법적으로 요구되고 있는지를 의료기기 규제, 개인정보보호 규제, 의사의 설명의무, 의사의 의료과오책임, 제조업자의 제조물책임 등의 관점에서 차례로 검토한다. 마지막으로, 앞으로 의료 인공지능에 설명가능성이 갖는 중요성이 어떻게 변화할 것인지를 전망하며 이 글을 끝맺는다.
Abstract
Artificial intelligence technology is demonstrating exceptional performance across various fields in healthcare. However, the opacity of AI algorithms presents a significant obstacle to their widespread clinical application. From a physician’s perspective, this lack of transparency can undermine the reliability of medical AI models, making it challenging to verify clinical recommendations of AI models or to identify errors and biases. Furthermore, from a patient’s viewpoint, explainable AI might be necessary to protect the rights to self-determination concerning personal health information and bodily integrity. The interest in explainability within the field of medical AI has skyrocketed recently, yet there has been no study analyzing the explainability of medical AI from a legal perspective in Korea. This paper aims to explore whether the law currently requires explainability in medical AI, and if it should in the future. It begins by discussing the concept of explainability and the explainable AI technologies commonly used in medical AI, then examines the legal requirements for explainable AI in healthcare, considering medical device regulations, personal information protection, physicians’ duty to inform, medical malpractice liability, and manufacturers’ product liability. Finally, the paper addresses the need for explainability in medical AI in the future before concluding.
참고문헌 (113)
- A. Michael Froomkin, "When AIs Outperform Doctors:Confronting the Challenges of a Tort-Induced Over-Reliance on Machine Learning", Ariz. L. Rev, 61, pp.33~ (2019)
- Alex John London, "Artificial intelligence and black-box medical decisions : accuracy versus explainability", Hastings Center Report, 49, pp.15~ (2019)
- Alina-Emilia Ciortea, "What Medical Risks Should Physicians Disclose to Their Patients : Towards a Better Standard in American and French Medical Malpractice Law", J. Civ. L. Stud, 10, pp.173~ (2017)
- Amina Adadi, "Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)", IEEE Access, 6, pp.52138~ (2018)
- Anand Avati, "Improving palliative care with deep learning", BMC Medical Informatics and Decision Making, 18, pp.55~ (2018)
- Anastasiya Kiseleva, "Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations", Frontiers in Artificial Intelligence, 5, pp.1~ (2022)
- Andrea Esteva, "Dermatologist-level classification of skin cancer with deep neural networks", Nature, 542, pp.115~ (2017)
- Andreas Holzinger, "What do we need to build explainable AI systems for the medical domain?", arXiv preprint arXiv:1712.09923, ~ (2017)
- Andrew D. Selbst, "The intuitive appeal of explainable machines", Fordham L. Rev, 87, pp.1085~ (2018)
- Aniek F. Markus, "The role of explainability in creating trustworthy artificial intelligence for health care: A comprehensive survey of the terminology, design choices, and evaluation strategies", J. Biomed. Inform, 113, pp.1~ (2021)
- Anto Čartolovni, "Ethical, legal, and social considerations of AI-based medical decision-support tools: A scoping review", Int’ J. Med. Inform, 161, pp.1~ (2022)
- "Art 29 Data Protection Working Party, Guidelines on Automated individual decision-making and profiling for the purposes of Regulation 2016", 679, 2017
- Ayanna Howard, "Are we trusting AI too much?", Proceedings of the 2020 ACM, 2020, ~
- Beauchamp, "Principles of Biomedical Ethics", ~ (2001)
- Berkeley J. Dietvorst, "Algorithm aversion: People erroneously avoid algorithms after seeing them err", Journal of Experimental Psychology : General, 144, pp.114~ (2015)
- Bolei Zhou, "Learning Deep Features for Discriminative Localization", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.2921~ (2016)
- Boris Babic, "Beware explanations from AI in health care", Science, 373, pp.284~ (2021)
- Bryth Goodman, "EU Regulations on Algorithmic Decision-Making and a “right to explanation,”", ICML workshop on human interpretability in machine learning (2016)
- Chang Ho Yoon, "Machine learning in medicine: should the pursuit of enhanced interpretability be abandoned?", J. Med. Ethics, 48, pp.581~ (2022)
- Cynthia Rudin, "Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead", Nature machine intelligence, 1, pp.206~ (2019)
- Daniel Schiff, "How Should Clinicians Communicate with Patients about the Roles of Artificial Intelligent Team Members?", AMA J. Ethics, 21, pp.138~140 (2019)
- Daniel Schönberger, "Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications", Int’l J. L. Inform. Tech, 27, pp.171~ (2019)
- David Gunning, "DARPA’s explainable artificial intelligence program", AI Mag, 40, pp.44~ (2019)
- David Gunning, "Explainable artificial intelligence (XAI)" (2017), Defense Advanced Research Projects Agency (DARPA)
- David Lyell, "Automation bias and verification complexity: a systematic review", J. Am. Med. Inform. Ass’n, 24, pp.423~ (2017)
- "Explainable AI meets healthcare: a study on heart disease dataset" (2020)[URL]
- Edward H. Shortliffe, "Clinical decision support in the era of artificial intelligence", JAMA, 320, pp.2199~ (2018)
- Eric J. Topol, "High-performance medicine : The convergence of human and artificial intelligence", Nat. Med, 25, pp.44~ (2019)
- Eric J. Topol, "High-performance medicine: the convergence of human and artificial intelligence", Nature medicine, 25, pp.44~ (2019)
- Eyal Winter, "The shapley value, in Handbook of Game Theory with Economic Applications, Volume 3" (2002)
- Proposed regulatory framework for modifications to aritifical intelligence, "machine learning (AI", ML)-based Software as a Medical Device(SaMD), pp.2020, ~
- Fei Wang, "Should health care demand interpretable artificial intelligence or accept “black box” medicine?", Annals of internal medicine, 172, pp.59~ (2020)
- Forough Poursabzi-Sangdeh, "Manipulating and measuring model interpretability", Proceedings of the 2021 CHI conference on human factors in computing systems, 1 (2021)
- Gagan Bansal, "Does the whole exceed its parts? the effect of ai explanations on complementary team performance", Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1 (2021)
- Gerhard Wagner, "Liability Rules for the Digital Age–Aiming for the Brussels Effect–", Journal of European, 13, pp.191~ (2023)
- Gerhard Wagner, "Robot Liability, in Liability for Artificial Intelligence and the Internet of Things 25" (2019)
- Gilmer Valdes, "Clinical decision support of radiotherapy treatment planning: A data-driven machine learning strategy for patient-specific dosimetric decision making", Radiotherapy and Oncology, 125, pp.392~ (2017)
- Hang Dong, "Explainable automated coding of clinical notes using hierarchical label-wise attention networks and label embedding initialisation", J. Biomed. Inform, 116, pp.103728~ (2021)
- High-Level Expert Group on Artificial Intelligence, "Ethics guidelines for trustworthy AI", European Commission, 1~ (2019)
- Hui Wen Loh, "Application of explainable artificial intelligence for healthcare : A systematic review of the last decade(2011–2022)", Computer Methods and Programs in Biomedicine, 226, pp.107161~ (2022)
- Jahmunah Vicnesh, "Explainable detection of myocardial infarction using deep learning models with Grad-CAM technique on ECG signals", Comput. Biol. Med, 146, pp.105550~ (2022)
- Jason W. Burton, "A systematic review of algorithm aversion in augmented decision making", Journal of Behavioral Decision Making, 33, pp.220~ (2020)
- Jenna Burrell, "How the machine ‘thinks’ : Understanding opacity in machine learning algorithms", Big Data and Society, 3, pp.1~ (2016)
- Jianxing He, "The practical implementation of artificial intelligence technologies in medicine", Nat. Med, 25, pp.30~ (2019)
- John Zerilli, "Transparency in algorithmic and human decision-making : is there a double standard?", Philosophy & Technology, 32, pp.661~ (2019)
- Jordan Joseph Wadden, "Defining the undefinable : the black box problem in healthcare artificial intelligence", J. Med Ethics, 48, pp.764~ (2022)
- Jordan Joseph Wadden, "Defining the undefinable : The blackbox problem in healthcare artificial intelligence", J. Med. l Ethics, 48, pp.764~ (2021)
- Joshua Hatherly, "The Virtues of Interpretable Medical Artificial Intelligence", Cambridge Quarterly of Healthcare Ethics, 1 (2022)
- Jlia Amann, "Explainability for artificial intelligence in healthcare : a multidisciplinary perspective", BMC Med. Inform. Decis. Mak, 20, pp.310~ (2020)
- Julia Amann, "To explain or not to explain?--Artificial intelligence explainability in clinical decision support systemas", PLOS Digital Health, 1, pp.1~ (2022)
- Karl Stöger, "Medical Artificial Intelligence:The European Legal Perspective", Communications of the ACM, 64, pp.34~ (2021)
- Laura Arbelaez Ossa, "Re-focusing explainability in medicine", Digital Health, 8, pp.1~ (2022)
- Linda J. Skitka, "Does automation bias decisionmaking?", Int’l J. Hum. Computer Stud, 51, pp.991~ (1999)
- Maia Jacobs, "How machine-learning recommendations influence clinician treatment selections : the example of antidepressant selection", Translational psychiatry, 11, pp.108~ (2021)
- Marco Tulio Ribeiro, "“Why should i trust you?”Explaining the predictions of any classifier", Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp.1135~ (2016)
- Marzyeh Ghassemi, "The false hope of current approaches to explainable artificial intelligence in health care", Lancet Digit Health, 3, pp.e745~ (2021)
- Maximilian Kiener, "Artificial Intelligence in Medicine and the Disclosure of Risks", AI & Soc, 36, pp.705~ (2021)
- Michael D. Greenberg, "Medical malpractice and new devices : defining an elusive standard of care", Health Matrix, 19, pp.423~ (2009)
- Michael Yeomans, "Sendhil Mullainathan & Jon Kleinberg, Making sense of recommendations", Journal of Behavioral Decision Making, 32, pp.403~ (2019)
- Four principles of explainable artificial intelligence, "P. Jonathan Philips", NIST, 1~ (2020)
- Philipp Hacker, "Varieties of AI Explanations under the Law-From the GDPR to the AIA, and Beyond", International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers, 343 (2020)
- Phillip Hacker, "Explainable AI under contract and tort law: legal incentives and technical challenges", Artificial Intelligence and Law, 28, pp.415~ (2020)
- Ramprasaath R. Selvaraju, "GradCAM: visual explanations from deep networks via gradient-based localization", Int. J. Comput. Vis, 128, pp.336~ (2020)
- Rich Caruana, "Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission", Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp.1721~ (2015)
- Robert Challen, "Artificial intelligence, bias and clinical safety", BMJ Quality & Safety, 28, pp.231~ (2019)
- Sabin N. van der Veer, "Trading off accuracy and explainability in AI decision-making: Findings from 2 citizens’ juries", J. Am. Inform. Ass’n, 28, pp.2128~ (2021)
- Sandeep Reddy, "A governance model for the application of AI in health care", J. Am. Med. Inform. Ass’n, 27, pp.4~91 (2020)
- Sandra Wachter, "Counterfactual explanations without opening the black box: Automated decisions and the GDPR", Harv. JL &Tech, 31, pp.841~ (2017)
- Sandra Wachter, "Brent Mittelstadt & Luciano Floridi, Why a right to explanation of automated decision-making does not exist in the general data protection regulation", Int’l Data Privacy L. Rev, 7, pp.76~ (2017)
- Suzanne Kawamleh, "Against explainability requirements for ethical artificial intelligence in health care", AI and Ethics (2022)
- Thomas Grote, "How competitors become collaborators—Bridging the gap(s)between machine learning algorithms and clinicians", Bioethics, 36, pp.134~ (2022)
- Umang Bhatt, "Explainable machine learning in deployment", Proceedings of the 2020 conference on fairness, accountability, and transparency, pp.648~ (2020)
- Usman Ahmed, "EANDC : an explainable attention network based deep adaptive clustering model for mental health treatment", Future Genereration Computing Systems, 130, pp.106~ (2022)
- W. Nicholson Price II, "Medical malpractice and black-box medicine, in Big Data, Health Law, and Bioethics" (2018), 2018
- W. Nicholson Price II, "Potential LIability for Physicians Using Artificial Intelligence", JAMA, 322, pp.1765~ (2019)
- Yu Cheng, "Risk prediction with electronic health records: A deep learning approach", Proceedings of the 2016 SIAM International Conference on Data Mining, pp.432~ (2016)
- Zachary C. Lipton, "The mythos of model interpretability", Comm. ACM, 61, pp.36~ (2018)
- "Canterbury v. Spence 464 F.2d 772, 776 (D.C. Cir. 1972)", ~ (1972)
- "Natanson v. Kline 354 P.2d 670 (Kan. 1960)", ~ (1960)
- "Scott v. Bradford, 606 P.2d 554, 556 (Okla. 1979)", ~ (1979)
- Evaluation of Council Directive 85, "374", EEC on the approximation of laws, regulations and administrative provisions of the Member States concerning liability for defective products, final report, pp.2018, ~
- European Commission, "Proposal for a Directive of the European Parliament and of the Council on adapting non-contractual civil liability rules to artificial intelligence", AILD Proposal, ~ (2022)
- European Commission, "Proposal for a Directive of the European Parliament and of the Council on Liability for Defective Products", PLD Proposal, ~ (2022)
- European Commission, "Questions and answers on the revision of the Product Liability Directive", ~ (2022)
- 강봉석, "의사의 설명의무", 법학논집, 7(1), pp.51~68 (2002), 법학연구소
- 고학수, "인공지능 원론 : 설명가능성을 중심으로" (2021), 박영사
- 김광수, "인공지능 기반 과학기술과 국민의 권익구제- 자율주행차, 드론 및 의료기기를 중심으로 -", 토지공법연구, 85, pp.231~258 (2019), 한국토지공법학회
- 김민중, "의료계약", 사법행정, 32(1) (1991)
- 김재선, "인공지능 의료기기 위험관리를 위한 규범론적 접근 ― 인공지능 소프트웨어 규범화 논의를 중심으로 ―", 공법연구, 46(2), pp.131~154 (2017), 한국공법학회
- 김지석, "의사의 설명의무의 범위에 관한 고찰", 재산법연구, 28(1), pp.341~361 (2011), 한국재산법학회
- 김진우, "인공지능: 제조물책임법의 업데이트 여부에 관하여", 재산법연구, 37(2), pp.25~54 (2020), 한국재산법학회[DOI]
- 김천수, "의사의 설명의무", 민사법학, (7) (1988)
- 제조물책임법 개정 방안 연구, "박동진", 법무부⋅공정거래위원회, ~ (2012)
- 박태신, "의료소송에 있어서 설명의무의 기능", 연세법학연구, 5(1) (1998)
- 박혜진, "의료 인공지능의 활용을 둘러싼 법적 과제: 규제의 진화 및 책임의 배분을 중심으로", 비교사법, 29(4), pp.217~251 (2022), 한국사법학회[DOI]
- 배현아, "보건의료법제 하에서 인공지능기술의 의료영역 도입의 의의와 법적 문제", 법조, 66(4), pp.43~89 (2017), 법조협회[DOI]
- 백경희, "인공지능을 이용한 의료행위와 민사책임에 관한 고찰", 법조, 66(4), pp.90~121 (2017), 법조협회[DOI]
- 백경희, "의료판례의 동향과 문제 : 민사법적 쟁점과 전망을 중심으로", 한국의료법학회지, 26(1), pp.223~249 (2018), 한국의료법학회[DOI]
- 설민수, "머신러닝 인공지능과 인간전문직의 협업의 의미와 법적 쟁점: 의사의 의료과실 책임을 사례로", 저스티스, 163, pp.255~284 (2017), 한국법학원[DOI]
- 식품의약품안전처, "인공지능 의료기기의 허가⋅심사 가이드라인(민원인 안내서)", ~ (2022)
- 신현호, "의료소송총론" (1997), 육법사
- 양종모, "재범의 위험성 예측 알고리즘과 설명가능성 및 공정성의 문제", 형사법의 신동향, (70), pp.207~240 (2021), 대검찰청[DOI]
- 엄주희, "인공지능 의료와 법제", 한국의료법학회지, 28(2), pp.53~77 (2020), 한국의료법학회
- 오병철, "인공지능 로봇에 의한 손해의 불법행위책임", 법학연구, 27(4), pp.157~231 (2017), 법학연구원[DOI]
- 이덕환, "미국법과 독일법상 의사의 설명의무", 비교사법, 3(1) (1996)
- 이상돈, "의료법강의" (2020), 법문사
- 이석배, "의료후견주의와 환자의 자기결정권 - 의사의 설명의무를 중심으로 -", 고려법학, (49), pp.935~963 (2007), 법학연구원
- 이종원, "의사의 설명의무에 관한 소고", 경성법학, (10) (2001)
- 이중기, "의료 AI에 대한 규제체제와 책임의 귀속 - 진단 AI와 수술로봇을 중심으로", 홍익법학, 21(4), pp.263~290 (2020), 법학연구소[DOI]
- 이해원, "인공지능과 제조물책임", 정보법학, 25(2), pp.69~101 (2021), 한국정보법학회
- 정채연, "의료 인공지능의 법적 수용을 위한 시론적 연구 - 쟁점과 과제 -", 법학논총, 45(3), pp.145~176 (2021), 법학연구소[DOI]
- 최행식, "醫師의 說明義務와 患者의 自己決定權", 법학연구, (17), pp.341~363 (2004), 한국법학회
- 한삼인, "설명의무의 판단기준", 인권과 정의, (222) (1995)
