XAI 설명 수준이 인지부하 및 사용자 만족도에 미치는 영향에 관한 연구: 금융 AI 시스템의 리스크 수준을 중심으로
Study on the Impact of XAI Explanation Levels on Cognitive Load and User Satisfaction : Focusing on Risk Levels in Financial AI Systems
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구에서는 EU AI 법안에서 정의한 AI 시스템의 위험 수준에 따라 XAI 설명이 사용자 만족도와인지 부하에 미치는 영향을 살펴보았다. XAI는 복잡한 블랙박스 모델의 내부 처리 과정을 인간이 이해하기 쉽도록 설명하는 기술로, 학계와 산업계에서 널리 활용되고 있다. 그러나 AI 시스템의 리스크 수준에따라 필요한 설명 수준을 결정하는 연구는 미흡하였다. 이에 본 연구는 120명의 참가자를 8개의 집단으로 나누어, 금융 AI 시스템의 리스크 수준에 따른 인지 부하와 사용자 만족도를 설문을 통해 조사하였다. 사용자의 응답 결과는 요인 분석과 판별 타당성 검토 등을 포함한 정량적 통계 기법을 통해 분석되었으며, 그 결과 위험 수준에 따라 설명의 정보량이 인지 부하와 사용자 만족도에 미치는 영향이 통계적으로유의미하게 다름을 확인하였다. 또한, 설명 수준이 사용자 만족도에 미치는 영향은 정보의 이해를 돕는디자인에 의해 매개된다는 사실을 발견하였다. 본 연구는 AI 시스템의 리스크 수준에 따른 필요한설명 수준을 결정하는 지침을 제공함으로써 실무적, 제도적, 학문적 기여를 할 수 있을 것으로 기대된다.
Abstract
In this paper, we examine the impact of XAI explanations on user satisfaction and cognitive load according to the risk levels defined in the EU AI Act. XAI aims to make the internal processes of complex AI models understandable to humans and is widely used in both academia and industry. The importance and value of XAI are continuously rising; however, there has been little research determining the necessary level of explanation according to AI system risk levels. To address this gap, we designed an experiment with 120 participants, divided into 8 groups, each exposed to one of four levels of explainability(XAI) within low-risk and high-risk financial AI systems. A quantitative approach was used to measure cognitive load, user satisfaction, mental effort, and the clarity of the material design across the different AI system interfaces. The results indicate that the amount of information in explanations significantly affects cognitive load and user satisfaction, depending on the risk level. However, the impact of the level of explanation on user satisfaction was mediated by the material design, which determined how easily the information was understood. This research provides practical, regulatory, and academic contributions by offering guidelines for determining the necessary level of explanation based on AI system risk levels.
참고문헌 (35)
- "Here's What To Know About OpenAI's ChatGPT-What It's Disrupting And How To Use It"[URL]
- "Chat-GPT is a tipping point for AI"[URL]
- "South Korea Trumpets $860-million AI Fund after AlphaGo’Shock"[URL]
- Taecharungroj, V., "What Can Chat-GPT Do? Analyzing Early Reactions to the Innovative AI Chatbot on Twitter", Big Data and Cognitive Computing, 7(1), pp.35~ (2023)[DOI]
- Radford, A., "Improving language understanding by generative pre-training", ~ (2018)
- "AI Used to Face-swap Hollywood Starts into Pornography Films"[URL]
- "Amazon scraps secret AI recruiting tool that showed bias against women"[URL]
- Filimowicz, M., "Deep Fakes : Algorithms and Society" (2022), Routledge
- "AI language models are rife with different political biases"[URL]
- "PREPARING FOR THE FUTURE OF ARTIFICIAL INTELLIGENCE"[URL]
- Goodman, B., "European Union regulations on algorithmic decision-making and a right to explanation", AI magazine, 38(3), pp.50~57 (2017)[DOI]
- Von Eschenbach, W. J., "Transparency and the black box problem: Why we do not trust AI", Philosophy & Technology, 34(4), pp.1607~1622 (2021)[DOI]
- Ehsan, U., "Expanding explainability: Towards social transparency in ai systems", Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp.1~19
- Nimmo, R., "User Characteristics in Explainable AI: The Rabbit Hole of Personalization?", Proceedings of the CHI Conference on Human Factors in Computing Systems, pp.1~13
- Sanneman, L., "An Information Bottleneck Characterization of the Understanding-Workload Tradeoff in Human-Centered Explainable AI", The 2024 ACM Conference on Fairness, Accountability, and Transparency, pp.2175~2198
- Jansen, A., "Contextualizing the “Why”: The Potential of Using Visual Map As a Novel XAI Method for Users with Low AI-literacy", Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pp.1~7
- Hu, P. J. H., "Examining the mediating roles of cognitive load and performance outcomes in user satisfaction with a website", Mis Quarterly, 41(3) (2017)
- Panigutti, C., "Understanding the impact of explanations on advice-taking: a user study for AI-based clinical Decision Support Systems", Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp.1~9
- Sweller, J., "Cognitive load during problem solving: Effects on learning", Cognitive science, 12(2), pp.257~285 (2003)[DOI]
- Mayer, R. E., "Nine ways to reduce cognitive load in multimedia learning", Educational psychologist, 38(1), pp.43~52 (2010)[DOI]
- Paas, F. G.M, "Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive-load approach", Journal of educational psychology, 84(4), pp.429~ (1992)[DOI]
- Ayres, P, "Using subjective measures to detect variations of intrinsic cognitive load within problems", Learning and Instruction, 16(5), pp.389~400 (2006)[DOI]
- Leppink, J., "Development of an instrument for measuring different types of cognitive load", Behavior Research Methods, 45(4) (2013)
- Kalyuga, S., "Managing cognitive load in adaptive multimedia learning", pp.58~80, IGI Global
- "AI Act Political Agreement Overview"[URL]
- "Guide to AI Development and Utilization in the Financial Sector"[URL]
- Rana, N. P., "Investigating success of an e-government initiative : Validation of an integrated IS success model", Information systems frontiers, 17 (2014)
- Han, C. W., "Influence of the characteristics of the O2O food delivery platform service on the trust, satisfaction and reuse intention", Journal of Hospitality and Tourism Studies, 21(1), pp.115~128 (2019)
- Wilkinson, D. L., "Effects of prior knowledge and spatial ability on learning outcomes and cognitive load associated with rich and lean multimedia presentation", pp.2004
- Miller, C. D., "Demystifying Aesthetics : An Examination of the Relationships and Effects of Emotional Design on Learner Cognitive Load and Task Performances", pp.2007
- Whelan, R. R., "The multimedia mind : Measuring cognitive load in multimedia learning", pp.2006
- Chulmo Koo, "A Study on the Explorative or Exploitive Use of Smartphone : Focusing on IS Continuance and Habit Theory", The Journal of Society for e-Business Studies, 16(3), pp.67~91 (2011)[DOI]
- Seddon, P. B., "A respecification and extension of the DeLone and McLean model of IS success", Information systems research, 8(3), pp.240~253 (1997)[DOI]