Development and Evaluation of a Retrieval-Augmented Generation-Based Electronic Medical Record Chatbot System
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Objectives: This study aimed to develop and evaluate a retrieval-augmented generation (RAG)-based chatbot system designedto optimize hospital operations. By leveraging electronic medical record (EMR) manuals, the system seeks to streamlineadministrative workflows and enhance healthcare delivery. Methods: The system integrated fine-tuned multilingualembedding models (Multilingual-E5-Large and BGE-M3) for indexing and retrieving information from EMR manuals. Adataset comprising 5,931 question-document pairs was constructed through query augmentation and validated by domainexperts. Fine-tuning was performed using contrastive learning to enhance semantic understanding, with performance assessedusing top-k accuracy metrics. The Solar Mini Chat API was adopted for text generation, prioritizing Korean-languageresponses and cost efficiency. Results: The fine-tuned models demonstrated marked improvements in retrieval accuracy,with BGE-M3 achieving 97.6% and Multilingual-E5-Large reaching 89.7%. The chatbot achieved high performance, withquery latency under 10 ms and robust retrieval precision, effectively addressing operational EMR queries. Key applicationsincluded administrative task support and billing process optimization, highlighting its potential to reduce staff workload andenhance healthcare service delivery. Conclusions: The RAG-based chatbot system successfully addressed critical challengesin healthcare administration, improving EMR usability and operational efficiency. Future research should focus on realworlddeployment and longitudinal studies to further evaluate its impact on administrative burden reduction and workflowimprovement.
Abstract
Objectives: This study aimed to develop and evaluate a retrieval-augmented generation (RAG)-based chatbot system designedto optimize hospital operations. By leveraging electronic medical record (EMR) manuals, the system seeks to streamlineadministrative workflows and enhance healthcare delivery. Methods: The system integrated fine-tuned multilingualembedding models (Multilingual-E5-Large and BGE-M3) for indexing and retrieving information from EMR manuals. Adataset comprising 5,931 question-document pairs was constructed through query augmentation and validated by domainexperts. Fine-tuning was performed using contrastive learning to enhance semantic understanding, with performance assessedusing top-k accuracy metrics. The Solar Mini Chat API was adopted for text generation, prioritizing Korean-languageresponses and cost efficiency. Results: The fine-tuned models demonstrated marked improvements in retrieval accuracy,with BGE-M3 achieving 97.6% and Multilingual-E5-Large reaching 89.7%. The chatbot achieved high performance, withquery latency under 10 ms and robust retrieval precision, effectively addressing operational EMR queries. Key applicationsincluded administrative task support and billing process optimization, highlighting its potential to reduce staff workload andenhance healthcare service delivery. Conclusions: The RAG-based chatbot system successfully addressed critical challengesin healthcare administration, improving EMR usability and operational efficiency. Future research should focus on realworlddeployment and longitudinal studies to further evaluate its impact on administrative burden reduction and workflowimprovement.