RAG 기반 모델의 성능 향상을 위한 문장화 추출 방법론
Sentence-Based Extraction Methodology from External References to Enhance Performance in RAG
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
LLM은 학습 데이터에 포함되지 않은 최신 정보나 특정 도메인 지식 부족으로 인해 신뢰성 문제와 환각 현상이 발생할 수 있으며, 이를 보완하기 위해 외부 자료를 활용하여 추가 학습 없이도 적절한 답변을 제공하는 RAG 모델의 활용도가 높아지고 있다. 이에 따라 RAG 모델이 LLM의신뢰성을 향상시킬 수 있다는 많은 연구 결과들이 보고되고 있으나, 외부 자료를 어떠한 형태로활용하면 RAG 모델의 성능을 더욱 향상시킬 수 있을지에 대한 연구는 상대적으로 미진하다. 이에 본 연구에서는 외부 참고자료의 문장화 추출을 통해 RAG 모델의 성능을 향상시킬 수 있는 방법론을 제안하고자 한다. 제안 방법론의 성능 평가를 위해 특정 도메인(군사영어)에서 사용하는약어와 두문자어 5,006개로 LLM과 두 가지 RAG 모델(텍스트 단순 추출, 문장화 추출)에 대한Q&A 태스크를 수행하여 정확도를 비교하였으며, 그 결과 제안 방법론인 문장화 추출 RAG 모델이 다른 모델보다 우수한 성능을 보임을 확인하였다.
Abstract
The reliability of Large Language Models(LLMs) can be compromised by limitations in up-to-date information and gaps in specific domain knowledge, often leading to issues like hallucination and decreased trustworthiness. To address these challenges, Retrieval-Augmented Generation(RAG) models are increasingly utilized, allowing LLMs to provide relevant answers by leveraging external data without additional training. While much research has demonstrated the potential of RAG models to enhance the reliability of LLMs, there has been limited investigation into how best to utilize external resources to improve RAG model performance. In this study, we propose a methodology to enhance RAG model performance through sentence-based extraction of external reference materials. To evaluate our proposed methodology, we conducted a Q&A task in a specialized domain (Military English) using 5,006 abbreviations and acronyms. We compared the accuracy of an LLM and two types of RAG models (simple text extraction and sentence-based extraction), finding that our proposed approach outperformed the other models.
참고문헌 (32)
- S. P. Shin, "The Concept and Standardization Trends of Foundation Models in LLM", Information and Communications Magazine(KICS), 40(6), pp.12~21 (2023)
- "Fine-Tuning (Deep Learning)" (2023)[URL]
- G. W. Yi, "Design of a Question-Answering System based on RAG Model for Domestic Companies", Journal of The Korea Society of Computer and Information, 29(7), pp.81~88 (2024)
- C. S. Jeong, "Generative AI Service Implementation Using LLM Application Architecture : based on RAG Model and LangChain Framework", Journal of Intelligent Informagion System, 29(4), pp.129~164 (2023)
- H. S. Kim, "Development of Chat Web Service Functions for Administrative and Public Institutions in The Cloud Environment Using the Implementation of RAG Technology Data Learning Automation", KCIS Winter Confrrence 2024, pp.416~417 (2024)
- M. Ch. Kim, "A Comparative Study of Performance between Large Language Model(LLM)of Open SourceModel and Closed Source Model Implementations for the Retrieval-Augmented Generation(RAG)System", KCIS Summer Confrrence 2024, 1, pp.1354~1355 (2024)
- J. H. Jeon, "Research on The Development of a Defense AI Platform Using a RAG-based mil-sLLM", Korea Computer Congress 2024, pp.44~46 (2024)
- C. Novelli, "Taking AI Risks Seriously : A New Assessment Model for The AI Act", AI & Society, pp.1~5 (2023)
- R. Jain, "A Code Centric Evaluation of C", C++ Vulnerability Datasets for Deep Learning based Vulnerability Detection Techniques, 2023, 1~10
- A. J. Thirunavukarasu, "Large Language Models in Medicine", Nature Medicine, 29(8), pp.1930~1940 (2023)
- B. B. Arcila, "Is It a Platform? Is It a Search Engine? It’s ChatGPT! The European Liability Regime for Large Language Models", J. Free Speech L, 3, pp.455~ (2023)
- Y. Yao, "A Survey on Large Language Model(LLM)Security and Privacy : The Good, The Bad, and The Ugly", High-Confidence Computing, 4, pp.100211~ (2024)
- "What is LangChain?"[URL]
- "LangChain Guide for Developers"[URL]
- "What is RAG?"[URL]
- "FAISS Vector Database: A High-Performance AI Similarity Search"[URL]
- H. Lee, "Exploring the Relationship among Language Learning Strategies, Motivation, and Achievement in ESP Learning : A Case of Learning Military English", Korean Journal of Military Art and Science, 6(2), pp.283~302 (2020)[DOI]
- G. Song, "A Study on the Improvement of ELT Materials to Enhance the Military English Communication Skills of the Korean Armed Forces", Korean Journal of Military Art and Science, 78(1) (2022)[DOI]
- D. Evans, "Development in English for Specific Purpose : A Multi-disciplinary Approach" (1998), Cambridge Univ. Press
- J. S. Lee, "Interpretation Officer Guide" (2011), Chaek-maru
- A. Leidinger, "The Language of Prompting: What Linguistic Properties Make a Prompt Successful?", Association for Computational Linguistics:EMNLP2023 (2023)