LLM-Based Response Generation for Korean Adolescents: A Study Using the NAVER Knowledge iN Q&A Dataset with RAG
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Objectives: This research aimed to develop a retrieval-augmented generation (RAG) based large language model (LLM)system that offers personalized and reliable responses to a wide range of concerns raised by Korean adolescents. Our workfocuses on building a culturally reflective dataset and on designing and validating the system’s effectiveness by comparing theanswer quality of RAG-based models with non-RAG models. Methods: Data were collected from the NAVER KnowledgeiN platform, concentrating on posts that featured adolescents’ questions and corresponding expert responses during the period2014–2024. The dataset comprises 3,874 cases, categorized by key negative emotions and the primary sources of worry. The data were processed to remove irrelevant or redundant content and then classified into general and detailed causes. TheRAG-based model employed FAISS for similarity-based retrieval of the top three reference cases and used GPT-4o mini forresponse generation. The responses generated with and without RAG were evaluated using several metrics. Results: RAGbasedresponses outperformed non-RAG responses across all evaluation metrics. Key findings indicate that RAG-based responsesdelivered more specific, empathetic, and actionable guidance, particularly when addressing complex emotional andsituational concerns. The analysis revealed that family relationships, peer interactions, and academic stress are significant factorsaffecting adolescents’ worries, with depression and stress frequently co-occurring. Conclusions: This study demonstratesthe potential of RAG-based LLMs to address the diverse and culture-specific worries of Korean adolescents. By integratingexternal knowledge and offering personalized support, the proposed system provides a scalable approach to enhancing mentalhealth interventions for adolescents. Future research should concentrate on expanding the dataset and improving multiturnconversational capabilities to deliver even more comprehensive support.
Abstract
Objectives: This research aimed to develop a retrieval-augmented generation (RAG) based large language model (LLM)system that offers personalized and reliable responses to a wide range of concerns raised by Korean adolescents. Our workfocuses on building a culturally reflective dataset and on designing and validating the system’s effectiveness by comparing theanswer quality of RAG-based models with non-RAG models. Methods: Data were collected from the NAVER KnowledgeiN platform, concentrating on posts that featured adolescents’ questions and corresponding expert responses during the period2014–2024. The dataset comprises 3,874 cases, categorized by key negative emotions and the primary sources of worry. The data were processed to remove irrelevant or redundant content and then classified into general and detailed causes. TheRAG-based model employed FAISS for similarity-based retrieval of the top three reference cases and used GPT-4o mini forresponse generation. The responses generated with and without RAG were evaluated using several metrics. Results: RAGbasedresponses outperformed non-RAG responses across all evaluation metrics. Key findings indicate that RAG-based responsesdelivered more specific, empathetic, and actionable guidance, particularly when addressing complex emotional andsituational concerns. The analysis revealed that family relationships, peer interactions, and academic stress are significant factorsaffecting adolescents’ worries, with depression and stress frequently co-occurring. Conclusions: This study demonstratesthe potential of RAG-based LLMs to address the diverse and culture-specific worries of Korean adolescents. By integratingexternal knowledge and offering personalized support, the proposed system provides a scalable approach to enhancing mentalhealth interventions for adolescents. Future research should concentrate on expanding the dataset and improving multiturnconversational capabilities to deliver even more comprehensive support.