도메인 특화된 경량 언어 모델을 이용한 학술 논문 이해도 분석
Understanding Analysis of Research Publications with Domain-specific Lightweight Language Models
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
최근 발전한 언어 모델을 이용하여 다양한 전문 분야의 연구에 활용하고자 하는 시도가 계속되고 있다. 본 논문에서는 경영학 도메인에 특화된 경량 언어 모델을 이용하여 경영학 논문의 초록을 이해하고 향후 연구에 도움이 될 수 있음을 보였다. 먼저, 경영학에 특화된 언어 모델의 논문 초록 문장에 대하여 Perplexity로 측정되는 이해도가 특화되지 않은 모델에 비해서 60% 정도 향상되었음을 보였다. 이를 이용하여 각 논문의 유사도를 비교함으로써 학술지 사이의 관계를 파악하였다. 마지막으로 특화 모델이 경영학적으로 의미를 갖는 단어를 선택함에 있어서 70%의 정확도를 보였는데, 이는 학문적인 가설 도출이나 지식 축적에 도움이 될 수 있다. 본 논문의 결과는 대규모 모델을 사용하기 어려운 경영학 분야의 연구뿐만 아니라 다른 전문 분야의 연구에도 적용되어, 학문과 기술의 발전을 가속화 할 수 있을 것으로 기대된다.
Abstract
Recent advancements in language models have led to ongoing efforts to apply them in various specialized research fields. In this paper, we demonstrate the potential of lightweight language models specific in the domain of management to understand the abstracts of research papers. First, we show that the understanding of sentences in abstracts by the specific language models improve by approximately 60% compared to general models. Using these models, the similarities between papers were compared, revealing relationships among academic journals. Finally, the specific models achieve an accuracy of 70% in selecting words that are meaningful in management contexts, which could assist in formulating academic hypotheses and accumulating knowledge. The findings of this paper are expected to apply not only to research in the field of management but also to other specific fields, thereby accelerating the advancement of academia and technology.
참고문헌 (20)
- M. Raissi, "Physics-informed neural networks : A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations", Journal of Computational Physics, 378, pp.686~707 (2019)
- J. Seo, "Grid-wise simulation acceleration of the electromagnetic fields of 2D optical devices using super-resolution", Scientific Reports, 13 (2023)
- H.S, Hwang, "Deep reinforcement learning with a critic-value-based branch tree for the inverse design of two-dimensional optical devices", Applied Soft Computing, 127 (2022)
- A. Radford, "Improving Language Understanding by Generative Pre-Training", preprint, ~ (2018)
- J. Devlin, "BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding", NAACL (2019)
- J. Seok, "Improving Small-scale Sentiment Analysis through Data Augmentation using Generative Artificial Intelligence", The Journal of Korean Institute of Next Generation Computing, 19(6) (2023)
- G. Xiao, "SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models", ICML (2023)
- V. Sanh, "DistilBERT, a distilled version of BERT : smaller, faster, cheaper and lighter", NeurIPS (2019)
- M. Sun, "A Simple and Effective Pruning Approach for Large Language Models", ICLR (2024)
- J. H. Kim, "FinBERT Fine-Tuning for Sentiment Analysis : Exploring the Effectiveness of Datasets and Hyperparameters", Journal of Internet Computing and Services, 24(4), pp.127~135 (2023)
- J. Lee, "BioBERT : a pre-trained biomedical language representation model for biomedical text mining", Bioinformatics, 36, pp.1234~1240 (2020)
- S. J. Kim, "BERT based Relation Extraction Model and Knowledge Graph for Economic Knowledge Analysis", The Journal of Korean Institute of Next Generation Computing, 18(6) (2022)
- M. Harper, "The MovieLens Datasets : History and Context", ACM Transactions on Interactive Intelligent Systems, 5(4), pp.1~19 (2015)