생성-선정을 통한 텍스트 증강 프레임워크
TAGS: Text Augmentation with Generation and Selection
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
텍스트 증강은 자연어처리 모델의 성능 향상을 목적으로 원본 텍스트의 변환, 생성을 통하여 새로운 증강 텍스트를 생성하는 방법론이다. 기존연구된 기법들은 표현적 다양성 부족, 의미 왜곡 , 한정적인 양의 증강 텍스트와 같은 한계점이 존재한다. 거대언어모델과 few-shot learning을활용한 텍스트 증강은 이러한 한계점의 극복이 가능하지만, 잘못된 생성으로 인한 노이즈 발생의 위험성이 존재한다. 본 논문에서는 여러 후보텍스트를 생성하고 적합한 텍스트를 증강 텍스트로 선정하는 TAGS를 제안한다. TAGS는 기존 텍스트 few shot learning을 통해 다양한 표현을생성하면서 대조 학습과 유사도 비교를 통해 원본 텍스트가 적더라도 적합한 데이터를 효과적으로 선정한다. 이를 텍스트 증강이 필수적인 업무용챗봇 데이터에 적용하여 60배 이상의 양적 향상을 달성하였다. 또한 증강 텍스트의 질적 향상을 확인하기 위해 실제 생성된 텍스트를 분석하여원본 텍스트에 비해 의미론적, 표현적으로 다양한 텍스트를 생성함을 확인하였으며, 증강 텍스트로 실제 분류 모델을 학습하고 실험하여 실질적으로자연어처리 모델 성능 향상에 도움이 되는 것을 확인하였다.
Abstract
Text augmentation is a methodology that creates new augmented texts by transforming or generating original texts for the purposeof improving the performance of NLP models. However existing text augmentation techniques have limitations such as lack of expressivediversity semantic distortion and limited number of augmented texts. Recently text augmentation using large language models and few-shotlearning can overcome these limitations but there is also a risk of noise generation due to incorrect generation. In this paper, we proposea text augmentation method called TAGS that generates multiple candidate texts and selects the appropriate text as the augmented text. TAGS generates various expressions using few-shot learning while effectively selecting suitable data even with a small amount of originaltext by using contrastive learning and similarity comparison. We applied this method to task-oriented chatbot data and achieved morethan sixty times quantitative improvement. We also analyzed the generated texts to confirm that they produced semantically andexpressively diverse texts compared to the original texts. Moreover, we trained and evaluated a classification model using the augmentedtexts and showed that it improved the performance by more than 0.1915, confirming that it helps to improve the actual model performance.
참고문헌 (17)
- W. Y. Wang, "That’s so annoying!!! : A lexical and frame-semantic embedding based data augmentation approach to automatic categorization of annoying behaviors using #petpeeve tweets", Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing, pp.2557~2563 (2015), Association for Computational Linguistics
- E. Pavlick, "PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification", Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp.425~430 (2015), Association for Computational Linguistics
- R. Sennrich, "Improving neural machine translation models with monolingual data", Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.86~96 (2016), Association for Computational Linguistics
- M. Suzgun, "Promptand-rerank : A method for zero-shot and few-shot arbitrary textual style transfer with small language models", Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp.2195~2222 (2022), Association for Computational Linguistics
- A. Fabbri, "Improving zero and few-shot abstractive summarization with intermediate fine-tuning and data augmentation", Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.704~717 (2021), Association for Computational Linguistics
- V. Gangal, "Nareor: The narrative reordering problem", Proceedings of the AAAI Conference on Artificial Intelligence, pp.10645~10653 (2022)
- H. Dai, "AugGPT: Leveraging ChatGPT for Text Data Augmentation", ~ (2023)
- Y. Wang, "Generalizing from a few examples : A survey on few-shot learning", ACM Computing Surveys(CSUR), 53(3), pp.1~34 (2020)
- W. Yin, "Benchmarking zero-shot text classification : Datasets, evaluation and entailment approach", CoRR, abs, pp.1909.00161 (2019), ~
- L. Cui, "Templatebased named entity recognition using BART", Findings of the Association for Computational Linguistics: ACLIJCNLP 2021, pp.1835~1845 (2021)
- N. S. Keskar, "Ctrl : A conditional transformer language model for controllable generation", arXiv preprint arXiv : 1909. 05858, ~ (2019)
- T. Gao, "SimCSE : Simple contrastive learning of sentence embeddings", 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021 (2021), Association for Computational Linguistics (ACL)
- C. Liu, "Learning a few-shot embedding model with contrastive learning", Proceedings of the AAAI Conference on Artificial Intelligence, 35(10), pp.8635~8643 (2021)
- J. Zhang, "Few-shot intent detection via contrastive pre-training and fine-tuning", Proceedings of the 2021Conference on Empirical Methods in Natural Language Processing, pp.1906~1912 (2021)
- "KoELECTRA: Pretrained ELECTRA Model for Korean"[URL]
- "KoGPT: KakaoBrain Korean(hangul)Generative Pre-trained Transformer"[URL]
- Lee, Junbum, "KcBERT: Korean Comments BERT", Proceedings of the 32nd Annual Conference on Human and Cognitive Language Technology, pp.437~440 (2020)