변형된 비속어 탐지를 위한 토큰 기반의 분류 및 데이터셋
Token-Based Classification and Dataset Construction for Detecting Modified Profanity
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
기존의 비속어 탐지 방법들은 의도적으로 변형된 비속어를 식별하는 데 한계가 있다. 이 논문에서는 자연어 처리의 한 분야인 개체명 인식에기반한 새로운 방법을 소개한다. 우리는 시퀀스 레이블링을 이용한 비속어 탐지 기법을 개발하고, 이를 위해 한국어 악성 댓글 중 일부 비속어를레이블링하여 직접 데이터셋을 구축하여 실험을 수행하였다. 또한 모델의 성능을 향상시키기 위하여 거대 언어 모델중 하나인 ChatGPT를 활용해한국어 혐오발언 데이터셋의 일부를 레이블링을 하는 방식으로 데이터셋을 증강하여 학습을 진행하였고, 이 과정에서 거대 언어 모델이 생성한데이터셋을 인간이 필터링 하는 것만으로도 성능을 향상시킬 수 있음을 확인하였다. 이를 통해 데이터셋 증강 과정에는 여전히 인간의 관리감독이필요함을 제시하였다.
Abstract
Traditional profanity detection methods have limitations in identifying intentionally altered profanities. This paper introduces a newmethod based on Named Entity Recognition, a subfield of Natural Language Processing. We developed a profanity detection techniqueusing sequence labeling, for which we constructed a dataset by labeling some profanities in Korean malicious comments and conductedexperiments. Additionally, to enhance the model's performance, we augmented the dataset by labeling parts of a Korean hate speechdataset using one of the large language models, ChatGPT, and conducted training. During this process, we confirmed that filtering thedataset created by the large language model by humans alone could improve performance. This suggests that human oversight is stillnecessary in the dataset augmentation process.
참고문헌 (16)
- H. -G. Kim, "An empirical study of SNS Language Violence(Expletives, Slang)", Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities and Sociology, 9(3), pp.99~108 (2019)
- H. -Y. Kang, "The mediating effect of self-control on the relationship between adolescents’ usage of slang and aggression", Journal of the Korea Institute of Youth Facility and Environment, 19(4), pp.33~42 (2021)
- D. Kim, "A study on the translation strategies of swearing in the game : Focusing on the semanticㆍphonological", morphological variations of game forbidden words, 2023, pp.21(3), 5~25
- J. Li, "A survey on deep learning for named entity recognition : Extended abstract", IEEE 39th International Conference on Data Engineering (ICDE), pp.3817~3818 (2023)
- A. Chaudhari, "Profanity detection and removal in videos using machine learning", 2021 6th International Conference on Inventive Computation Technologies (ICICT), pp.572~576 (2021)
- J. Kim, "Developing a connection restrictions filtering system for websites based on swear words extraction", Journal of KIISE, 46(12), pp.1272~1278 (2019)
- Y. -L. Choi, "Development of profanity response module for artificial intelligence service for english education", Journal of Korean Institute of Intelligent Systems, 31(3), pp.192~197 (2021)
- M. Yi, "Method of profanity detection using word embedding and LSTM", Mobile Information Systems, 2021 (2021)
- Y. Kim, "Swear word detection through convolutional neural network", Proceedings of the Korea Information Processing Society Conference, 28(2), pp.685~686 (2021)
- S. Lee, "Analyzing the classification results for korean hatespeech and bias detection models in malicious comment dataset", Journal of the Korean Institute of Industrial Engineers, 48(6), pp.636~643 (2022)
- J. H. Choi, "Design and implementation of abuse sentence detecting system based on deep learning", Master, pp.2020
- J. Yoo, "A study on the improvement of text filtering based on image learning", Master, pp.2019
- T. Brown, "Language models are few-shot learners", Advances in Neural Information Processing Systems, f33, pp.1877~1901 (2020)
- S. Ko, "Token Classification for Detecting Modified Profanity", Proceedings of the Annual Conference of Korea Information Processing Society Conference (KIPS), 30(2), pp.498~499 (2023)