BERT-Based Logits Ensemble Model for Gender Bias and Hate Speech Detection
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Malicious hate speech and gender bias comments are common in online communities, causing social problemsin our society. Gender bias and hate speech detection has been investigated. However, it is difficult becausethere are diverse ways to express them in words. To solve this problem, we attempted to detect malicious commentsin a Korean hate speech dataset constructed in 2020. We explored bidirectional encoder representationsfrom transformers (BERT)-based deep learning models utilizing hyperparameter tuning, data sampling, andlogits ensembles with a label distribution. We evaluated our model in Kaggle competitions for gender bias,general bias, and hate speech detection. For gender bias detection, an F1-score of 0.7711 was achieved usingan ensemble of the Soongsil-BERT and KcELECTRA models. The general bias task included the gender biastask, and the ensemble model achieved the best F1-score of 0.7166.
Abstract
Malicious hate speech and gender bias comments are common in online communities, causing social problemsin our society. Gender bias and hate speech detection has been investigated. However, it is difficult becausethere are diverse ways to express them in words. To solve this problem, we attempted to detect malicious commentsin a Korean hate speech dataset constructed in 2020. We explored bidirectional encoder representationsfrom transformers (BERT)-based deep learning models utilizing hyperparameter tuning, data sampling, andlogits ensembles with a label distribution. We evaluated our model in Kaggle competitions for gender bias,general bias, and hate speech detection. For gender bias detection, an F1-score of 0.7711 was achieved usingan ensemble of the Soongsil-BERT and KcELECTRA models. The general bias task included the gender biastask, and the ensemble model achieved the best F1-score of 0.7166.
참고문헌 (23)
- A. Schmidt, "A survey on hate speech detection using natural language processing", Proceedings of the 5th International Workshop on Natural Language Processing for Social Media, pp.1~10 (2017)
- H. Zhong, "Content-driven detection of cyberbullying on the Instagram social network", Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI), pp.3952~3958 (2016)
- J. H. Park, "One-step and two-step classification for abusive language detection on twitter", Proceedings of the 1st Workshop on Abusive Language Online, pp.41~45 (2017)
- H. Mulki, "L-HSAB: a Levantine twitter dataset for hate speech and abusive language", Proceedings of the 3rd Workshop on Abusive Language Online, pp.111~118 (2019)
- J. Zhao, "Men also like shopping: reducing gender bias amplification using corpus-level constraints", Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.2979~2989 (2017)
- S. Kiritchenko, "Examining gender and race bias in two hundred sentiment analysis systems", Proceedings of the 7th Joint Conference on Lexical and Computational Semantics, pp.43~53 (2018)
- Kaiji Lu, "Logic, Language, and Security", pp.189~202 (2020), Springer
- Zo Ahmed, "Tackling racial bias in automated online hate detection: Towards fair and accurate detection of hateful users with geometric deep learning", EPJ Data Science, 11(1) (2022), Springer Science and Business Media LLC[DOI]
- C. Nobata, "Abusive language detection in online user content", Proceedings of the 25th International Conference on World Wide Web, pp.145~153 (2016)
- H. Rizwan, "Hate-speech and offensive language detection in roman Urdu", Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.2512~2522 (2020)
- Patricia Chiril, "Emotionally Informed Hate Speech Detection: A Multi-target Perspective", Cognitive Computation, 14(1), pp.322~352 (2021), Springer Science and Business Media LLC[DOI]
- Paula Fortuna, "How well do hate speech, toxicity, abusive and offensive language classification models generalize across datasets?", Information Processing & Management, 58(3), pp.102524~ (2021), Elsevier BV[DOI]
- Nanlir Sallau Mullah, "Advances in Machine Learning Algorithms for Hate Speech Detection in Social Media: A Review", IEEE Access, 9, pp.88364~88376 (2021), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- P. Badjatiya, "Deep learning for hate speech detection in tweets", Proceedings of the 26th International Conference on World Wide Web Companion, pp.759~760 (2017)
- Raghad Alshalan, "A Deep Learning Approach for Automatic Hate Speech Detection in the Saudi Twittersphere", Applied Sciences, 10(23), pp.8614~ (2020), MDPI AG[DOI]
- Omar Sharif, "Tackling cyber-aggression: Identification and fine-grained categorization of aggressive texts on social media using weighted ensemble of transformers", Neurocomputing, 490, pp.462~481 (2022), Elsevier BV[DOI]
- "BERT: pre-training of deep bidirectional transformers for language understanding" (2018)[URL]
- J. Moon, "BEEP! Korean corpus of online news comments for toxic speech detection", Proceedings of the 8th International Workshop on Natural Language Processing for Social Media, pp.25~31 (2020)
- W. I. Cho, "On measuring gender bias in translation of gender-neutral pronouns", Proceedings of the 1st Workshop on Gender Bias in Natural Language Processing, pp.173~181 (2019)
- W. I. Cho, "A study on the construction of Korean Hate speech corpus: based on the attributes of online toxic comments", Proceedings of Annual Conference on Human and Language Technology, pp.298~303 (2020)
- S. Zimmerman, "Improving hate speech detection with deep learning ensembles", Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC), pp.2546~2553 (2018)
- M. R. Karim, "DeepHateExplainer: explainable hate speech detection in under-resourced Bengali language", Proceedings of 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), pp.1~10 (2021)
- Kellie Webster, "Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns", Transactions of the Association for Computational Linguistics, 6, pp.605~617 (2018), MIT Press - Journals[DOI]