Winogender 데이터 세트를 활용한 한국어 언어모델의 성 편향성 분석
Analysis of Gender Bias in Korean Language Models Utilizing the Winogender Dataset
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This paper investigates the impact of gender bias inoccupation nouns and adjectives in Korean language models on the genderreferred to by pronouns, utilizing the Winogender dataset. The experiment wasconducted in three ways: Measuring surprisal scores in Korean and English usingencoder models, and conducting experiments with a decoder model, namely isChatGPT-4, to test its responses. The encoder models showed that, regardlessof the gender bias differences in occupation nouns and adjectives, the malepronoun was more naturally used in sentences than the female pronoun. Onthe other hand, the decoder model detected gender bias especially in sentencescontaining adjectives. This result identifies the influence of gender imbalancein training data and the functional differences between the language generationmodel and the language comprehension model. This study suggests to constructa unique dataset that reflects the characteristics of Korean, in order to moreeffectively analyze gender bias in Korean language models.
Abstract
This paper investigates the impact of gender bias inoccupation nouns and adjectives in Korean language models on the genderreferred to by pronouns, utilizing the Winogender dataset. The experiment wasconducted in three ways: Measuring surprisal scores in Korean and English usingencoder models, and conducting experiments with a decoder model, namely isChatGPT-4, to test its responses. The encoder models showed that, regardlessof the gender bias differences in occupation nouns and adjectives, the malepronoun was more naturally used in sentences than the female pronoun. Onthe other hand, the decoder model detected gender bias especially in sentencescontaining adjectives. This result identifies the influence of gender imbalancein training data and the functional differences between the language generationmodel and the language comprehension model. This study suggests to constructa unique dataset that reflects the characteristics of Korean, in order to moreeffectively analyze gender bias in Korean language models.
참고문헌 (33)
- Social Communication Research: A case study of sexist verbalizations and a study of countermeasures, "Ann, Sangsu", National Institute of the Korean Language and Korean Women’s Development Institute, ~ (2007)
- Bolukbasi, Tolga, "Man is to computer programmer as woman is to homemaker? debiasing word embeddings", Advances in Neural Information Processing Systems, 29 (2016)
- Caliskan, Aylin, "Gender bias in word embeddings: A comprehensive analysis of frequency, syntax, and semantics", Proceedings of the 2022 AAAI, 2022, 156~170
- Cho, Won Ik, "Towards cross-lingual generalization of translation gender bias", Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp.449~457 (2021)
- Choi, Kiseon, "Validating corpus integration", National Institute of the Korean Language, ~ (2020)
- Devlin, Jacob, "Bert:Pre-training of deep bidirectional transformers for language understanding", Proceedings of naacL-HLT 2019, 1, pp.4171~4186 (2019)
- Fei, Li, "Do language models know the distinctions between men and women? An insight into the relationships between gender and profession Through ‘Fill-Mask’ task", Hangul and Korean Information Processing, pp.3~9 (2022)
- Floridi, Luciano, "GPT-3 : Its nature, scope, limits, and consequences", Minds and Machines, 30, pp.681~694 (2020)
- Graells-Garrido, Eduardo, "First women, second sex: Gender bias in Wikipedia", Proceedings of the 26th ACM conference on hypertext & social media, pp.165~174 (2015)
- Hale, John, "A probabilistic Earley parser as a psycholinguistic model", Second Meeting of the North American Chapter of the Association for Computational Linguistics, 2, pp.159~166 (2001)
- Han, Na-Rae, "Korean zero pronouns: Analysis and resolution", Ph D, pp.2006
- 강소영, "성별 대상어의 의미와 성별 분포와의 상관관계-남성다움과 여성다움의 연상어휘를 중심으로-", 여성학논집, 27(1), pp.37~72 (2010)
- Mass media gender equality monitoring project report, "Korea Institute for Gender Equality Education", ~ (2017)
- Kwak, Yongjin, "Constructing a subject zero anaphora resolution corpus", National Institute of Korean Language, ~ (2020)
- 이준범, "KcBERT: 한국어 댓글로 학습한 BERT", 한글 및 한국어정보처리, pp.437~440 (2020)
- Levy, Roger, "Expectation-based syntactic comprehension", Cognition, 106(3), pp.1126~1177 (2008)
- Moss-Racusin, Corinne, "Science faculty's subtle gender biases favor male students", PNAS Proceedings of the National Academy of Sciences of the United States of America, 109(41), pp.16474~16479 (2012)
- Park, Eunyoung, "An Investigation on the coreference assignment of pronouns", MA, pp.1988
- 2021 Job information in Korea – 2021 KNOW research report -, "Park, Gayeol", Research Project Report. Korea Empolyment Information Service, 1~157 (2022)
- Park, Jinho, "Features of the Korean pronoun system from a typological perspective", Korean Linguistics, 50, pp.115~147 (2007)
- Petersen, Erika, "Lexical Semantics with large language models: A case study of English “break", Findings of the Association for Computational Linguistics: EACL 2023, pp.490~511 (2023)
- Shin, Unsub, "BERT learns more than word frequency information: A case study of do-be constructions", Korean Journal of Linguistics, 47(3), pp.467~489 (2022)
- 송상헌, "딥러닝 언어모델과Surprisal을 활용한 언어분석", 인공지능인문학연구, 12, pp.9~39 (2023)
- "Korean BERT pre-trained cased (KoBERT)" (2019)[URL]
- Vaswani, Ashish, "Attention is all you need", Advances in neural information processing systems, 30 (2017)