한국어 쓰기 평가에서의 채점 편향 분석 연구 -논설문에서의 찬반 입장을 중심으로-
Analysis of Rating Bias in Korean Writing Assessment: Focusing on Proponent and Opponent Stances in Persuasive Writing.
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
The purpose of this study is to analyze the phenomenon of rater's in-group bias in the assessment of persuasive writing. To this end, an experiment was conducted in which 18 Korean teachers rated 20 essays written by fifth-level Korean learners. The objective was to determine whether raters assigned higher scores to writers who shared the same viewpoints as themselves. Statistical tests such as t-tests and one-way ANOVAs were employed to analyze the data. The results of the experimental analysis can be summarized as follows. Firstly, both proponent and opponent raters of a given position tended to assign higher scores to essays reflecting their own perspective, indicating the presence of in-group bias in essay scoring. Secondly, the in-group bias was more pronounced among the opponent raters. Lastly, no statistically significant difference in scores was observed for `Test Constructs 3', which encompassed relatively objective aspects such as grammar, vocabulary, and spelling. The results of this study can be used to improve intra-rater reliability in Korean writing assessment.
Abstract
The purpose of this study is to analyze the phenomenon of rater's in-group bias in the assessment of persuasive writing. To this end, an experiment was conducted in which 18 Korean teachers rated 20 essays written by fifth-level Korean learners. The objective was to determine whether raters assigned higher scores to writers who shared the same viewpoints as themselves. Statistical tests such as t-tests and one-way ANOVAs were employed to analyze the data. The results of the experimental analysis can be summarized as follows. Firstly, both proponent and opponent raters of a given position tended to assign higher scores to essays reflecting their own perspective, indicating the presence of in-group bias in essay scoring. Secondly, the in-group bias was more pronounced among the opponent raters. Lastly, no statistically significant difference in scores was observed for `Test Constructs 3', which encompassed relatively objective aspects such as grammar, vocabulary, and spelling. The results of this study can be used to improve intra-rater reliability in Korean writing assessment.
참고문헌 (25)
- 강현화, "한국어 평가론" (2023), 한국문화사
- 김성숙, "학문 목적 한국어 쓰기 능력에 대한 분석적 평가의 일반화가능도 검증", 한국어교육, 22(3), pp.29~48 (2011), 국제한국어교육학회[DOI]
- 김수현, "한국어 학습자의 쓰기 수행 평가 신뢰도 분석 - 다국면 라쉬 모형을 사용하여", 한국어 의미학, 61, pp.1~31 (2018), 한국어의미학회
- 김은실, "학문목적 한국어 쓰기 평가 오차요인 분석-일반화가능도이론을 적용하여-", 한국리터러시학회 학술대회자료집, pp.200~217 (2020), 한국리터러시학회
- 김지영, "한국어 말하기 평가 채점자의 채점 경향 연구", 박사, pp.2018
- 박지수, "한국어 쓰기 평가의 채점자 요인 연구", 석사, pp.2021
- 백유진, "논설문 채점에 나타난 국어 교사의 채점 편향의 특성 분석: 텍스트 특징에 따른 채점 편향 분석을 중심으로", 청람어문교육, (76), pp.67~101 (2020), 청람어문교육학회[DOI]
- 안수현, "한국어능력시험(TOPIK) 쓰기 평가의 채점 특성 연구", 한국어교육, 28(1), pp.173~196 (2017), 국제한국어교육학회[DOI]
- 원미진, "한국어 말하기 평가 개발을 위한 채점 경향 분석 연구", 외국어로서의 한국어교육, 47, pp.169~192 (2017), 언어연구교육원 한국어학당
- 이성준, "한국어 말하기 평가의 채점 변인 연구", 박사, pp.2019
- 이은하, "학문 목적 한국어 쓰기 수행 평가의 채점자 신뢰도 검증 - 다국면 Rasch 모형을 활용하여 -", 새국어교육, (103), pp.311~354 (2015), 한국국어교육학회[DOI]
- 이향, "말하기 수행 평가에서 발음 범주 채점의 최적화 방안 연구 -일반화가능도 이론을 활용하여-", 한국어교육, 23(2), pp.301~329 (2012), 국제한국어교육학회[DOI]
- 장미, "국어교사의 주제 찬반 입장에 따른 논설문 평가 결과 분석", 석사, pp.2017
- 한강우, "한국어능력시험 쓰기 영역 54번 문항 답안 작성 방안 연구 : 글의 구성 방식을 중심으로", 문화교류와 다문화교육, 11(4), pp.201~233 (2022), 한국국제문화교류학회
- 황지유, "한국어 말하기 평가의 최적의 채점자 배정 방식 연구 - 일반화가능도 이론을 중심으로 -", 언어사실과 관점, 50, pp.99~122 (2020), 언어정보연구원[DOI]
- Billig, M., "Social categorization and similarity in intergroup behaviour", European Journal of Social Psychology, 3(1), pp.27~52 (1973)
- Charney, D., "The validity of using holistic scoring to evaluate writing: a critical overview", Research in the Teaching of English, 18(1), pp.65~81 (1984)
- Dobbs, M., "Outgroup accountability in the minimal group paradigm: implications for aversive discrimination and social identity theory", Personality and Social Psychology Bulletin, 27(3), pp.355~364 (2001)
- Du, Y., "Differential facet functioning detection in direct writing assessment", Paper presented at the Annual Meeting of American Educational Research Association (1996)
- Eckes, T., "Introduction to many-facet Rasch measurement: analyzing and evaluating rater-mediated assessments" (2015), Peter Lang
- Falk, C. F., "Cultural variation in the minimal group effect", Journal of Cross-Cultural Psychology, 45(2), pp.265~281 (2014)
- Ferguson, C. K., "Significant factors in overevaluation of own-group's product", The Journal of Abnormal & Social Psychology, 69(2), pp.223~228 (1964)
- Schaefer, E., "Rater bias patterns in an EFL writing assessment", Language Testing, 25(4), pp.465~493 (2008)
- Shaw, S. D., "Examining writing: research and practice in assessing second language writing" (2007), Cambridge University Press
- Sherif, M., "Intergroup conflict and cooperation: the Robbers Cave experiment" (1961), University of Oklahoma Book Exchange