다국면 Rasch 모형을 적용한 대학 교양 말하기 과목의 동료평가 연구 - 설득적 말하기와 정보전달 말하기를 중심으로 -
Peer Evaluation Study in the Liberal Arts Speaking Subject of University Applying Multi-faceted Rasch Model: Focusing on Persuasive and Informative Speaking
0
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
This study aims to verify the reliability of peer evaluation during persuasive and informative speaking in university liberal arts speech classes. To this end, it presented 46 university students with persuasive and informative speaking assignments and conducted a peer evaluation during their presentations. Subsequently, the students’ evaluation results were analyzed by applying the multi-faceted Rasch model. The analysis focused on evaluator appropriateness, evaluator bias, and the appropriateness of criteria and evaluation standards. The results of the analysis showed that 6 evaluators demonstrated inappropriate evaluation behavior during the persuasive speaking evaluation, and 13 were confirmed as over-fit and unfit evaluators in the informative speaking evaluation. Further, a greater over-fit reaction was found in the informative speaking evaluation for severity variability and evaluator agreement. Moreover, differences were found in using evaluation standards. The most severe evaluation was for the contents of persuasive speaking, and the most severe reaction was found in the delivery of informative speaking. In addition, a tendency for over-fit evaluation was found in the organization criteria for informative speaking. Thus, the results of the research highlighted different evaluation tendencies from the same evaluator based on the genre of speech. Therefore, there is a need for evaluator training based on genre.
Abstract
This study aims to verify the reliability of peer evaluation during persuasive and informative speaking in university liberal arts speech classes. To this end, it presented 46 university students with persuasive and informative speaking assignments and conducted a peer evaluation during their presentations. Subsequently, the students’ evaluation results were analyzed by applying the multi-faceted Rasch model. The analysis focused on evaluator appropriateness, evaluator bias, and the appropriateness of criteria and evaluation standards. The results of the analysis showed that 6 evaluators demonstrated inappropriate evaluation behavior during the persuasive speaking evaluation, and 13 were confirmed as over-fit and unfit evaluators in the informative speaking evaluation. Further, a greater over-fit reaction was found in the informative speaking evaluation for severity variability and evaluator agreement. Moreover, differences were found in using evaluation standards. The most severe evaluation was for the contents of persuasive speaking, and the most severe reaction was found in the delivery of informative speaking. In addition, a tendency for over-fit evaluation was found in the organization criteria for informative speaking. Thus, the results of the research highlighted different evaluation tendencies from the same evaluator based on the genre of speech. Therefore, there is a need for evaluator training based on genre.
참고문헌 (38)
- 김태경, "설명적 프레젠테이션의 평가 항목 설정", 한국언어문화, (52), pp.81~101 (2013), 한국언어문화학회
- 김평원, "의사소통 과정으로서의 말하기 평가 : 대규모 성인 말하기 평가 시스템을 중심으로", 화법연구, 11, pp.9~34 (2007)
- 김평원, "말하기 평가의 분석 모형 연구", 박사, 국어교육, pp.2010
- 김현정, "대학 ‘말하기’ 교과 운영 현황과 개선 방안", 교양교육연구, 7(3), pp.597~625 (2013), 한국교양교육학회
- 민병곤, "성인의 국어 말하기 능력 평가 결과 분석", 화법연구, (24), pp.101~136 (2014), 한국화법학회[DOI]
- 박미영, "화법 교육 내용 선정을 위한 담화 구성 연구 - 학습자 자기 평가를 바탕으로 -", 문법 교육, 9, pp.159~184 (2008), 한국문법교육학회
- 박영목, "국어과 교수 학습 방법 탐구" (1995), 교학사
- 박영민, "예비국어교사의 중학생 논설문 평가에서 발견되는 엄격성 및 일관성의 특성", 국어교육학연구, (43), pp.253~283 (2012), 국어교육학회[DOI]
- 박재현, "화법 평가의 쟁점과 발전 방향", 국어교육학연구, 50(2), pp.5~25 (2015), 국어교육학회[DOI]
- 박종임, "국어교사의 쓰기 평가 특성 연구", 박사, 국어교육전공, pp.2013
- 서영진, "작문 활동에서 동료 피드백 의견의 유형별 타당도 연구", 국어교육학연구, (44), pp.369~395 (2012), 국어교육학회[DOI]
- 서종훈, "말하기 수행에 드러난 반성적 사고 양상 - 고1 학습자들의 공적인 자리에서의 말하기를 중심으로", 새국어교육, (83), pp.239~260 (2009), 한국국어교육학회[DOI]
- 손세모돌, "설명 능력 평가 방법", 화법연구, 11, pp.67~108 (2007)
- 이도영, "말하기 평가 목표와 평가 기준 설정 방안", 교육논총, 30(2), pp.19~35 (2010)
- 이삼형, "국어교육학과 사고" (2005), 역락
- 이윤빈, "대학생 필자의 동료 피드백 수용 양상 연구 - 동료 피드백에 대한 필자의 반응 및 수정고 반영 양상을 중심으로", 작문연구, (21), pp.201~234 (2014), 한국작문학회
- 장민정, "대학생 발표에서 동료 피드백의 가치에 대한 연구 - 동료 피드백과 자기성찰일지 분석", 한국언어문화, (62), pp.357~387 (2017), 한국언어문화학회
- 장소영, "언어교육평가 연구를 위한 FACETS 프로그램: 기초과정편" (2009), 글로벌콘텐츠
- 전은주, "말하기 듣기 영역 수행 평가의 방법", 어문학교육, 27, pp.39~65 (2003)
- 전은주, "대학 화법 교육의 내용 구성", 국어교육학연구, (30), pp.403~430 (2007), 국어교육학회[DOI]
- 전은진, "대학생 말하기 교육의 현황과 개선 방안", 인문과학연구, (32), pp.167~191 (2012), 인문과학연구소[DOI]
- 정민주, "대학생 연설 화자들의 동료 피드백 수용 양상에 관한 질적 고찰", 새국어교육, (93), pp.47~82 (2012), 한국국어교육학회[DOI]
- 정보배, "고등학교 학습자를 위한 말하기 상호평가의 기준과 방법 연구", 교사교육연구, 57(2), pp.260~278 (2018), 과학교육연구소[DOI]
- 조재윤, "말하기 평가의 요소 설정 연구- 델파이 기법을 이용하여", 새국어교육, (75), pp.337~358 (2007), 한국국어교육학회[DOI]
- 최영인, "설득 화법의 청중 평가 양상에 대한 고찰 - 영향력 평가와 분석적 평가를 중심으로 -", 국어교육연구, (33), pp.319~352 (2014), 국어교육연구소[DOI]
- 최영인, "국어과 교사들의 설득 담화 평가 양상 및 인식에 관한 고찰", 국어교육연구, (29), pp.219~250 (2012), 국어교육연구소[DOI]
- 최윤희, "동료 및 자기 평가 피드백을 통한 대학생 말하기 교육 방안", 한국콘텐츠학회 논문지, 20(1), pp.573~580 (2020), 한국콘텐츠학회[DOI]
- Adachi, C., "Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education", Assessment & Evaluation in Higher Education, 43(2), pp.294~306 (2018)
- Butterfield, E. C., "Environmental, cognitive, and metacognitive influences on text revision : Assessing the evidence", Educational Psychology Review, 8(3), pp.239~297 (1996)
- Eckes, T., "Introduction to many-facet Rasch measurement", Frankfurt am. Language Testing and Evaluation, 22, pp.1~6 (2011)
- Fauzan, U., "Enhancing speaking ability of EFL students through debate and peer assessment", EFL journal, 1(1), pp.49~57 (2016)
- "Winsteps", 2003
- Linacre, J. M., "A user’s guide to FACETS Rasch-model computer programs" (2014), Mesa Press
- McNamara, T. F., "Measuring second language performance" (1996), Addison Wesley Longman
- Reinholz, D., "The assessment cycle : A model for learning through peer assessment", Assessment & Evaluation in Higher Education, 41(2), pp.301~315 (2016)
- Thompson, G., "Introducing functional grammar" (2013), Routledge
- Trace, J., ""I can see that" : Developing shared rubric category interpretations through score negotiation", Assessing Writing, 30, pp.32~43 (2016)
- Wind, S. A., "Do raters use rating scale categories consistently across analytic rubric domains in writing assessment?", Assessing Writing, 43, pp.1~14 (2020)