척도의 활용이 영작문 평가에 미치는 영향—예비영어교사의 평가전문성 신장의 관점에서
The Effect of Different Types of Rubric on Writing Assessment: Focus on Development of Pre-Service English Teachers’ Assessment Expertise
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
This study examines how different types of rubrics affect pre-service teachers’ assessment expertise in writing assessment. The current study has three research questions: 1) Is there a significant difference in the reliability coefficient between assessment conducted with a task-specific rubric and assessment done with a general rubric? 2) Does peer assessment in which various types of rubrics are used show acceptable psychometric reliability? 3) How do pre-service teachers perceive their rating activities? Sixty-five pre-service teachers participated in this study. Each carried out the rater training program, wrote six types of English essays, rated his or her partner’s essays with the given rubrics, and responded to the questionnaire with items measuring whether subjects were self-confident regarding the accuracy of their ratings and how they perceived their participation in the ratings. The results show that the reliability values were higher when raters used a task-specific rubric than when they employed a general rubric. Facets analysis revealed that peer raters were marked as misfit raters. Finally, survey analysis showed that the difficulty level of essays affected subjects’ perceptions of the accuracy of their ratings.
Abstract
This study examines how different types of rubrics affect pre-service teachers’ assessment expertise in writing assessment. The current study has three research questions: 1) Is there a significant difference in the reliability coefficient between assessment conducted with a task-specific rubric and assessment done with a general rubric? 2) Does peer assessment in which various types of rubrics are used show acceptable psychometric reliability? 3) How do pre-service teachers perceive their rating activities? Sixty-five pre-service teachers participated in this study. Each carried out the rater training program, wrote six types of English essays, rated his or her partner’s essays with the given rubrics, and responded to the questionnaire with items measuring whether subjects were self-confident regarding the accuracy of their ratings and how they perceived their participation in the ratings. The results show that the reliability values were higher when raters used a task-specific rubric than when they employed a general rubric. Facets analysis revealed that peer raters were marked as misfit raters. Finally, survey analysis showed that the difficulty level of essays affected subjects’ perceptions of the accuracy of their ratings.
참고문헌 (37)
- 김신영, "교사의 학생평가전문성과 중등교사 양성과정", 교육평가연구, 20(1), pp.1~16 (2007), 한국교육평가학회
- 김연희, "쓰기 수행 평가의 채점 기준 변이성이 영작문 평가에 미치는 영향", 영어학, 13(3), pp.465~496 (2013), 한국영어학회[DOI]
- 박정민, "영어 쓰기 평가 기준에 대한 재고", 새한영어영문학, 52(3), pp.217~237 (2010), 새한영어영문학회[DOI]
- 신상근, "예비교사의 평가 전문성 신장을 위한 동료평가의 유용성에 관한 연구", 영어교육연구, 19(1), pp.183~8 (2007), 팬코리아영어교육학회[DOI]
- 신상근, "50년간 영어교육에 게재된 영어 평가 관련논문 분석", 영어교육, 70, pp.109~132 (2015), 한국영어교육학회[DOI]
- 안경자, "영어교육에 게재된 영어 교사교육 관련 연구 분석", 영어교육, 70, pp.161~192 (2015), 한국영어교육학회[DOI]
- 이영아, "중등학교 영어교사들의 교육과정에 대한 인식과 개선 요구", 교과교육학연구, 19(2), pp.571~596 (2015), 교과교육연구소[DOI]
- 이영주, "대학 영작문 수업에서 EBB 평가척도의 환류효과", 중등영어교육, 11(2), pp.99~123 (2018), 한국중등영어교육학회[DOI]
- 이지연, "예비영어교사의 평가 전문성 신장 방안: 자기성찰 기반 평가과업 개발 프로그램을 중심으로", 교사교육연구, 56(2), pp.107~123 (2017), 과학교육연구소[DOI]
- 이지연, "예비영어교사의 마이크로티칭에 대한다각적 평가 연구", Foreign Languages Education, 18(3), pp.209~233 (2011), 한국외국어교육학회
- 이지영, "사전 과업 계획 방식의 차이가 영어 말하기 수행능력에 미치는 영향 탐색—임용 목적의 면접 평가 연습 상황에서 협력적 방식을 중심으로", 영어영문학21, 29(2), pp.249~281 (2016), 21세기영어영문학회[DOI]
- 이호, "영작문에서의 대면평가와 익명평가의 비교 연구", 영어학, 9(3), pp.403~427 (2009), 한국영어학회[DOI]
- 이호, "영어논술평가에서의 일반 척도, 세부과업별 척도, 이분식 척도의 비교 연구", 영어학, 11(3), pp.601~626 (2011), 한국영어학회[DOI]
- 이희경, "중등 영어교사들의 교수자로서의 자질과 평가자로서의 자질에 관한 연구", 영어교육, 62(2), pp.309~330 (2007), 한국영어교육학회[DOI]
- 이희경, "영어 말하기 능력의 다집단 교실 평가 가능성에 대한 연구 - 자기, 동료, 교사평가를 중심으로 -", 영어교육연구, 21(4), pp.241~264 (2009), 팬코리아영어교육학회[DOI]
- 장소영, "언어교육평가 연구를 위한 FACETS 프로그램" (2009), 글로벌콘텐츠
- 조영아, "교생실습에 대한 예비 영어교사들의 인식 분석과 개선 방안 연구", 영어영문학21, 32(1), pp.163~182 (2019), 21세기영어영문학회[DOI]
- 주경숙, "중학교 2학년 영어 말하기 수행평가에서 평가 방법에 따른 역류효과의 비교연구 : 교사평가, 자가평가, 동료평가를 중심으로", 석사, 영어교육, pp.2003
- Bachman, L. F., "Language testing in practice" (1995), Oxford University Press
- Barkaoui, K, "Effects of marking method and rater experience on ESL essay scores and rater performance", Assessment in Education: Principles, Policy & Practice, 18, pp.279~293 (2011)
- Becker, A, "Examing rubrics used to measure writing performance in U. S. intensive English programs", The CATESOL Journal, 22(1), pp.113~130 (2011)
- Chapelle, C. A., "Interface between second language acquisition and language testing research", pp.32~70 (1998), Cambridge University Press
- Elder, C, "Exploring the limits of authenticity in LSP testing : The case of a specific-purpose language test for health professionals", Language Testing, 33(2), pp.147~152 (2015)
- Fleiss, J. L., "The design and analysis of clinical experiments" (1986), Wiley
- Fulcher, G., "Testing second language speaking" (2003), Pearson Education
- Galti, A. M., "Rating scale in writing assessment : Holistic vs. Analytical scales : A review", International Journal of English Research, 4(6), pp.4~6 (2018)
- Hamp-Lyons, L., "Assessing second language writing in academic contexts" (1991), Ablex Publishing Corporation
- 황성삼, "Discourse analysis of EFL interview test at secondary school : a focus on oral fluency", 박사, 영어교육, pp.2010
- Knoch, U, "Diagnostic assessment of writing : A comparison of two rating scales", Language Testing, 26(2), pp.275~304 (2009)
- 이호, "Validity of Self- and Peer-ratings in an EFL Essay-writing Test", 영어교육, 60(3), pp.195~219 (2005), 한국영어교육학회
- Matsuno, S, "Self-, peer-, and teacher-assessments in Japanese university EFL writing classrooms", Language Testing, 26(1), pp.75~100 (2009)
- McGriff, S. J., "The effect of an analytic trait, task-specific rubric with or without explanation on self-efficacy and academic acheivement", doctoral, pp.2007
- Moskal, B. M., "Scoring rubrics: What, when, and how?", Practical Assessment Research & Evaluation, 7(3), pp.1~5 (2000)
- Weigle, S. C., "Assessing writing" (2002), Cambridge University Press
- White, E. M., "Holisticism", College Composition and Communication, 35(4), pp.400~409 (1984)
- Wu, W. M., "Towards authenticity of task in test development", Language Testing, 18(2), pp.187~206 (2001)
- Yune, S. Y., "Holistic rubric vs. analytic rubric for measuring clinical performance levels in medical students", BMC Medical Education, 18, pp.1~6 (2018)
