대학수학능력시험 영어영역 읽기 문항 난이도 요인 탐색: EBS 연계정책 변화를 중심으로
Investigating factors affecting the item difficulty of the reading section in the CSAT English: Focusing on recent EBS-CSAT linkage policy changes
0
조회
0
다운로드
8
인용
0
참고문헌
키워드
초록
This study investigates factors that affect the difficulty levels of CSAT English reading items. For this, 56 English reading items from two recent CSAT tests, those administered in 2021 and 2022, were carefully examined to identify what elements of the reading stimuli had significant effects on the item difficulty. For data analysis, correct answer rates for each item were used to examine the extent to which different item-related variables, both quantitative and qualitative, might be associated with the difficulty of the items. The findings show that the reading items from the 2022 test were significantly more difficult than those from the 2021 test. When specific elements of the reading items of both tests were examined, it was found that the reduced number of EBS-linkage items was one of the main causes of the drop in the correct answer rates on the 2022 test. Furthermore, multiple regression models show that higher Gunning-Fog-Index readability level and logical connectives use ratio in the reading stimuli are predicted to increase the item difficulty. The present paper concludes by presenting a model that predicts the difficulty levels of CSAT reading items, which consist of a number of different variables.
Abstract
This study investigates factors that affect the difficulty levels of CSAT English reading items. For this, 56 English reading items from two recent CSAT tests, those administered in 2021 and 2022, were carefully examined to identify what elements of the reading stimuli had significant effects on the item difficulty. For data analysis, correct answer rates for each item were used to examine the extent to which different item-related variables, both quantitative and qualitative, might be associated with the difficulty of the items. The findings show that the reading items from the 2022 test were significantly more difficult than those from the 2021 test. When specific elements of the reading items of both tests were examined, it was found that the reduced number of EBS-linkage items was one of the main causes of the drop in the correct answer rates on the 2022 test. Furthermore, multiple regression models show that higher Gunning-Fog-Index readability level and logical connectives use ratio in the reading stimuli are predicted to increase the item difficulty. The present paper concludes by presenting a model that predicts the difficulty levels of CSAT reading items, which consist of a number of different variables.
참고문헌 (33)
- 고나은, "Coh-Metrix를 이용한 수능 영어 읽기 영역 지문 난이도 비교: EBS-수능 연계 정책 전후", 중등영어교육, 10(4), pp.3~24 (2017), 한국중등영어교육학회
- 권서경, "EBS 연계정책에 따른 대학수학능력시험 영어영역 어휘 난이도 변화 분석", 영어교과교육, 13(4), pp.97~121 (2014), 한국영어교과교육학회[DOI]
- 김재은, "고등학교 영어 교과서, EBS 수능 연계 교재, 대학수학능력시험의 코퍼스기반 난이도 비교 분석", 멀티미디어 언어교육, 18(1), pp.59~92 (2015), 한국멀티미디어언어교육학회[DOI]
- 김제우, "코퍼스기반 고등학교 어휘 사용 연구: 영어 I·II 교과서, 대학수학능력시험, EBS 교재 읽기 지문을 중심으로", 언어과학연구, (80), pp.51~74 (2017), 언어과학회[DOI]
- "영어 고교 교과서 난도, 미국 고등학교 수준" (2014)[URL]
- 대학수학능력시험 10년사 I, "남명호", 한국교육과정평가원, ~, pp.연구보고 ORM 2005-32-2 (2005)
- "학력평가풀서비스"[URL]
- "출제본부 “영어, 다양한 소재·사용빈도 높은 어휘 출제”" (2021)[URL]
- "2015 개정 영어과 교육과정 기본어휘 분석 프로그램" (2015), 신동광[URL]
- 이종승, "대학수학능력시험 문항난이도 추정모형 개발 - 언어 영역, 수리 영역, 영어 영역을 중심으로 -", 교육평가연구, 16(2), pp.1~23 (2003), 한국교육평가학회
- 장경숙, "대학수학능력시험 외국어(영어)영역 읽기 난이도 예측 모형 개발", Foreign Languages Education, 11(1), pp.111~130 (2004), 한국외국어교육학회
- 장지연, "절대평가 도입 전후 수능 영어 독해 지문의 통사적 복잡성 비교", 영어교과교육, 18(2), pp.161~188 (2019), 한국영어교과교육학회[DOI]
- 정요한, "절대평가 도입 후 수능 영어 듣기 및 읽기 지문의 난이도 비교: 통사적 복잡성, 소재, 이독성을 중심으로", 멀티미디어 언어교육, 23(3), pp.206~232 (2020), 한국멀티미디어언어교육학회[DOI]
- 문제은행식 출제 체제 구축을 위한 난이도 예측 방안 연구: 수리와 외국어(영어) 영역을 중심으로, "조윤동", 한국교육과정평가원, ~, pp.연구보고 CAT 2009-18 (2009)
- 최민주, "수능 영어 문항 유형간 응집력과 어휘정보 분석", 한국콘텐츠학회 논문지, 17(12), pp.378~385 (2017), 한국콘텐츠학회[DOI]
- 황이수, "수학능력시험 영어 지문의 텍스트 요인과 문항 난이도의 상관관계 분석: 통사적 복잡성을 중심으로", 영어영문학연구, 46(1), pp.265~283 (2020), 대한영어영문학회[DOI]
- 황이수, "수학능력시험 영어 영역 읽기 지문의 이독성 및 어휘 정보와 문항 난이도 간의 상관관계", 인문사회 21, 11(2), pp.389~400 (2020), 인문사회 21
- "Text Inspector: Online text analysis tool" (2012)[URL]
- "Web Vocabprofile: An adaptation of Heatley, Nation, & Coxhead's (2002) Range"[URL]
- Cohen, J., "Statistical power analysis for the behavioral sciences" (1988), Routledge
- Coxhead, A., "The development and evaluation of an academic word list", Master, pp.1998
- Coxhead, A., "A new academic word list", TESOL Quarterly, 34(2), pp.213~238 (2000)[DOI]
- Department of Defense, "Military specification manual, technical: General style and format requirement (DOD MIL-M-38784B)", Navy Publishing and Printing Service Office, ~ (1983)
- Freedle, R., "The prediction of SAT reading comprehension item difficulty for expository prose passages(P", J969-60), 1991
- The prediction of TOEFL reading comprehension item difficulty for expository prose passages for three item types: Main idea, inference, and supporting idea items, "Freedle, R.", Educational Testing Service, ~, pp.ETS-RR-93-13 (1993)
- Gunning, R., "The technique of clear writing" (1952), McGraw Hill
- Hyland, K., "Metadiscourse in academic writing : A reappraisal", Applied Linguistics, 25(2), pp.156~177 (2004)[DOI]
- Jin, K., "The prediction of English item difficulty in college scholastic ability test", English Teaching, 59(1), pp.267~278 (2004)
- 김차현, "A Corpus-Based Comparative Analysis of CSAT English from 2015 to 2020 on the Basis of Criterion-Referenced Assessment", 멀티미디어 언어교육, 23(2), pp.98~120 (2020), 한국멀티미디어언어교육학회[DOI]
- McDonald, P. J., "Prose and cons of scholarly articles : How readability tests expose poor knowledge mobilization in academic publications", Journal of Professional Communication, 6(2), pp.87~119 (2020)
- "Coh-Metrix 3.0" (2017)[URL]
- Scheuneman, J. D., "Using differential item functioning procedures to explore sources of item difficulty and group performance characteristics", Journal of Educational Measurement, 21, pp.109~131 (1990)
- West, M., "A general service list of English words" (1953), Longman