영어 시험 오답 반응 통계 검정을 통한 구인 타당성과 수준별 영어 능력 연구
A study on construct validity and level- specific English ability through statistical testing of incorrect responses to an English test
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The purpose of this study is to statistically analyze incorrect answer choices from an English grammar test taken by a large number of test-takers (687), aiming to examine the implications for the test validity and constructs. The study also explored differences between proficiency groups, particularly in their selection of notable distractors. It was found that the constructs of 16 questions required redefinition due to significant attractors. Differences between groups, as detected by Chi-square and two-proportion Z-tests, revealed key distinctions and commonalities in their mastery of various English grammar points. These findings underscore the importance of analyzing learner responses after test administration and confirm that expert-defined constructs may differ from those derived from student responses. This research hopes to provide an actionable example of analyzing test data for many test practitioners in the field of language education.
Abstract
The purpose of this study is to statistically analyze incorrect answer choices from an English grammar test taken by a large number of test-takers (687), aiming to examine the implications for the test validity and constructs. The study also explored differences between proficiency groups, particularly in their selection of notable distractors. It was found that the constructs of 16 questions required redefinition due to significant attractors. Differences between groups, as detected by Chi-square and two-proportion Z-tests, revealed key distinctions and commonalities in their mastery of various English grammar points. These findings underscore the importance of analyzing learner responses after test administration and confirm that expert-defined constructs may differ from those derived from student responses. This research hopes to provide an actionable example of analyzing test data for many test practitioners in the field of language education.
참고문헌 (19)
- 김경석, "수학능력시험 문항 분석: 선택지 매력도를 중심으로", 영어교육연구, 16(3), pp.181~205 (2004), 팬코리아영어교육학회
- 김해동, "표준화 시험 평가 문항의 변별도 변인 분석: 듣기, 문법, 어휘 평가 문항", 외국어교육연구, 23(1), pp.57~79 (2009), 외국어교육연구소[DOI]
- 이연숙, "대학생 영어 글쓰기 오류 분석 결과 활용을 위한 선행 연구의 비판적 고찰", 언어과학연구, (104), pp.403~429 (2023), 언어과학회
- 이완기, "영어평가방법론" (2012), 북플랜트
- 이현우, "절대평가 시험으로서의 수능 영어시험 분석: 주요 난이도 예측 변인을 중심으로", 중등영어교육, 14(4), pp.90~124 (2021), 한국중등영어교육학회[DOI]
- 장소영, "영어듣기시험 문항 타당도에 영향을 주는 요인 분석", Studies in English Education, 23(2), pp.619~649 (2018), 글로벌영어교육학회[DOI]
- Ahmad, K. A., "Distractor efficiency of an intermediate English proficiency course final examination paper", Asian Journal of Assessment in Teaching and Learning, 11(2), pp.103~112 (2021)
- Bakytbekovich, O., "Distractor analysis in multiple-choice items using the Rasch model", International Journal of Language Testing, 13, pp.69~78 (2023)
- Choi, I. K., "A survey of methodological approaches employed to validate language assessments: 1999-2009", Paper presented at the 33rd annual conference of the Language Testing Research Colloquium (2011)
- Ebel, R. L., "Essentials of educational measurement" (1986), Prentice Hall
- Fulcher, G., "Language testing and assessment" (2007), Routledge
- Gan, L., "A review on language assessment literacy : Trends, foci and contributions", Language Assessment Quarterly, 19(5), pp.503~525 (2022)
- Hartati, N., "Item analysis for a better quality test", English Language in Focus(ELIF), 2(1), pp.59~70 (2019)
- Hughes, A., "Testing for language teachers" (2002), Cambridge
- Iimura, H., "Distractor plausibility in a multiple-choice listening test", JLTA Journal, 21, pp.65~81 (2018)
- Jamalzadeh, M., "Assessing the validity of an IAU General English Achievement Test through hybridizing differential item functioning and differential distractor functioning", Language Testing in Asia, 11(8), pp.1~17 (2021)[DOI]
- Rejeki, S., "Discrimination index, difficulty index, and distractor efficiency in MCQs English for academic purposes midterm test", Journal of English Language and Pedagogy, 6(1), pp.1~11 (2023)[DOI]
- Terao, T., "A comparison of distractor selection among proficiency levels in reading tests : A focus on summarization processes in Japanese EFL learners", SAGE Open, 10(1), pp.1~14 (2020)
- Yi, Y. -S., "Another reason for CDA in language assessment : A critical synthesis from a perspective of validating test constructs", Language Teaching Research Quarterly, 37, pp.248~265 (2023)[DOI]