An Item Analysis of the English Mock CSAT
0
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
Because the College Scholastic Ability Test (CSAT) is such a high-stakes national examination, it influences not only high school students, parents, and educators, but also English teaching and learning in Korean EFL classrooms in high schools. Many studies analyzed the English-language section of the CSAT. In the same vein, regularly conducted mock CSATs are also valuable data points for test-takers and test-developers. However, little research examined whether Mock CSATs measure what they are supposed to measure. This study aims to report the item analysis results for the English section of the Mock CSAT administered by the Seoul Metropolitan Office of Education in March 2018. It consists of listening and reading sections with five option multiple-choice items. Two hundred forty-nine Korean first grade high school students participated in this study. Their responses to each test item were reported as the result of item analysis with respect to three indices: item facility, item discrimination and distractor analysis. The results identified that the average of item facility was 64.8%, which means that the difficulty level was suitable for the test-takers. Item discrimination was 0.42, which indicates the test had a high discrimination power. Therefore, this study suggests that providing more plausible distractors enhance the overall test validity and reliability for future CSAT administrations.
Abstract
Because the College Scholastic Ability Test (CSAT) is such a high-stakes national examination, it influences not only high school students, parents, and educators, but also English teaching and learning in Korean EFL classrooms in high schools. Many studies analyzed the English-language section of the CSAT. In the same vein, regularly conducted mock CSATs are also valuable data points for test-takers and test-developers. However, little research examined whether Mock CSATs measure what they are supposed to measure. This study aims to report the item analysis results for the English section of the Mock CSAT administered by the Seoul Metropolitan Office of Education in March 2018. It consists of listening and reading sections with five option multiple-choice items. Two hundred forty-nine Korean first grade high school students participated in this study. Their responses to each test item were reported as the result of item analysis with respect to three indices: item facility, item discrimination and distractor analysis. The results identified that the average of item facility was 64.8%, which means that the difficulty level was suitable for the test-takers. Item discrimination was 0.42, which indicates the test had a high discrimination power. Therefore, this study suggests that providing more plausible distractors enhance the overall test validity and reliability for future CSAT administrations.
참고문헌 (26)
- American Psychological Association, "Standards for educational and psychological testing" (1985), American Psychological Association
- Bachman, L., "Fundamental considerations in language testing" (1990), Oxford University Press
- Brown, H. D., "Language assessment: Principles and classroom practices" (2004), Longman
- Cha, K. W., "English listening items on the college scholastic ability test of Korean", English Teaching, 52(2), pp.265~288 (1997)
- Chae, J. S., "An item analysis of an intramural regular high school English test", English Language Assessment, 5, pp.66~93 (2011)
- Choi, I. C., "An application of item response theory to language testing" (1992), Peter Lang
- 전유아, "Lexical Threshold of L2 Reading in the Korean CSAT", 영미연구, 31, pp.341~376 (2014), 영미연구소
- A study on the development of the 2017 national assessment of educational achievement(NAEA), "Do, H. G.", KICE, ~ (2017)
- Ebel, R. L., "Essentials of educational measurement" (1991), Prentice Hall
- Finch, A. E., "An Analysis of EBS Preparation Materials for the CSAT, English Section", 영어교육, 59(3), pp.307~332 (2004), 한국영어교육학회
- 고광윤, "한국 대입영어시험 어휘의 코퍼스 분석", 영어학연구, 16(2), pp.1~26 (2010), 한국영어학학회[DOI]
- Hughes, A., "Testing for language teachers" (2003), Cambridge University Press
- Jin, H. J., "Effective English reading comprehension strategies through an analysis of the reading section of CSAT", Master’s, pp.2009
- "College scholastic ability test" (2012)[URL]
- Kim, J. I., "Analysis of test and item characteristics of the English section of 2000 college scholastic ability test", Foreign Language Education, 8(2), pp.1~22 (2001)
- Kim, M., "An analysis of listening items in Korean SAT of the 7th national curriculum", Master’s, pp.2005
- 김용명, "수준별 영어 시험 검사지 구성을 위한 외국어 영역 문항 유형의 외적 타당성 분석", 현대영어교육, 13(4), pp.239~270 (2012), 현대영어교육학회
- Kwon, S. K., "Educational assessment in the republic of Korea: Lights and shadows of high-stake exam-based education system", Assessment in Education: Principles, Policy and Practice, 24(1), pp.60~77 (2017)
- Lee, D. H., "A study of validation on a modular-type for test construction in the college scholastic ability test", International Journal of Software Engineering and Its Applications, 10(12), pp.131~138 (2016)
- Lee, D. H, "A validation analysis on the modular-type of test construction", Advanced Science and Technology Letters, 127, pp.183~187 (2016)
- McNamara, T. F., "Language testing" (2000), Oxford University Press
- Shin, I. S., "Test items in the speaking section of college scholastic abilities test from the viewpoint of pragmatic competence : Analysis and suggestions", Foreign Languages Education, 10(3), pp.219~235 (2003)
- Song, H. S., "The effect of the college scholastic ability test(CSAT)on Korean college students’ reading processes and their perceptions of reading in English", English Teaching, 53(4), pp.265~290 (1998)
- Takanori, S., "Test-taker perception of what test items measure : A potential impact of face validity on student learning", Language Testing in Asia; Heidelberg, 5(1), pp.1~16 (2015)
- Yim, B. G., "An analysis of the listening comprehension items in the scholastic ability English test", Master’s, pp.2004
- Weir, Cyril J., "Language testing and validation: An evidence-based approach" (2005), Palgrave-Macmillan