Item Difficulty Predictors of a Multiple-choice Reading Test
0
조회
0
다운로드
13
인용
0
참고문헌
키워드
초록
The researchers’ experiential knowledge demonstrates that the task of predicting and controlling the difficulty level of the multiple-choice items of the College Scholastic Ability Test (CSAT) for English is substantially left to the subjective judgment of experienced item writers. The present study accordingly recognizes a need to identify item difficulty predictors and build an item difficulty prediction model to handle this pertinent issue. While taking separate interest in constructing a model for the multiple-choice reading subset of the CSAT, the study was conducted by identifying item difficulty predictor variables from previous research, and by validating the candidate predictors via questionnaires by highly experienced teacher-raters when asked to analyze reading items from the English reading subset of the preliminary CSAT (i.e., yun-hap-hak-ryuk-pyung-ga) administered in March 2009. Using multiple regression technique and maximum likelihood estimation, an item difficulty prediction model was generated. In order to check validity and applicability of the prediction model, the hypothetical model was finally tested on a subsequent version of the test administered in September 2009. This type of model building is expected to guide test developers design an item pool in accordance with special needs, such as to construct multiple test forms, which have similar mean difficulties.
Abstract
The researchers’ experiential knowledge demonstrates that the task of predicting and controlling the difficulty level of the multiple-choice items of the College Scholastic Ability Test (CSAT) for English is substantially left to the subjective judgment of experienced item writers. The present study accordingly recognizes a need to identify item difficulty predictors and build an item difficulty prediction model to handle this pertinent issue. While taking separate interest in constructing a model for the multiple-choice reading subset of the CSAT, the study was conducted by identifying item difficulty predictor variables from previous research, and by validating the candidate predictors via questionnaires by highly experienced teacher-raters when asked to analyze reading items from the English reading subset of the preliminary CSAT (i.e., yun-hap-hak-ryuk-pyung-ga) administered in March 2009. Using multiple regression technique and maximum likelihood estimation, an item difficulty prediction model was generated. In order to check validity and applicability of the prediction model, the hypothetical model was finally tested on a subsequent version of the test administered in September 2009. This type of model building is expected to guide test developers design an item pool in accordance with special needs, such as to construct multiple test forms, which have similar mean difficulties.
참고문헌 (33)
- Abrahamsen, E. P, "Reading comprehension in adolescents with learning disabilities: Semantic and syntactic effects", Journal of Learning Disabilities, 22(10), pp.596~572 (1989)
- 장경숙, "대학수학능력시험 외국어(영어)영역 읽기 난이도 예측 모형 개발", Foreign Languages Education, 11(1), pp.111~130 (2004), 한국외국어교육학회
- Choi, I.-C, "The impact of EFL testing on EFL education in Korea", Language Testing, 25(10), pp.39~62 (2008)
- Davey, B, "Factors affecting the difficulty of reading comprehension items for successful and unsuccessful readers", Journal of Experimental Education, 56(2), pp.67~76 (1988)
- Drum, P. A, "The effects of sentence structure variables on performance in reading comprehension tests", Reading Research Quarterly, 16, pp.486~514 (1981)
- Embretson, S. E, "Component latent trait models for paragraph comprehension tests", Applied Psychological Measurement, 11(2), pp.175~193 (1987)
- Freedle, R, "Predictors of good and bad essays", Paper presented at the annual Georgetown University Roundtable on languages and linguistics (1981)
- J 969-60), "The prediction of SAT reading comprehension item difficulty for expository prose passages (P", pp.1991, Freedle, R
- Freedle, R, "The prediction of GRE reading comprehension item difficulty for expository prose passages (ETS Research Report RR-91-59)" (1992), Educational Testing Service
- Freedle, R, "The prediction of TOEFL reading item difficulty:implications for construct validity", Language Testing, 10(2), pp.133~167 (1993)
- Freedle, R, "The prediction of TOEFL listening comprehension item difficulty for minitalk passages: Implications for construct validity (ETS-RR-96-29)" (1996), Educational Testing Service
- Graves, M. F, "Vocabulary learning and instruction", Review of Research in Education, 13, pp.49~89 (1986)
- Hare, V, "Text effects on main idea comprehension", Reading Research Quarterly, 24, pp.72~88 (1989)
- Hites, R. W, "The relation of readability and format to retention in communication", 박사, pp.1950
- Hudson, T, "Teaching second language reading" (2007), Oxford University Press
- Jin, Kyung-Ae, "The Prediction of English Item Difficulty in College Scholatic Ability Test", 영어교육, 59(1), pp.267~278 (2004), 한국영어교육학회
- Kieras, D. E, "Thematic processes in prose. In Understanding expository text", pp.11~64 (1985), Erlbaum
- "EBS Suneung yunjae chool je. Chosun IlBo" (2010)[URL]
- Klare, G. R, "Assessing readability", Reading Research Quarterly, 1, pp.1~47 (1975)
- Klare, G. R, "Readability. (Signed chapter.) In Handbook of Reading Research", pp.681~744 (1984), Longman
- Kostin, I, "Exploring item characteristics that are related to the difficulty of TOEFL dialogue items (ETS-RR-4-11)" (2004), Educational Testing Service
- Jeong-Won Lee, "An Exploratory Study on Reading Comprehension Test-Taking Processes and Strategies in the EFL Context", 영어교육, 57(4), pp.177~195 (2002), 한국영어교육학회
- Leow, R, "To simplify or not to simplify: A look at intake", Studies in Second Language Acquisition, 15(3), pp.333~356 (1993)
- Mehrpour, S, "The impact of text length on EFL students’ reading comprehension", Asian EFL Journal, 6(3) (2004)
- Nation, I. S. P, "Learning vocabulary in another language" (2001), Cambridge University Press
- Nissan, S, "An analysis of factors affecting the difficulty of dialogue items in TOEFL listening comprehension (ETS-RR-95-37)" (1996), Educational Testing Service
- Ozuru, Y, "Where's the difficulty in standardized reading tests: The passage or the question?", Behavior Research Methods, 40(4), pp.1001~1015 (2008)
- Paivio, A, "Mental representations" (1986), Oxford University Press
- Perkins, K, "Predicting item difficulty in a reading comprehension test with an artificial neural network", Language Testing, 12(1), pp.34~53 (1995)
- Qian, D. D, "Assessing the roles of depth and breadth of vocabulary knowledge in reading comprehension", The Canadian Modern Language Review, 56(2), pp.282~308 (1999)
- Qian, D. D, "Investigating the relationship between vocabulary knowledge and academic reading performance: An assessment perspective", Language Learning, 52(3), pp.513~536 (2002)
- Read, J, "Assessing vocabulary" (2000), Cambridge University Press
- Yano, Y, "The effect of simplified and elaborated texts on foreign language reading comprehension", Language Learning, 44(2), pp.189~219 (1994)