An Application of the Rasch Model to Assessing an English Achievement Test in a High School Setting
0
조회
0
다운로드
0
인용
0
참고문헌
초록
This study investigated the acceptability of reading test items developed by non-native Korean teachers of English in a Korean high school by the application of the Rasch model analysis. Data on 17 multiple-choice items of reading achievement for 256 high school students at a high school were collected. The data were submitted to Winstep, a Rasch analysis software, to study general model fit, item difficulty measures, person ability measures, fit indices, reliability, and separation indices. The results of the first Rasch analysis of 17 items indicated 5 items did not fit the Rasch model. The second analysis of the 12 items, after the deletion of 5 misfit items, revealed that the set of test items did not match individual persons’ ability or were poorly targeted with high-and low- ability persons. The content analysis of those 5 misfit items confirmed that the teachers had made serious mistakes in construction. Differential item functioning showed that only 1 item of the 12 test items was biased against gender. The results of the study suggested that Korean teachers of English need more thorough preparation and a higher sense of responsibility for constructing items and assigning grades to their students.
Abstract
This study investigated the acceptability of reading test items developed by non-native Korean teachers of English in a Korean high school by the application of the Rasch model analysis. Data on 17 multiple-choice items of reading achievement for 256 high school students at a high school were collected. The data were submitted to Winstep, a Rasch analysis software, to study general model fit, item difficulty measures, person ability measures, fit indices, reliability, and separation indices. The results of the first Rasch analysis of 17 items indicated 5 items did not fit the Rasch model. The second analysis of the 12 items, after the deletion of 5 misfit items, revealed that the set of test items did not match individual persons’ ability or were poorly targeted with high-and low- ability persons. The content analysis of those 5 misfit items confirmed that the teachers had made serious mistakes in construction. Differential item functioning showed that only 1 item of the 12 test items was biased against gender. The results of the study suggested that Korean teachers of English need more thorough preparation and a higher sense of responsibility for constructing items and assigning grades to their students.
참고문헌 (13)
- Bachman, L. F., "Language testing in practice" (1996), Oxford University Press
- Bond, T., "Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.)" (2007), Lawrence Erlbaum
- Brown, D., "Language assessment: principles and classroom practices" (2004), Person Education
- Brown, A., "Interviewer style and candidate performance in the IELTS oral interview. in : Studies in language testing (19) IELTS collected papers", pp.37~60 (2007), Cambridge University Press
- Grabe, W., "Reading research and its implications for reading assessment. in : Studies in language testing (9): Fairness and validation in language assessment", pp.226~262 (2000), Cambridge University Press
- Hillocks, G., "A taxonomy of skills in reading and interpreting fiction", American Educational Research Journal, 21(1), pp.7~24 (1984)
- Hughes, A., "Testing for language teachers" (2003), Cambridge University Press
- Kulikowich, J., "Evaluating fifth-and sixth-grade students’expository writing: Task development, scoring, and psychometric issues", Reading and Writing, 21, pp.153~175 (2008)
- Lee, C., "Signifying as a scaffold for literary interpretation", Journal of Black Psychology, 21(4), pp.357~381 (1995)
- Merryless, B., "A survey of examiner attitudes and behavior in the IELTS oral interview. in : Studies in language testing (19) IELTS collected papers", pp.142~178 (2007), Cambridge University Press
- McNamara, T., "Measuring second language performance" (1996), Addison Wesley Longman
- Moore, T., "Authenticity in the IELTS academic module writing test: A comparative study of task 2 items and university assignments. in : Studies in language testing (19) IELTS collected papers", pp.197~248 (2007), Cambridge University Press
- Waugh, R., "Approaches to studying for students in higher education: A Rasch measurement model analysis", British Journal of Educational Psychology, 69, pp.63~79 (1999)