Examining Measurement Characteristics of an English Listening Comprehension Test
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
The purpose of this study was to investigate some measurement characteristics of an English listening test for high school students annually developed and administered by the 15 provincial Offices of Education in Korea. The study applied item response theory (IRT) models and a series of exploratory and confirmatory factor analysis (EFA, CFA) procedures to a data set from 284 students at a local high school. The IRT analyses showed the test could not distinguish students properly along the continuum of listening proficiency because the test items were too easy and the test was too short with a relatively low reliability for such a large scale test. Subsequent EFA and CFA procedures revealed that the test was primarily unidimensional and should function as a summative test but not as a diagnostic one. Further implications from the analyses were discussed together with some suggestions to improve the test quality.
Abstract
The purpose of this study was to investigate some measurement characteristics of an English listening test for high school students annually developed and administered by the 15 provincial Offices of Education in Korea. The study applied item response theory (IRT) models and a series of exploratory and confirmatory factor analysis (EFA, CFA) procedures to a data set from 284 students at a local high school. The IRT analyses showed the test could not distinguish students properly along the continuum of listening proficiency because the test items were too easy and the test was too short with a relatively low reliability for such a large scale test. Subsequent EFA and CFA procedures revealed that the test was primarily unidimensional and should function as a summative test but not as a diagnostic one. Further implications from the analyses were discussed together with some suggestions to improve the test quality.
참고문헌 (31)
- Akaike, H., "A new look at the statistical model identification", IEEE Transactions on Automatic Control, 19(6), pp.716~723 (1974)
- Alderson, J. C., "Diagnosing foreign language proficiency: The interface between learning and assessment" (2005), Continuum International Publishing
- Alderson, J. C., "Assessing reading" (2000), Cambridge University Press
- Alderson, J. C., "Testing reading comprehension skills (Part One)", Reading in a Foreign Language, 6, pp.425~438 (1990)
- Bock, R. D., "Full-information factor analysis", Applied Psychological Measurement, 12, pp.261~280 (1988)
- Buck, G., "Assessing listening" (2001), Cambridge University Press
- Buja, A., "Remarks on parallel analysis", Multi. Behav. Res, 27, pp.509~540 (1992)
- Cai, L., "Limited information goodness of fit testing of item response theory models for sparse 2p tables", British Journal of Mathematical and Statistical Psychology, 59, pp.173~194 (2006)
- Fulcher, G., "Language Testing and Assessment" (2008), Routledge
- Gibbons, R. D., "Full-information item bi-factor analysis", Psychometrika, 57, pp.423~436 (1992)
- Jang, H-J., "An analysis of the English listening proficiency test items in 2007-2008 school years: Co-hosted by 15 Boards of Education", 석사, pp.2006
- Jeon, H-S., "An analysis of English listening test items at Middle School level", 석사, pp.2006
- Lumley, T. J. N., "Reading comprehension sub-skills: teachers’ perceptions of content in an EAP test", Melbourne Papers in Applied Linguistics, 2, pp.25~55 (1993)
- Kamata, A., "Multilevel Modeling of Educational Data", pp.345~388 (2008), Information Age Publishing
- Kim, C-H., "The analysis of English listening test at the middle school level", 석사, pp.2005
- Kim, M., "An analysis of listening items in Korean SAT of the 7th national curriculum", 석사, pp.2005
- Lee, K-H., "An analysis of the middle school English listening proficiency test items in 2007-2008 school years: co-hosted by the 15 Boards of Education", 석사, pp.2009
- Maydeu-Olivares, A., "Limited and full information estimation and testing in 2n contingency tables: A unified framework", Journal of the American Statistical Association, 100, pp.1009~1020 (2005)
- Maydeu-Olivares, A., "Limited information goodness-of-fit testing in multidimensional contingency tables", Psychometrika, 71, pp.713~732 (2006)
- Park, Y-M., "The analysis of listening section in the Korean scholastic ability test", 석사, pp.2007
- Park, B-J., "The analysis of English listening comprehension tests", 석사, pp.2008
- Peterson, P. W., "Teaching English as a second language (2nd Ed.)" (1991), Heinle & Heinle Publishers
- Reckase, M. D., "The difficulty of test items that measure more than one ability", Applied Psychological Measurement, 9, pp.401~412 (1985)
- Reckase, M. D., "Handbook of Modern Item Response Theory", pp.271~286 (1997), Springer
- Reckase, M. D., "The discriminating power of items that measure more than one dimension", Applied Psychological Measurement, 15, pp.361~373 (1991)
- Schwarz, G. E., "Estimating the dimension of a model", Annals of Statistics, 6(2), pp.461~464 (1978)
- Song, M-Y., "Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach", Language Testing, 25, pp.435~464 (2008)
- Swygert, K. A., "Test scoring", pp.217~249 (2001), Lawrence Erlbaum
- Takane, Y., "On the relationship between item response theory and factor analysis of discretized variables", Psychometrika, 52, pp.393~408 (1987)
- Woo, S-Y., "An analysis of English listening comprehension tests in Middle School", 석사, pp.2005
- Eom, J-S., "An analysis of question items in an English listening test", 석사, pp.2010