An Application of Testlet-based Response Theory Model to a Reading Comprehension Section of a TOEIC Practice Test
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study applied Testlet-based response theory (TRT) model to a set of reading comprehension data from a TOEIC practice test. The parameter estimates and test information were compared to those of the standard item response theory (IRT) model. The results show that the model choice could have a serious effect on item slope and guessing parameter estimates. However, the two models did not yield any significant difference in item difficulty and proficiency parameter estimates. Test information as a measure of global precision shows that the two models could yield a differential degree of test precision depending on the range of ability distribution. A further textual analysis was also performed to see why certain testlets have more dependence than others in the same test
Abstract
This study applied Testlet-based response theory (TRT) model to a set of reading comprehension data from a TOEIC practice test. The parameter estimates and test information were compared to those of the standard item response theory (IRT) model. The results show that the model choice could have a serious effect on item slope and guessing parameter estimates. However, the two models did not yield any significant difference in item difficulty and proficiency parameter estimates. Test information as a measure of global precision shows that the two models could yield a differential degree of test precision depending on the range of ability distribution. A further textual analysis was also performed to see why certain testlets have more dependence than others in the same test
참고문헌 (30)
- Albert, J, "Bayesian Analysis of Binary and Polychotomous Response Data", Journals of American Statistical Association, 88, pp.669~679 (1993)
- Bradlow, E. T, "A Bayesian Random Effects Model for Testlets", Psychometrika, 64, pp.153~168 (1999)
- Choi, I, "An Investigation Into the Adequacy of Three IRT Models for Data From Two EFL Reading Tests", Language Testing, 9, pp.51~78 (1992)
- DeMars, C. E, "Application of the Bi-factor Multidimensional Item Response Theory Model to Testlet-based Tests", Journal of Educational Measurement, 43, pp.145~168 (2006)
- Gelfand, A. E, "Sampling-based Approaches to Calculating Marginal Density", Journal of the American Statistical Association, 85, pp.298~409 (1990)
- Gibbons, R. D, "Full-information Item Bi-factor Analysis", Psychometrika, 57, pp.423~436 (1992)
- Henning, G, "Advantages of Latent Trait Measurement in Language Testing", Language Testing, 1, pp.237~241 (1984)
- Henning, G, "Item Banking via DBASE II: The UCLA ESL Proficiency Examination Experience in and Language Testing", pp.69~77 (1986), TESOL
- Henning, G, "Scalar Analysis of Comparison Ratings in Language Testing Research: Selected Papers From the 1986 Colloquium", pp.24~38 (1987), Defense Language Institute
- Hicks, M, "A Comparative Study of Methods of Equating TOEFL Test Scores" (1984), Educational Testing Service
- Madsen, H. S, "Computerized Rasch Analysis of Item Bias in ESL Tests in Technology and Language Testing", pp.47~67 (1986), Teachers of English to Speakers of Other Languages
- McLeod, L. D, "Factor Analysis for Items Scored in Two Categories in Test Scoring", pp.189~216 (2001), Erlbaum
- McNamara, T. F, "Item Response Theory and the Validation of an ESP Test for Health Professionals", Language Testing, 7, pp.52~76 (1990)
- Pollit, A, "Calibrating Graded Assessments: Rasch Partial Credit Analysis of Performance in Writing", Language Testing, 4, pp.71~92 (1987)
- Saida, C, "Post-hoc IRT Equating of Previously Administered English Tests for Comparison of Test Scores", Language Testing, 25, pp.187~210 (2008)
- Sheehan, K, "Understanding Performance on the TOEFL Reading Comprehension Section: A Tree-based Regression Approach", A paper presented at the 1998 Annual Conference of the American Association of Applied Linguistics (1998)
- Sireci, S. G, "On the Reliability of Testlet-Based Tests", Journal of Educational Measurement, 28, pp.237~247 (1991)
- Szabo, G, "Applying Item Response Theory in Language Test Item Bank Building" (2007), Peter Lang Publishing
- Thissen, D, "Trace Lines for Testlets: A Use of Multiple-Categorical Response Models", Journal of Educational Measurement, 26, pp.247~260 (1989)
- Wainer, H, "Precision and Differential Item Functioning on a Testlet-based Test: The 1991 Law School Admissions Test as an Example", Applied Measurement in Education, 8, pp.157~187 (1995)
- Wainer, H, "Testlet Response Theory: An Analog for the 3PL Model Useful in Testlet-based Adaptive Testing in Computerized Adaptive Testing: Theory and Practice", pp.245~269 (2000), Kluwer
- Wainer, H, "Item Clusters and Computerized Adaptive Testing: A Case for Testlets", Journal of Educational Measurement, 24, pp.185~201 (1987)
- Wainer, H, "How is Reliability Related to the Quality of Test Scores? What is the Effect of Local Dependence on Reliability?", Educational Measurement: Issues and Practices, 15, pp.22~29 (1996)
- Using a New Statistical Model for Testlets to Score TOEFL, "Wainer, H", Educational Testing Service, ~, pp.ETS RR-01-09 (2001)
- Wang, X, (2005), Educational Testing Service
- Wang, W, "Exploring Local Item Dependence Using a Random-Effects Facet Model", Applied Psychological Measurement, 29, pp.296~318 (2005)
- Yen, W, "Scaling Performance Assessments: Strategies for Managing Local Item Dependence", Journal of Educational Measurement, 30, pp.187~213 (1993)
- Zenisky, A. L, "Identification and Evaluation of Local Item Dependencies in the Medical College Admissions Test", Journal of Educational Measurement, 39, pp.291~309 (2002)
- Zhang, B, "Assessing the Accuracy and Consistency of Language Proficiency Classification Under Comparing Measurement Models", Language Testing, 27, pp.119~140 (2010)
- Zimowski, M, "BILOG_MG: Multiple-group IRT Analysis and Test Maintenance for Binary Items" (1996), Scientific Software International