본문 바로가기
KERD
논문KCI 등재

An Analysis of an English Reading Comprehension Test Through Explanatory Item Response Models

1

조회

0

다운로드

0

인용

0

참고문헌

키워드

주제분류언어학

초록

Extending the conventional item response theory(IRT) as a measurement model to explanatory IRT(EIRT) based on generalized (non)linear mixed modeling has opened a new possibility to incorporate many item and person properties in its modeling process so that their effects can be considered simultaneously. The purpose of the current study was to apply EIRT to a complex data set that consisted of a reading comprehension test, a student survey of English study, and information about the item categories of the reading test. Sequential EIRT modeling of the data set showed that some of the item and person properties consistently had significant effects on the item difficulty and on the probability of correct answers. The modeling process also revealed some statistical or computational challenges researchers might encounter when they try to apply EIRT to complex language test data.

Abstract

Extending the conventional item response theory(IRT) as a measurement model to explanatory IRT(EIRT) based on generalized (non)linear mixed modeling has opened a new possibility to incorporate many item and person properties in its modeling process so that their effects can be considered simultaneously. The purpose of the current study was to apply EIRT to a complex data set that consisted of a reading comprehension test, a student survey of English study, and information about the item categories of the reading test. Sequential EIRT modeling of the data set showed that some of the item and person properties consistently had significant effects on the item difficulty and on the probability of correct answers. The modeling process also revealed some statistical or computational challenges researchers might encounter when they try to apply EIRT to complex language test data.

참고문헌 (49)

  1. Adams, R., "Multilevel Item Response Models: An Approach to Errors in Variable Regression", Journal of Educational and Behavioral Statistics, 22(1), pp.47~76 (1997)
  2. Ahmadi, A., "Issues Affecting Item Response Theory Fit in Language Assessment: A Study of Differential Item Functioning in the Iranian National University Entrance Exam", Journal of Language Teaching and Research, 3(3), pp.401~412 (2012)
  3. Alderson, J, "Testing Reading Comprehension Skills", Reading in a Foreign Language, 6(2), pp.425~438 (1990)
  4. Alderson, J., "Assessing Reading" (2000), Cambridge University Press
  5. Alderson, J., "Diagnosing Foreign Language Proficiency: the Interface Between Learning and Assessment" (2005), Continuum
  6. Alderson, J., "Cognition and Reading: Cognitive Levels as Embodied in Test Questions", Reading in a Foreign Language, 5(2), pp.253~270 (1989)
  7. Allen, M., "Understanding Regression Analysis" (1997), Springer
  8. Bachman, L., "Fundamental Considerations in Language Testing" (1990), Oxford University Press
  9. Baghaei, P., "A Cognitive Processing Model of Reading Comprehension in English as a Foreign Language Using the Linear Logistic Test Model", Learning and Individual Differences, 43, pp.100~105 (2015)
  10. Bates, D., "lme4: Linear Mixed-effects Models Using S4 Classes", R Package Version 0.999999-0, ~ (2012)
  11. Bouwmeester, S., "Understanding Phoneme Segmentation Performance by Analyzing Abilities and Word Properties", European Journal of Psychological Assessment, 27(2), pp.95~102 (2011)
  12. Buck, G., "The Subskills of Reading: Rule-space Analysis of a Multiple Choice Test of Second Language Reading Comprehension", Language Learning, 47, pp.423~466 (1997)
  13. European Survey on Language Competences: Language Proficiency in England, "Burge, B.", NFER, ~ (2013)
  14. Choi I-C., "The Impact of EFL Testing on EFL Education in Korea", Language Testing, 25(1), pp.39~62 (2008)
  15. Choi, I-C., "An Investigation into the Adequacy of Three IRT Models for Data from Two EFL Reading Tests", Language Testing, 9, pp.51~78 (1992)
  16. Daniel, R., "Designing Cognitive Complexity in Mathematical Problem-solving Items", Applied Psychological Measurement, 34(5), pp.348~364 (2010)
  17. De Boeck, P., "Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach" (2004), Springer
  18. De Boeck, P., "Handbook of Item Response Theory. Vol. 1. Models", pp.2016, Chapman & Hall
  19. Ellis, D., "Companion to Language Assessment, Vol. 3", pp.1262~1281 (2014), Wiley Blackwell
  20. Embretson, S., "Improving Construct Validity with Cognitive Psychology Principles", Journal of Educational Measurement, 38(4), pp.343~368 (2001)
  21. Fischer, G, "The Linear Logistic Test Model as an Instrument in Educational Research", Acta Psychologica, 37, pp.359~374 (1973)
  22. Fischer, G., "Some Applications of Logistic Latent Trait Models with Linear Constraints on the Parameters", Applied Psychological Measurement, 6, pp.397~416 (1982)
  23. Fox, J-P., "Bayesian Item Response Modeling: Theory and Applications" (2010), Springer
  24. Geerlings, H., "Modeling Rule-based Item Generation", Psychometrika, 76, pp.337~359 (2011)
  25. Gilbert, J., "Word and Person Effects on Decoding Accuracy: A New Look at an Old Question", Journal of Educational Psychology, 103(2), pp.489~507 (2011)
  26. Hartig, J., "Assessment of Competencies in Educational Contexts" (2008), Hogrefe & Huber
  27. Henning, G., "Dimensionality and Construct Validity of Language Tests", Language Testing, (1), pp.1~11 (1992)
  28. Jeon, M., "A Modular Approach for Item Response Theory Modeling with the R Package Flirt", Behavior Research Methods, 48(2), pp.742~755 (2016)
  29. Jeon, M., "Flexible Item Response Theory Modeling with Flirt", Applied Psychological Methods, 38, pp.404~405 (2014)
  30. Khalifa, H., "Examining Reading: Research and Practice in Assessing Second Language Reading, Studies in Language Testing 29" (2009), Cambridge University Press
  31. Lumley, T, "The Notion of Subskills in Reading Comprehension Tests: An EAP Example", Language Testing, 10(3), pp.211~234 (1993)
  32. Lynch, B, "Rethinking Assessment from a Critical Perspective", Language Testing, 18(4), pp.351~372 (2001)
  33. McNamara, T, "Item Response Theory and the Validation of an ESP Test for Health Professionals", Language Testing, 7, pp.52~76 (1990)
  34. McCulloch, C., "Generalized, Linear, and Mixed Models" (2001), Wiley
  35. Pae, T-I., "Examining the Relationship Between Differential Item Functioning and Differential Test Functioning", Language Testing, 23(4), pp.475~496 (2006)
  36. Pinheiro, J., "Mixed-Effects Models in S and S-PLUS" (2000), Springer
  37. Ravand, H, "Application of a Cognitive Diagnostic Model to a High-stakes Reading Comprehension Test", Journal of Psychoeducational Assessment, 34(8), pp.1~18 (2015)
  38. Reutzel, D, "Effects of Fluency Training on Second Graders’ Reading Comprehension", Journal of Educational Research, 86, pp.325~331 (1993)
  39. Efficient Full Information Maximum Likelihood Estimation for Multidimensional IRT Models, "Rijmen, F.", ETS, 1~31, pp.ETS Research Report Series, 2009(1) (2009)
  40. Rijmen, F., "BNLflirt: Flexible Item Response Theory Modeling with BNL", Matlab File Exchange, ~ (2013)
  41. Saida, C., "Post-hoc IRT Equating of Previously Administered English Tests for Comparison of Test Scores", Language Testing, 25(2), pp.187~210 (2008)
  42. Shin, S, "Investigating the Validity of KFL Text Difficulty as Defined by the ILR Reading Scales", Journal of the National Council of Less Commonly Taught Languages, 18, pp.159~174 (2016)
  43. Song, M, "Do Divisible Subskills Exist in Second Language(L2) Comprehension? A Structural Equation Modelling Approach", Language Testing, 25(1), pp.435~464 (2008)
  44. Spearitt, D, "Identification of Subskills of Reading Comprehension by Maximum Likelihood Factor Analysis", Reading Research Quarterly, 8, pp.92~111 (1972)
  45. Szabo, G., "Applying Item Response Theory in Language Test Item Bank Building" (2007), Peter Lang Publishing
  46. Tan, M., "The Impact of Communication and Collaboration Between Test Developers and Teachers on a High-stakes ESL Exam: Aligning External Assessment and Classroom Practices", Language Assessment Quarterly, 12(1), pp.29~49 (2015)
  47. Teemant, A, "ESL Student Perspectives on University Classroom Testing Practices", Journal of the Scholarship of Teaching and Learning, 10(3), pp.89~05 (2010)
  48. Tengberg, M, "Validation of Sub-constructs in Reading Comprehension Tests Using Teachers’ Classification of Cognitive Targets", Language Assessment Quarterly, 15(2), pp.169~182 (2018)
  49. Verbeke, G., "The Use of Score Tests for Inference on Variance Components", Biometrics, 59, pp.254~262 (2003)

관련 논문AI추천

이 간행물의 다른 논문 (18건)