본문 바로가기
KERD
논문KCI 등재

Exploring the Cognitive Validity of the iBT TOEFL Reading Test

Exploring the Cognitive Validity of the iBT TOEFL Reading Test1

0

조회

0

다운로드

2

인용

0

참고문헌

키워드

초록

This study explores the cognitive validity of the iBT TOEFL reading test by investigating Chinese learners’ reading processes during the test. Forty six Chinese ESL learners in a large Midwestern university were asked to complete one computerized testlet from a TOEFL reading practice test on a computer screen. During the test, learners’ eye-movements were recorded and analyzed both quantitatively (e.g. total fixation durations) and qualitatively (e.g. gaze plots, heat maps, and videos). Results showed that some of the test items failed to elicit the intended cognitive processes from test takers; participants did not always use inferencing reading skills to solve vocabulary items and often did not read a given text expeditiously. Further discussed are the value of cognitive validity and the direction of future test validation.

Abstract

This study explores the cognitive validity of the iBT TOEFL reading test by investigating Chinese learners’ reading processes during the test. Forty six Chinese ESL learners in a large Midwestern university were asked to complete one computerized testlet from a TOEFL reading practice test on a computer screen. During the test, learners’ eye-movements were recorded and analyzed both quantitatively (e.g. total fixation durations) and qualitatively (e.g. gaze plots, heat maps, and videos). Results showed that some of the test items failed to elicit the intended cognitive processes from test takers; participants did not always use inferencing reading skills to solve vocabulary items and often did not read a given text expeditiously. Further discussed are the value of cognitive validity and the direction of future test validation.

참고문헌 (46)

  1. Allan, A., "Development and validation of a scale to measure test-wiseness in EFL", ESL reading test takers, 1992, pp.9(2), 101~119
  2. Bax, S., "Investigating learners’ cognitive reading processes during a computer-based CAE reading test", University of Cambridge ESOL Examinations Research Note, 47, pp.3~14 (2012)
  3. Bax, S., "The cognitive processing of candidates during reading tests : Evidence from eye-tracking", Language Testing, 30(3), pp.1~25 (2013)
  4. Bernhardt, E., "Progress and procrastination in second language reading", Annual Review of Applied Linguistics, 25, pp.133~150 (2005)
  5. Bernhardt, E., "Understanding advanced second-language reading" (2011), Routledge
  6. Breznitz, Z., "Fluency in reading : Synchronization of processes" (2006), Erlbaum
  7. Borsboom, D., "Measuring the mind : Conceptual issues in contemporary psychometrics" (2005), Cambridge University Press
  8. Carver, R. P., "Reading rate : A review of research and theory" (1990), Academic Press
  9. Carver, R., "Merging the simple view of reading with rauding theory", Journal of Literacy Research, 25(4), pp.439~455 (1993)
  10. Strategies in responding to the new TOEFL reading tasks, "Cohen, A. D.", Educational Testing Service, ~, pp.TOEFL Monograph Series Report No. 33 (2006)
  11. Cohen, A., ""I want to go back to the text" : Response strategies on the reading subtest of the new TOEFL(R)", Language Testing, 24(2), pp.209~250 (2007)
  12. Cushing-Weigle, S., "Reading rate improvement in university ESL classes", CATESOL Journal, 9, pp.55~71 (1996)
  13. Field, J., "Into the mind of the academic listener", Journal of English for Specific Purposes, 10, pp.102~112 (2011)
  14. Fraser, C. A., "Reading rate in L1 Mandarin Chinese and L2 English across five reading tasks", The Modern Language Journal, 91(3), pp.372~394 (2007)
  15. Gao, L., "Use of tree-based regression in the analyses of L2 reading test items", Language Testing, 28(1), pp.77~104 (2011)
  16. Grabe, W., "Fluency in reading-Thirty-five years later", Reading in a Foreign Language, 22(1), pp.71~83 (2010)
  17. Innami, Y., "A meta-analysis of test format effects on reading and listening test performance : Focus on multiple-choice and open-ended formats", Language Testing, 26(2), pp.219~244 (2009)
  18. Jeon, E. H., "L2 Reading comprehension and its correlates : A meta-analysis", Language Learning, 64(1), pp.160~212 (2014)
  19. Just, M. A., "A theory of reading : From eye fixations to comprehension", Psychological Review, 87, pp.329~354 (1980)
  20. Khalifa, H., "Examining reading: Research and practice in assessing second language reading" (2009), Cambridge University Press
  21. Killen, R., "Validity in outcomes-based assessment", Perspectives in Education, 21(1), pp.1~14 (2003)
  22. Koda, K., "Insights into second language reading: A cross-linguistic approach" (2005), Cambridge University Press
  23. Koda, K., "Reading and language learning : Crosslinguistic constraints on second language reading development", Language Learning, 57(1), pp.1~44 (2007)
  24. Nation, P., "A bilingual vocabulary size test of English for Vietnamese learners", RELC Journal, 42(1), pp.86~99 (2011)
  25. Lee, K. -G., "Analysis of the discriminating variables on a FLEX Spanish : Around the listening, grammar, and vocabulary testing items", Studies in Foreign Language Teaching, 24(1), pp.47~72 (2010)
  26. Lemaire, B., "A cognitive computational model of eye movements investigating visual strategies on textual material", 33rd annual meeting of the Cognitive Science Society, pp.1146~1151 (2011)
  27. Logan, G. D., "Automaticity and reading : Perspectives from the instance theory of automatization", Reading & Writing Quarterly : Overcoming Learning Difficulties, 13(2), pp.123~146 (1997)
  28. Matin, E., "Saccadic suppression : A review and an analysis", Psychological Bulletin, 81, pp.899~917 (1974)
  29. Morrison, R. E., "Manipulation of stimulus onset delay in reading : Evidence for parallel programming of saccades", Journal of Experimental Psychology : Human Perception and Performance, 10(5), pp.667~682 (1984)
  30. Paulson, E. J., "Adult readers' eye movements during the production of oral miscues", doctor, pp.2000
  31. Paulson, E. J., "Does the degrees of reading power assessment reflect the reading process? An eye-movement examination", Journal of Adolescent & Adult Literacy, 46(3), pp.234~244 (2002)
  32. Pollatsek, A., "Tests of the E-Z Reader model : Exploring the interface between cognition and eye-movement control", Cognitive Psychology, 52, pp.1~56 (2006)
  33. Rayner, K., "Eve movements in reading and information processing : 20 years of research", Psychological Bulletin, 124, pp.372~422 (1998)
  34. Rayner, K., "Eye movements and attention in reading, scene perception, and visual search", The Quarterly Journal of Experimental Psychology, 62(8), pp.1457~1506 (2009)
  35. Rayner, K., "Eye movements as reflections of comprehension processes in reading", Scientific Studies of Reading, 10(3), pp.241~255 (2006)
  36. Rayner, K., "The psychology of reading" (1989), Prentice Hall
  37. Reichle, E. D., "Toward a model of eye movement control in reading", Psychological Review, 105, pp.125~157 (1998)
  38. Reichle, E. D., "Using E-Z Reader to model the effects of higher level language processing on eye movements during reading", Psychonomic Bulletin & Review, 16, pp.1~21 (2009)
  39. Sawaki, Y., "Factor structure of the TOEFL Internet-based test", Language Testing, 26(1), pp.5~30 (2009)
  40. Sereno, S. C., "Measuring word recognition in reading : Eye movements and event-related potentials", Trends in Cognitive Sciences, 7(11), pp.489~493 (2003)
  41. Solheim, O. J., "Eye-tracking as a tool in process-oriented reading test validation", International Electronic Journal of Elementary Education, 4(1), pp.153~168 (2011)
  42. Urquhart, S., "Reading in a second language : Process, product and practice" (1998), Longman
  43. Van Gelderen, A., "Development of adolescent reading comprehension in language 1 and language 2 : A longitudinal analysis of constituent components", Journal of Educational Psychology, 99(3), pp.477~491 (2007)
  44. Weir, C. J., "Language testing and validation" (2005), Palgrave McMillan
  45. Weir, C., "The cognitive processes underlying the academic reading construct as measured by IELTS", IELTS Collected Papers : Research in Reading and Listening Assessment, 2, pp.157~185 (2006)
  46. Winke, P., "The effects of testwiseness and test-taking anxiety on L2 listening test performance: A visual (eye-tracking) and attentional investigation", IELTS Research Reports Online Series, (3), pp.1~30 (2014)

관련 논문AI추천

이 간행물의 다른 논문 (9건)