Exploring the Role of Emotioncy in Reading Comprehension Test Bias
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Given the significance of psychological factors in forming biases, this study intends to introduce emotioncy, emotions evoked by senses, as a potential source of reading comprehension test bias. To this end, 514 English as a Foreign Language (EFL) learners were asked to take a 30-item multiple- choice reading comprehension test along with the emotioncy scale. Based on the emotioncy scores, participants were classified into two groups of Low and High-emotioncy. Subsequently, Rasch model- based Differential Item and Test Functioning (DIF/DTF) analyses were employed across the two target groups. The results showed that the reading comprehension test functioned differentially both at the level of the individual items and the whole test as a set of items, favoring the examinees with higher levels of emotioncy. Thus, the study provides evidence for emotioncy as a potential psychological source of reading comprehension test bias and discusses implications for educators and test developers.
Abstract
Given the significance of psychological factors in forming biases, this study intends to introduce emotioncy, emotions evoked by senses, as a potential source of reading comprehension test bias. To this end, 514 English as a Foreign Language (EFL) learners were asked to take a 30-item multiple- choice reading comprehension test along with the emotioncy scale. Based on the emotioncy scores, participants were classified into two groups of Low and High-emotioncy. Subsequently, Rasch model- based Differential Item and Test Functioning (DIF/DTF) analyses were employed across the two target groups. The results showed that the reading comprehension test functioned differentially both at the level of the individual items and the whole test as a set of items, favoring the examinees with higher levels of emotioncy. Thus, the study provides evidence for emotioncy as a potential psychological source of reading comprehension test bias and discusses implications for educators and test developers.
참고문헌 (55)
- Aryadoust, V., "An investigation of differential item functioning in the MELAB listening test", Language Assessment Quarterly, 8(4), pp.361~385 (2011)[DOI]
- Birnbaum, A., "Statistical theories of mental test scores", pp.397~479 (1968), Addison-Wesley
- Bond, T. G., "Applying the Rasch model: Fundamental measurement in the human sciences" (2015), Lawrence Erlbaum
- Borsipour, B., "The role of sensory emotions in increasing willingness to read in EFL learners", Publicaciones, 49(2), pp.169~189 (2019)[DOI]
- Bower G. W., "The handbook of emotion and memory: Research and theory", pp.3~31 (1992), Erlbaum
- Breland, H., "An analysis of TOEFL-CBT writing prompt difficulty and comparability for different gender groups", ETS Research Report Series, 1, pp.1~54 (2004)[DOI]
- Camilli, G., "Methods for identifying biased test items Vol. 4" (1994), Sage
- Characteristics associated with Differential Item Functioning on the Scholastic Aptitude Test: Gender and majority, "minority group comparisons", Carlton, S. T., 1992, ETS Research Report 92-64
- Chalmers, R. P, "It might not make a big DIF: Improved Differential Test Functioning statistics that account for sampling variability", Educational and Psychological Measurement, 76(1), pp.114~140 (2015)[DOI]
- Chen, Z., "Linguistic and cultural bias in language proficiency tests", Language Testing, 2(2), pp.155~163 (1985)
- Elder, C., "The effect of language background on “foreign” language test performance: The case of Chinese, Italian, and Modern Greek", Language Learning, 46(2), pp.233~282 (1996)[DOI]
- Frenzel, A. C., "Girls and mathematics: A “hopeless” issue? A controlvalue approach to gender differences in emotions towards mathematics", European Journal of Psychology of Education, 22(4), pp.497~514 (2007)[DOI]
- "Differential Item Functioning: Performance by sex on reading comprehension tests" (1991)[URL]
- Garrett, P., "Theorizing affect in foreign language learning: An analysis of one learner’s responses to a communicative Portuguese course", The Modern Language Learner, 93(2), pp.209~226 (2009)[DOI]
- Masoud Geramipour, "A Gender–Related Differential Item Functioning Study of an English Test", Journal of Asia TEFL, 16(2), pp.674~682 (2019), 아시아테플
- Greenspan, S. I., "The first idea: How symbols, language, and intelligence evolved from our primate ancestors to modern humans" (2004), Da Capo Press
- Hulin, C. L., "Item response theory: Applications to psychological measurement" (1983), Irwin
- Karami, M., "A probe into EFL learners’ emotioncy as a source of test bias: Insights from differential item functioning analysis", Studies in Educational Evaluation, 60, pp.170~178 (2019)[DOI]
- Kunnan, A. J., "Test fairness", European language testing in a global context: proceedings of the ALTE Barcelona Conference, pp.27~48 (2004), Cambridge University Press
- Differential Item Functioning for males and females on SATVerbal Reading sub score items: follow-up study, "Lawrence, I. M.", Educational Testing Service, ~, pp.ETS Research Report 89-22 (1989)
- Linacre, J. M., "Investigating rating scale category utility", Journal of Outcome Measurement, 3, pp.103~122 (1999)
- "A user guide to WINSTEPS MINISTEPS Rasch-model computer programs" (2012)[URL]
- Lord, F. M., "Applications of item response theory to practical testing problems", Lawrence Erlbaum Associates (1980)[DOI]
- Maclntyre, P. D., "Sex and age effects on willingness to communicate, anxiety, perceived competence, and L2 motivation among junior high school French immersion students", Language Learning, 52, pp.537~564 (2002)[DOI]
- Makransky, G, "Modeling differential item functioning with group-specific item parameters: A computerized adaptive testing application", Measurement, 46(9), pp.3228~3237 (2013)[DOI]
- Maller, S. J., "Deafness and WISC-III item difficulty: Invariance and fit", Journal of School Psychology, 35(3), pp.299~314 (1997)[DOI]
- Maller, S. J., "Differential item functioning in the WISC-III: Item parameters for boys and girls in the national standardization sample", Educational and Psychological Measurement, 61(5), pp.793~817 (2001)[DOI]
- McNamara, T., "Language testing: The social dimension" (2006), Blackwell Publishing
- Moghadam, M., "The application of Kunnan’s test fairness framework (TFF) on a reading comprehension test", Language Testing in Asia, 10 (2020)[DOI]
- Nisbet, I., "Fair high-stakes assessment in the long shadow of Covid-19", Assessment in Education: Principles, Policy & Practice (2022)[DOI]
- Pae, T. I., "DIF for examinees with different academic backgrounds", Language Testing, 21(1), pp.53~73 (2004)[DOI]
- Pekrun, R., "The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice", Educational Psychology Review, 18, pp.315~334 (2006)[DOI]
- Pekrun, R., "Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative- search", Educational Psychologist, 37(2), pp.91~105 (2002)[DOI]
- Pishghadam, R, "Emotioncy in language education: From exvolvement to involvement", 2nd conference of interdisciplinary approaches to anguage teaching, literature, and translation studies (2015)
- Pishghadam, R., "Emotioncy, extraversion, and anxiety in willingness to communicate in English", 5th International conference on language, education, and innovation (2016)
- Pishghadam, R, "Introducing emotioncy tension as a potential source of identity crises", Interdisciplinary conference on cultural identity and philosophy of self (2016)
- Pishghadam, R., "Emotion-based language instruction (EBLI) as a new perspective in bilingual education", Multilingual Education, 3(9), pp.1~16 (2013)[DOI]
- Pishghadam, R., "Introducing emotioncy as a potential source of test bias: A mixed Rasch modeling study", International Journal of Testing, 17(2), pp.127~140 (2016)[DOI]
- Pishghadam, R., "Conceptualizing sensory relativism in light of emotioncy: A movement beyond linguistic relativism", International Journal of Society, Culture & Language, 4(2), pp.11~21 (2016)
- Pishghadam, R., "Emotioncy: A post-linguistic approach toward vocabulary learning and retention", Sri Lanka Journal of Social Sciences, 39(1), pp.27~36 (2016)[DOI]
- Philips, D., "Longman complete course for the TOEFL test: Preparation for the computer and paper tests" (2005), Longman
- Ryan, K. E., "Differential item functioning on two tests of EFL proficiency", Language Testing, 9(1), pp.12~29 (1992)[DOI]
- Scheuneman, J. D., "Using differential item functioning procedures to explore sources of item difficulty and group performance characteristics", Journal of Educational Measurement, 27(2), pp.109~131 (1990)[DOI]
- Schutz, P. A., "Introduction: Emotions in education", Educational Psychologist, 37(2), pp.67~68 (2002)[DOI]
- Schutz, P. A., "Emotion in education" (2007), Elsevier
- Sehmitt, A. P., "Differential item functioning for minority examinees on the SAT", Journal of Educational Measurement, 27(1), pp.67~81 (1990)[DOI]
- Shahian, L., "Flow and reading comprehension: Testing the mediating role of emotioncy", Issues in Educational Research, 27(3), pp.527~549 (2017)
- Shayesteh, S., "FN400 and LPC responses to different degree of sensory involvement: A study of sentence comprehension", Advances in Cognitive psychology, 16(1), pp.45~58 (2020)
- Shohamy, E., "Testing methods, testing consequences: Are they ethical? Are they fair?", Language Testing, 14(3), pp.340~349 (1997)[DOI]
- Stark, S., "Examining the effects of Differential Item (Functioning and Differential) Test Functioning on selection decisions: When are statistically significant effects practically important?", Journal of Applied Psychology, 89(3), pp.497~508 (2004)[DOI]
- Thissen, D., "Test validity", pp.147~172 (1988), Lawrence Erlbaum Associates, Inc
- Thissen, D., "Differential item functioning", pp.67~113 (1993), Lawrence Erlbaum Associates
- Yu, L., "Using a Differential Item Functioning (DIF) procedure to detect differences in opportunity to learn (OTL)", The annual meeting of the American Educational Research Association (2006)
- Zumbo, B. D., "Does item-level DIF manifest itself in scale-level analyses? Implications for translating language tests", Language Testing, 20(2), pp.136~147 (2003)[DOI]
- Matthew P. Wallace, "Fairness and Justice in L2 Classroom Assessment: Perceptions from Test Takers", Journal of Asia TEFL, 15(4), pp.1051~1064 (2018), 아시아테플
