Effects of Examinees’ Perceptions of Interlocutor Proficiency on Paired Oral Assessment Results
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This study aimed to explore the effects of examinees’ perceptions of interlocutors’ proficiency on their test results in paired oral tests. A group of 22 Korean English as a foreign language (EFL) high school students participated in the study. They were divided into three groups based on high, intermediate, and low English proficiency and were interviewed twice: once before the test and once after. Each examinee was paired with and tested with three interlocutors with high, intermediate, and low English proficiency levels. Raw scores from each rater were analyzed to examine potential interactions among the factors using the bias analysis from the Many-Facet Rasch Model. The results showed that the examinees’ perceptions about the effect of interlocutor proficiency on their test results were not consistent nor reliable enough to influence the test results. This study provides evidence that the paired format can be used as a valid and reliable tool to assess EFL learners’ speaking ability regardless of interlocutor proficiency levels, which can be assumed to influence examinees’ perceptions of the oral test results.
Abstract
This study aimed to explore the effects of examinees’ perceptions of interlocutors’ proficiency on their test results in paired oral tests. A group of 22 Korean English as a foreign language (EFL) high school students participated in the study. They were divided into three groups based on high, intermediate, and low English proficiency and were interviewed twice: once before the test and once after. Each examinee was paired with and tested with three interlocutors with high, intermediate, and low English proficiency levels. Raw scores from each rater were analyzed to examine potential interactions among the factors using the bias analysis from the Many-Facet Rasch Model. The results showed that the examinees’ perceptions about the effect of interlocutor proficiency on their test results were not consistent nor reliable enough to influence the test results. This study provides evidence that the paired format can be used as a valid and reliable tool to assess EFL learners’ speaking ability regardless of interlocutor proficiency levels, which can be assumed to influence examinees’ perceptions of the oral test results.
참고문헌 (38)
- Alderson, J. C., "Language test construction and evaluation" (1995), Cambridge University Press
- Aryadoust, V., "Gender and academic major bias in peer assessment of oral presentations", Language Assessment Quarterly, 13(1), pp.1~24 (2016)
- Bachman, L. F., "Problems in examining the validity of the ACTFL oral proficiency interview", Studies in Second Language Acquisition, 10(2), pp.149~164 (1988)
- Bennett, R., "Is linguistic ability variation in paired oral language testing problematic?", ELT Journal, 66(3), pp.337~346 (2011)
- Berry, V., "Personality differences and oral test performance (Vol. 7)" (2007), Peter Lang
- outgoingness, match of interlocutors’ proficiency level, and gender on individual scores, "L2 group oral testing: The influence of shyness", pp.2004, Bonk, W. J.
- Brown, A., "The devil is in the detail", TESOL Quarterly, 38(3), pp.524~538 (2004)
- Csépes, I., "Measuring oral proficiency through paired-task performance (Vol. 14)" (2009), Peter Lang
- Davis, L., "The influence of interlocutor proficiency in a paired oral assessment", Language Testing, 26(3), pp.367~396 (2009)
- Együd, G., "Oral testing in pairs-a secondary school perspective", ELT Journal, 55(1), pp.70~76 (2001)
- Foot, M. C., "Relaxing in pairs", ELT Journal, 53(1), pp.36~41 (1999)
- He, A., "Talking and testing: Discourse approaches to the assessment of oral proficiency", pp.1~24 (1998), John Benjamins Publishing Company
- Iwashita, N., "The validity of the paired interview format in oral performance assessment", Melbourne Papers in Language Testing, 5(2), pp.51~65 (1996)
- Jacoby, S., "Co-construction : An introduction", Research on Language and Social Interaction, 28, pp.171~183 (1995)
- Jenkins, J., "Testing pronunciation in communicative exams", Speak Out, 20, pp.7~11 (1997)
- Johnson, M., "The art of non-conversation: A re-examination of the validity of the oral proficiency interview" (2001), Yale University Press
- Katona, L., "Talking and testing: Discourse approaches to the assessment of oral proficiency", pp.239~270 (1998), John Benjamins Publishing Company
- Kramsch, C., "From language proficiency to interactional competence", The Modern Language Journal, 70, pp.366~372 (1986)
- Lazaraton, A., "The structural organization of a language interview : A conversation analytic perspective", System, 20(3), pp.373~386 (1992)
- "A user’s guide to FACETS: Rasch model computer program" (2014)[URL]
- Long, M. H., "Three approaches to task-based syllabus design", TESOL Quarterly, 26(1), pp.27~56 (1992)
- Luoma, S., "Assessing speaking" (2004), Cambridge University Press
- Designing second language performance assessments (No. 18), "Norris, J. M.", National Foreign Language Resource Center, ~ (1998)
- Norton, J., "The paired format in the Cambridge Speaking Tests", ELT Journal, 59(4), pp.287~297 (2005)
- O’Loughlin, K., "The impact of gender in oral proficiency testing", Language Testing, 19(2), pp.169~192 (2002)
- O’Sullivan, B., "Using observation checklists to validate speaking test pair tasks", Language Testing, 19(1), pp.33~56 (2002)
- O’Sullivan, B., "Modelling performance in tests of spoken language (Vol. 7)" (2008), Peter Lang
- Saville, N., "Assessing speaking in the revised FCE", ELT Journal, 53(1), pp.42~51 (1999)
- Skehan, P., "researching pedagogic tasks: Second language learning, teaching and testing", pp.167~185 (2001), Longman
- Son, Y. A., "The interlocutor proficiency effect on test-taker performance in a paired oral assessment", master’s, pp.2013
- Swain, M., "Examining dialogue : Another approach to content specification and to validating inferences drawn from test scores", Language Testing, 18(3), pp.275~302 (2001)
- Taylor, L., "Investigating the paired speaking test format", Cambridge ESOL Research Notes, 2, pp.14~15 (2000)
- Teng, H. C., "Interlocutor proficiency in paired speaking tests", JALT 2013 Conference Proceedings (2014), JALT
- Van Lier, L., "Reeling, writing, drawling, stretching, and fainting in coils : Oral proficiency interviews as conversation", TESOL Quarterly, 23, pp.489~508 (1989)
- Young, R., "Talking and testing: Discourse approaches to the assessment of oral proficiency", pp.355~382 (1998), John Benjamins Publishing Company
- Young, R., "Discourse variation in oral proficiency interviews", Studies in Second Language Acquisition, 14(4), pp.403~424 (1992)
- Vo, S., "Effects of task types on interactional competence in oral communication assessment", doctoral, pp.2019
- Wang, L., "Assessing interactional competence in second language paired speaking tasks", doctoral, pp.2015
