Testing the Comparability of Different L2 Oral Test Tasks
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study examined the comparability of three L2 oral test tasks (topic discussion, information gap, and semi-direct speaking) in the measurement of the four oral traits (pronunciation, fluency, grammar, and vocabulary). In examining the comparability, a CFA approach to a correlated traits/correlated methods design was employed, and evidence of convergent and discriminant validity was tested in the measurement of the traits by the methods. Through the model comparison approach (Byrne, 2006; Wideman, 1985), the hypothesized model was compared against a series of more restrictive models using χ2 difference tests, and the information from the path coefficients was examined to check the equivalence further. The findings revealed that the method effect in the measurement model was so strong that the trait was not fully reflected in the measurement process. Also, the effect of method on trait was not consistent in that topic discussion produced most trait-related information among the test tasks. Moreover, methods were found discriminant although traits were not. Thus, the three methods cannot be considered comparable in measuring the L2 oral traits, and such incomparability was particularly evident between the two methods of group oral and semi-direct speaking.
Abstract
This study examined the comparability of three L2 oral test tasks (topic discussion, information gap, and semi-direct speaking) in the measurement of the four oral traits (pronunciation, fluency, grammar, and vocabulary). In examining the comparability, a CFA approach to a correlated traits/correlated methods design was employed, and evidence of convergent and discriminant validity was tested in the measurement of the traits by the methods. Through the model comparison approach (Byrne, 2006; Wideman, 1985), the hypothesized model was compared against a series of more restrictive models using χ2 difference tests, and the information from the path coefficients was examined to check the equivalence further. The findings revealed that the method effect in the measurement model was so strong that the trait was not fully reflected in the measurement process. Also, the effect of method on trait was not consistent in that topic discussion produced most trait-related information among the test tasks. Moreover, methods were found discriminant although traits were not. Thus, the three methods cannot be considered comparable in measuring the L2 oral traits, and such incomparability was particularly evident between the two methods of group oral and semi-direct speaking.
참고문헌 (31)
- Bachman, L. F., "The construct validation of the FSI oral interview", Language Learning, 31(1), pp.67~85 (1981)
- Bentler, P. M., "EQS 6 structural equations program manual" (2003), Multivariate Software
- Berry, V., "A study of the interaction between individual personality differences and oral performance test facets", doctor, pp.2004
- Bonk, W. J., "A many-facet Rasch analysis of the second language group oral discussion task", Language Testing, 20(1), pp.89~110 (2003)
- Brown, A., "Interviewer variation and the co-construction of speaking proficiency", Language Testing, 20(1), pp.1~25 (2003)
- Byrne, B. M., "Burnout: Testing for the validity, replication, and invariance of causal structure across elementary, intermediate, and secondary teachers", American Educational Research Journal, 31(3), pp.645~673 (1994)
- Byrne, B. M., "Structural equation modeling with EQS and EQS Windows: Basic concepts, applications, and programming" (2006), Lawrence Erlbaum Associates, Inc
- Duff, P. A., "Tasks and language learning: Integrating theory and practice", pp.57~95 (1993), Multilingual Matters
- Folland, D., "Towards objectivity in group oral testing", English Language Teaching Journal, 30(2), pp.156~167 (1976)
- Fulcher, G., "Testing tasks: Issues in task design and the group oral", Language Testing, 13(1), pp.23~51 (1996)
- Fulcher, G., "Testing second language speaking" (2003), Pearson Education Limited
- Gan, Z., "Interaction in group oral assessment: A case study of higher- and lower-scoring students", Language Testing, 27(4), pp.585~602 (2010)
- Henning, G., "Oral proficiency testing: Comparative validities of interview, imitation, and completion methods", Language Learning, 33(3), pp.315~332 (1983)
- Hildon, J., "Language testing in the 1990s", pp.189~197 (1991), Modern English Publications and the British Council
- Hu, L. T., "Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives", Structural Equation Modeling, 6(1), pp.1~55 (1999)
- Leaper, D. A., "The influence of prompt on group oral tests", Language Testing, 31(2), pp.177~204 (2014)
- Luoma, S., "Comparability of a tape-mediated and face-to-face test of speaking: A triangulation study", pp.1997
- Nakatsuhara, F., "Effects of test-taker characteristics and the number of participants in group oral tests", Language Testing, 28(4), pp.483~508 (2011)
- Nakatsuhara, F., "The co-construction of conversation in group oral tests" (2013), Peter Lang
- Ockey, G., "The effects of group members' personalities on a test taker's L2 group oral discussion test scores", Language Testing, 26(2), pp.161~186 (2009)
- O’Donnell, D., "A FACETS analysis of the sub-traits of the KEPT oral rating scale", The Journal of Kanda University of International Studies, 20, pp.385~404 (2008)
- O’Loughlin, K., "The equivalence of direct and semi-direct speaking tests (Studies in language testing 13)" (2001), CUP
- O’Sullivan, B., "Learner acquaintanceship and oral proficiency test pair-task performance", Language Testing, 19(3), pp.277~295 (2002)
- Park, S., "An exploration of examinee abilities, rater performance, and task differences using diverse analytic techniques", doctor, pp.2008
- Pica, T., "Information gap tasks: Their multiple roles and contributions to interaction research methodology", SSLA, 28(2), pp.301~338 (2006)
- Shohamy, E., "Introducing a new comprehensive test of oral proficiency", English Language Teaching Journal, 40(3), pp.212~240 (1986)
- Skehan, P., "Researching pedagogic tasks: Second language learning, teaching and testing", pp.167~185 (2001), Pearson Educational Limited
- Swain, M., "Examining dialogue: Another approach to content specification and to validating inferences drawn from test scores", Language Testing, 18(3), pp.275~302 (2001)
- Van Moere, A., "Validity evidence in a university group oral test", Language Testing, 23(4), pp.411~440 (2006)
- Weir, C. J., "Establishing test form and individual task comparability: A case study of a semi-direct speaking test", Language Testing, 23(2), pp.167~197 (2006)
- Widaman, K. F., "Hierarchically tested covariance structure models for multitrait-multimethod data", Applied Psychological Measurement, 9(1), pp.1~26 (1985)