Investigating Effects of Tasks on Examinee Performance in a Computer-Delivered Speaking Test
2
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
With growing interest in test tasks to define language ability, numerous task effects have been examined in second language speaking assessment, especially in semi-direct tests. Due to the nature of semi-direct tests, where real interaction is missing, tasks play a crucial role in eliciting examinee language for evaluation. However, task effects have rarely been investigated separately for different proficiency levels based on both statistical and qualitative analyses. To this end, the current study examined the effects of independent and integrated speaking tasks on the test performance at two proficiency levels of examinees. Twelve examinees (six intermediate and six advanced) responded to five speaking tasks (two independent and three integrated) in a computer-delivered, semi-direct speaking test. A many-facet Rasch measurement analysis of examinee scores on five scoring criteria found that the two groups of examinees did not have the same level of difficulty with independent and integrated tasks. The following qualitative analysis of actual responses showed that both intermediate and advanced examinees were influenced by listening texts of integrated tasks, but for different reasons and at varying degrees. The findings contribute to a better understanding of task effects that vary across different speaking levels.
Abstract
With growing interest in test tasks to define language ability, numerous task effects have been examined in second language speaking assessment, especially in semi-direct tests. Due to the nature of semi-direct tests, where real interaction is missing, tasks play a crucial role in eliciting examinee language for evaluation. However, task effects have rarely been investigated separately for different proficiency levels based on both statistical and qualitative analyses. To this end, the current study examined the effects of independent and integrated speaking tasks on the test performance at two proficiency levels of examinees. Twelve examinees (six intermediate and six advanced) responded to five speaking tasks (two independent and three integrated) in a computer-delivered, semi-direct speaking test. A many-facet Rasch measurement analysis of examinee scores on five scoring criteria found that the two groups of examinees did not have the same level of difficulty with independent and integrated tasks. The following qualitative analysis of actual responses showed that both intermediate and advanced examinees were influenced by listening texts of integrated tasks, but for different reasons and at varying degrees. The findings contribute to a better understanding of task effects that vary across different speaking levels.
참고문헌 (35)
- Bachman, L. F., "Fundamental considerations in language testing" (1990), Oxford University Press
- Bachman, L. F., "Some reflections on task-based language performance assessment", Language Testing, 19, pp.453~476 (2002)
- Brindley, G., "Task-centred assessment in language learning: The promise and the challenge, In Language and learning", pp.79~94 (1994), Hong Kong Education Department
- An examination of rater orientations and test-taker performance on English-for academic-purposes speaking tasks, "Brown, A.", Educational Testing Service, ~, pp.TOEFL Monograph Report No. 29 (2005)
- Brown, J. D., "Essay prompts and topics: Minimizing the effect of mean differences", Written Communication, 8, pp.533~556 (1991)
- Brown, J. D., "An investigation of second language task-based performance assessments" (2002), University of Hawaii Press
- Chapelle, C., "Construct definition and validity inquiry in SLA research, In Interfaces between second language acquisition and language testing research", pp.32~70 (1998), Cambridge University Press
- Clark, J. L. D., "Theoretical and technical considerations in oral proficiency testing, In Testing language proficiency", pp.10~28 (1975), Center for Applied Linguistics
- Clark, J. L. D., "Validation of a tape-mediated ACTFL", ILR-scale based test of Chinese speaking proficiency, 1988, pp.5, 187~205
- Cumming, A., "A teacher-verification study of speaking and writing prototype tasks for a new TOEFL", Language Testing, 21, pp.159~197 (2004)
- Cumming, A., "Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL", Assessing Writing, 10, pp.5~43 (2005)
- Elder, C., "Estimating the difficulty of oral proficiency tasks: What does the test-taker have to offer", Language Testing, 19, pp.347~368 (2002)
- "Test of English as a foreign language and test of spoken English information and registration bulletin"[URL]
- Farris, C. S., "A semiotic analysis of sajiao as a gender marked communication style in Chinese, In Unbound Taiwan: Close-ups from a distance", pp.1~29 (1995), Center for East Asian Studies, University of Chicago
- TOEFL 2000-Writing: Composition, community, and assessment, "Hamp-Lyons, L.", Educational Testing Service, ~, pp.TOEFL Monograph Report No. 5 (1997)
- Jones, R. L., "Performance testing of second language proficiency, In Concepts in language testing: Some recent studies", pp.50~57 (1979), TESOL
- Jones, R. L., "Second language performance testing: An overview, In Second language performance testing", pp.15~24 (1985), University of Ottawa Press
- Kim, H., "Investigating the effects of context and task type on second language speaking ability", 박사, pp.2009
- Kim, H. J., "Investigating raters’ development of rating ability on a second language speaking assessment", 박사, pp.2011
- Lado, R., "Language testing: The construction and use of foreign language tests" (1961), Longman
- Lee, Y., "Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks", Language Testing, 23, pp.131~166 (2006)
- Lewkowicz, J., "Writing from sources: Does source material help or hinder students’ performance?, In Language and learning", pp.204~217 (1994), Hong Kong Education Department
- Linacre, J. M., (2010), Winsteps.com
- Lumley, T., "The effect of test-taker gender, audience and topic on task performance in tape-mediated assessment of speaking", Language Testing, 22, pp.415~437 (2005)
- Malabonga, V., "Self-assessment, preparation and response time on a computerized oral proficiency test", Language Testing, 22, pp.59~92 (2005)
- McNamara, T. F., "Measuring second language performance" (1996), Longman
- Messick, S., "The interplay of evidence and consequences in the validation of performance assessments", Educational Researcher, 23, pp.13~23 (1994)
- Read, J., "Providing relevant content in an EAP writing test", English for Specific Purposes, 9, pp.109~121 (1990)
- The reading, writing, speaking, and listening tasks important for academic success at the undergraduate and graduate levels, "Rosenfeld, M.", Educational Testing Service, ~, pp.TOEFL Monograph Report No. 21 (2001)
- Shohamy, E., "The validity of direct versus semi-direct oral tests", Language Testing, 11, pp.99~123 (1994)
- Skehan, P., "A cognitive approach to language learning" (1998), Oxford University Press
- Watanabe, Y., "Read-to-write tasks for the assessment of second language academic writing skills: Investigating text features and rater reactions", 박사, pp.2001
- Weir, C., "Understanding and Developing Language Tests" (1993), Prentice Hall
- Wiersma, W., "Research methods in education: An introduction" (2005), Pearson
- Wigglesworth, G., "An investigation of planning time and proficiency level on oral test discourse", Language Testing, 14, pp.85~106 (1997)