The Validation Process in Developing a Web-based English Speaking and Writing Test
1
조회
0
다운로드
10
인용
0
참고문헌
키워드
초록
This article summarizes findings from investigations into the development and use of a Web-based English speaking and writing test for local use as a low-stakes test. The study focused on the validation process in language test development of the test following Kim’s (2006a) testing model. Two research questions were proposed to examine test validity and task difficulty of the test. 53 test takers at a Korean university participated in the test implementation and a post-survey. Validity arguments were gathered from design and validation implications in test development, test results, and survey data, which built validity evidence in the Usefulness Analysis Table (Bachman & Palmer, 1996). They evidenced validity of the test and its development. Research findings from the statistical analysis of the test results and survey reported that test takers need more preparation time and response time on writing and familiarity enhancement with task content. The findings on task difficulty across proficiency levels indicated that both groups showed similar perceptions of task difficulty while the test takers in the low level group revealed that their difficulty perceptions were inappropriate for overall and writing scores.
Abstract
This article summarizes findings from investigations into the development and use of a Web-based English speaking and writing test for local use as a low-stakes test. The study focused on the validation process in language test development of the test following Kim’s (2006a) testing model. Two research questions were proposed to examine test validity and task difficulty of the test. 53 test takers at a Korean university participated in the test implementation and a post-survey. Validity arguments were gathered from design and validation implications in test development, test results, and survey data, which built validity evidence in the Usefulness Analysis Table (Bachman & Palmer, 1996). They evidenced validity of the test and its development. Research findings from the statistical analysis of the test results and survey reported that test takers need more preparation time and response time on writing and familiarity enhancement with task content. The findings on task difficulty across proficiency levels indicated that both groups showed similar perceptions of task difficulty while the test takers in the low level group revealed that their difficulty perceptions were inappropriate for overall and writing scores.
참고문헌 (27)
- Breiner-Sanders, K. E., "Revised ACTFL proficiency guidelines-speaking", Foreign Language Annals, 33(1), pp.13~18 (2000)
- Breiner-Sanders, K. E., "Preliminary proficiency guidelines—writing revised 2001", Foreign Language Annals, 35(1), pp.9~15 (2002)
- Bachman, L. F., "Language testing in practice" (1996), Oxford University Press
- Brindley, G., "Exploring task difficulty in ESL listening assessment", Language Testing, 19(4), pp.369~394 (2002)
- Brown, H. D., "Language assessment: principles and classroom practices" (2003), Pearson Longman
- Chapelle, C. A., "Validation of a Web-based ESL test", Language Testing, 20(4), pp.409~439 (2003)
- Daniel A. Craig, "Anxiety and Performance in Videoconferenced and Face-to-Face Oral Interviews", 멀티미디어 언어교육, 13(3), pp.9~32 (2010), 한국멀티미디어언어교육학회
- Davidson, F., "Testcraft: A teacher’s guide to writing and using language test specifications" (2002), Yale University Press
- Davies, A., "Dictionary of language testing" (1999), Cambridge University Press
- Elder, C., "Estimating the difficulty of oral proficiency tasks: what does the test-taker have to offer", Language Testing, 19(4), pp.347~368 (2002)
- Fulcher, G, "Interface design in computer-based language testing", Language Testing, 20(4), pp.384~408 (2003)
- Fulcher, G., "Testing second language speaking" (2003), Pearson Education Limited
- Kim, J. T., "The effectiveness of test-takers’ participation in development of an innovative web-based speaking test for international Teaching Assistantsat American colleges" (2006), University of Illinois, Urbana-Champaign
- Kim, Jung Tae, "Context Validity of a Speaking Test: Needs Analysis and Content Analysis", 응용언어학, 22(2), pp.137~158 (2006), 한국응용언어학회
- Kim, Jung Tae, "Validation of a test screen design", 멀티미디어 언어교육, 10(1), pp.62~80 (2007), 한국멀티미디어언어교육학회
- 김정태, "말하기 평가 시행 도구 기술과 문항 유형간의 관계성 연구", 멀티미디어 언어교육, 12(1), pp.177~202 (2009), 한국멀티미디어언어교육학회
- Lynch, K. B., "Using G-theory and Many-facet Rasch measurement in the development of performance assessments of the ESL speaking skills of immigrants", Language Testing, 15(2), pp.158~180 (1998)
- Messick, S., "Validity, In Educational measurement", pp.13~103 (1989), Macmillan
- EFL listening and speaking, "Teaching ESL", pp.2009, Nation, I. S. P.
- Norris, J. M., "Examinee abilities and task difficulty in task-based second language performance assessment", Language Testing, 19(4), pp.395~418 (2002)
- Popham, W. J., "Criterion-referenced measurement" (1978), Prentice-Hall
- Robinson, P., "Task complexity, task difficulty, and task production: exploring interactions in a componential framework", Applied Linguistics, 22(1), pp.27~57 (2001)
- Skehan, P., "A framework for the implementation of task-based theory", Applied Linguistics, 17, pp.38~62 (1996)
- Skehan, P., "A cognitive approach to language learning" (1998), Oxford University Press
- Taylor, C., "Examining the relationship between computer familiarity and performance on computer-based language tasks", Language Learning, 49, pp.219~274 (1999)
- Weir, C., "Language testing and validation: An evidence-based approach" (2005), Palgrave Macmillan
- Wigglesworth, G., "An investigation of planning time and proficiency level on oral test discourse", Language Testing, 14, pp.85~106 (1997)