Efficacy of Corpus-driven Scoring Methods for Computer-based Dictation Tests
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The present research is a follow-up research of an exploratory validation study of computer-based dictation tests utilizing TTS (text-to-speech) (Choi & Park, 2015). The study attempted to explore the efficacy of different scoring methods utilizing corpus linguistic techniques. To this end, the data collected from a prior study was employed in the research. 117 Korean EFL learners participated in this research and were divided into three groups depending on educational levels as well as their proficiency levels: 37 UNI (advanced), 40 HSU (intermediate), and 40 HSL (low). Participants' responses were scored using six different methods: GS (graduated scoring), AM (appropriate meaning-based scoring), EW (exact-word scoring), WM (wordlist match scoring), WML (wordlist match scoring including lemmas), and WMS (wordlist match scoring including synonyms). Comparative analysis of the efficacy of the scoring methods revealed that the four methods were highly correlated with each other and that corpus-based wordlist matching was found to be the most effective and robust scoring method. Among the three wordlist match scoring methods producing robust results, WM was proven to be the most effective semi-automated method with high practicality, requiring the least amount of time and energy for scoring.
Abstract
The present research is a follow-up research of an exploratory validation study of computer-based dictation tests utilizing TTS (text-to-speech) (Choi & Park, 2015). The study attempted to explore the efficacy of different scoring methods utilizing corpus linguistic techniques. To this end, the data collected from a prior study was employed in the research. 117 Korean EFL learners participated in this research and were divided into three groups depending on educational levels as well as their proficiency levels: 37 UNI (advanced), 40 HSU (intermediate), and 40 HSL (low). Participants' responses were scored using six different methods: GS (graduated scoring), AM (appropriate meaning-based scoring), EW (exact-word scoring), WM (wordlist match scoring), WML (wordlist match scoring including lemmas), and WMS (wordlist match scoring including synonyms). Comparative analysis of the efficacy of the scoring methods revealed that the four methods were highly correlated with each other and that corpus-based wordlist matching was found to be the most effective and robust scoring method. Among the three wordlist match scoring methods producing robust results, WM was proven to be the most effective semi-automated method with high practicality, requiring the least amount of time and energy for scoring.
참고문헌 (24)
- Bacheller, F, "Research in language testing. Rowley", pp.66~71 (1980), Newbury House
- Bachman, L. F., "Language testing in practice" (1996), Oxford University Press
- Buck, G., "Assessing listening" (2001), Cambridge University Press
- Bormuth, J. R., "Validities of grammatical and semantic classifications of close test scores", Proceedings of the International Reading Association Conference, 10, pp.253~286 (1965)
- Bormuth, J. R., "Factor validity of cloze tests as measures of reading comprehension ability", Reading Research Quarterly, 4(3), pp.358~365 (1969)
- Choi, I-C, "Efficacy of an ICALL tutoring system & process-oriented corrective feedback", Computer-Assisted Language Learning, 29(2), pp.334~364 (2016)
- 최인철, "An Exploratory Validation Study of Computer-based Dictation Tests Utilizing TTS", 멀티미디어 언어교육, 18(4), pp.11~41 (2015), 한국멀티미디어언어교육학회[DOI]
- Coniam, D., "Interactive evaluation of listening comprehension: How the context may help", Computer-Assisted Language Learning, 11(1), pp.35~53 (1998)
- Coniam, D., "Evaluating computer-based and paper-based versions of an English-language listening test", ReCALL, 18(2), pp.193~211 (2006)
- Cziko, G. A., "Improving the psychometric, criterion referenced, and practical qualities of integrative tests", TESOL Quarterly, 16(3), pp.367~379 (1982)
- Farhady, H., "Issues in language testing research", pp.100~114 (1983), Newbury House Publishers
- Farhady, H., "Different methods of scoring dictation tests", Journal of Social Sciences, 12(1), pp.203~216 (1997)
- Fouly, K. A., "Determining the reliability, validity and scalability of graduated dictation test", Language Learning, 35(4), pp.555~566 (1985)
- Irvine, P., "Cloze, dictation, and the test of English as a foreign language", Language Learning, 24(2), pp.245~252 (1974)
- 이선주, "An analysis of the validity of the computerized dictation test and its scoring method", 멀티미디어 언어교육, 9(2), pp.174~196 (2006), 한국멀티미디어언어교육학회
- Lin, N. J., "Developing integrative language testing techniques, the graduated dictation, and the copy test", Master, pp.1981
- Norris, R. W., "Using creative dictation to manage, motivate, and activate large groups of false beginners", Fukuoka Woman's Junior College Studies, 47, pp.71~82 (1993)
- Oller, J. W., "Language tests at school: A pragmatic approach" (1979), Longman
- Oller, J. W., "A cloze test of English prepositions", TESOL Quarterly, 5(4), pp.315~326 (1971)
- Oller, J. W. Jr., "Scoring methods and difficulty levels for cloze tests of proficiency in English as a second language", Modern Language Journal, 56, pp.151~158 (1972)
- Oller, J. W., "Dictation: A test of grammar-based expectancies", ELT Journal, 30, pp.25~36 (1975)
- Oller, J. W., "Research in language testing", pp.13~23 (1980), Newbury House Publishers, Inc
- Rankin, B. P., "An evaluation of the cloze procedure as a technique for measuring reading comprehension", doctor, pp.1957
- Ruddell, B. B., "An investigation of the effect of oral and written patterns of language structure on reading comprehension", doctor, pp.1963