A Corpus-based Study on Content Validity of Proofreading Tasks in Gaokao
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This study investigated the content validity of proofreading tasks in newly reformed high-stake Gaokao English tests in China from 2016 to 2021. The tests specifications and Readability analyzer were jointly used to examine the topics, genres and difficulty level of the input of Gaokao proofreading tasks, Chinese Learner English Corpus (CLEC) was employed to check the appropriateness of test items, and iWeb corpus was used specifically to observe the representativeness of collocational errors designed in proofreading tasks. With above examinations, the content validity of Gaokao proofreading tasks was considered not very high in terms of possibly inadequate coverage of genres and topics in specifications, and the tested errors’ relatively low correlation with the error types categorized in CLEC st2 group (with Pearson correlation r = .353). However, the difficulty level of task input was suitable for students’ level (with all Readability Ease scores > 60) and collocational errors in proofreading tasks were revealed to be highly representative since their frequencies in iWeb corpus were significantly fewer than those of expected forms (with Chi-square x² = 12,619,260.6). Some implications on modification of corpus tools for their better adaptation to language testing and multimedia-based English teaching were further proposed.
Abstract
This study investigated the content validity of proofreading tasks in newly reformed high-stake Gaokao English tests in China from 2016 to 2021. The tests specifications and Readability analyzer were jointly used to examine the topics, genres and difficulty level of the input of Gaokao proofreading tasks, Chinese Learner English Corpus (CLEC) was employed to check the appropriateness of test items, and iWeb corpus was used specifically to observe the representativeness of collocational errors designed in proofreading tasks. With above examinations, the content validity of Gaokao proofreading tasks was considered not very high in terms of possibly inadequate coverage of genres and topics in specifications, and the tested errors’ relatively low correlation with the error types categorized in CLEC st2 group (with Pearson correlation r = .353). However, the difficulty level of task input was suitable for students’ level (with all Readability Ease scores > 60) and collocational errors in proofreading tasks were revealed to be highly representative since their frequencies in iWeb corpus were significantly fewer than those of expected forms (with Chi-square x² = 12,619,260.6). Some implications on modification of corpus tools for their better adaptation to language testing and multimedia-based English teaching were further proposed.
참고문헌 (41)
- Alderson, J., "Language test construction and evaluation" (1995), Cambridge University Press
- Alderson, J., "Does washback exist?", Applied Linguistics, 14(2), pp.115~129 (1993)[DOI]
- Bachman, L. F., "Fundamental considerations in language testing" (1990), Oxford University Press
- Bahns, J., "Lexical collocations: A contrastive view", ELT Journal, 47(1), pp.56~63 (1993)[DOI]
- Belz, J., "Learner corpus analysis and the development of L2 pragmatic competence in networked inter-cultural language study: The case of German modal particles", Canadian Modern Language Review, 62(1), pp.17~48 (2005)[DOI]
- Best, R., "Differential competencies contributing to children's comprehension of narrative and expository texts", Reading Psychology, 29(2), pp.137~164 (2008)[DOI]
- Biber, D., "University language: A corpus-based study of spoken and written registers" (2006), John Benjamins
- Choi, I-C., "Comparability of a paper-based language test and a computer-based language test", Language Testing, 20(3), pp.295~320 (2003)[DOI]
- Coté, N., "Students making sense of informational text: Relations between processing and representation", Discourse Processes, 25(1), pp.1~53 (1998)[DOI]
- Courtis, J. K., "An investigation into annual report readability and corporate risk-return relationships", Accounting & Business Research, 16(64), pp.285~294 (1986)[DOI]
- Cushing, S. T., "Corpus linguistics in language testing research", Language Testing, 34(4), pp.441~449 (2017)[DOI]
- Crosthwaite, P. R., "Definite article bridging relations in L2: A learner corpus study", Corpus Linguistics and Linguistic Theory, 15(2), pp.297~319 (2019)[DOI]
- Mark Davies, "The advantages and challenges of “big data”: Insights from the 14 billion word iWeb corpus", 언어연구, 36(1), pp.1~34 (2019), 언어정보연구소[DOI]
- Egbert, J., "Corpus linguistics and language testing: Navigating uncharted waters", Language Testing, 34(4), pp.555~564 (2017)[DOI]
- Farley, A., "Comparison of Chinese Gaokao and Western university undergraduate admission criteria: Australian ATAR as an example", Higher Education Research & Development, 39(3), pp.470~484 (2019)[DOI]
- Futagi, Y., "A computational approach to detecting collocation errors in the writing of non-native speakers of English", Computer Assisted Language Learning, 21(4), pp.353~367 (2008)[DOI]
- Flesch, R., "A new readability yardstick", Journal of Applied Psychology, 32(3), pp.221~233 (1948)[DOI]
- Gui. S. C., "Chinese learner English corpus" (2002), Shanghai Foreign Language Education Press
- Huang, P., "Using collocation clusters to detect and correct English L2 learners’ collocation errors", Computer Assisted Language Learning, 34(3), pp.270~296 (2021)[DOI]
- Hughes, A., "Testing for language teachers" (1989), Cambridge University Press
- Li, F. L., "Analyzation on content validity of proofreading in NMET", Journal of Heilongjiang Vocational Institute of Ecological Engineering, 31(1), pp.138~143 (2018)
- Li Jiang, "Comparing Cross-disciplinary Test Takers’ Performance between Internet-based and Paper-based Diagnostic Writing Tests in EFL College English Classrooms", 외국학연구, (57), pp.121~148 (2021), 외국학연구소[DOI]
- Li, X., "The science and art of language testing" (2001), Hunan Education Press
- Liu, C., "Analysis of content validity of proofreading in Gaokao 2019", Journal of Changchun Education Institute, 35(9), pp.7~10 (2019)
- Liu, H. Z., "A corpus-based study into content validity of TEM-8 proofreading task", master, pp.2010
- MacKenzie, H., "Class matters: 12-month-olds’ word-object associations privilege content over function words", Developmental Science, 15(6), pp.753~761 (2012)[DOI]
- Oakland, T., "Language, reading, and readability formulas: Implications for developing and adapting tests", International Journal of Testing, 4(3), pp.239~252 (2004)[DOI]
- Pan, M., "Embedding corpora into the content validation of the grammar test of the National Matriculation English Test (NMET) in China", Language Assessment Quarterly, 14(2), pp.120~139 (2017)[DOI]
- Park, K., "Corpora and language assessment: The state of the art", Language Assessment Quarterly, 11(1), pp.27~44 (2014)[DOI]
- Peng, L., "Rethinking the way out for college English teaching—After China’s reform in national college entrance exam in English", Journal of Language Teaching and Research, 5(6), pp.1393~1398 (2014)[DOI]
- Qian, D. D., "Researching English language assessment in China: Focusing on high-stakes testing", Language Assessment Quarterly, 14(2), pp.97~100 (2017)[DOI]
- Schmitt, N., "Vocabulary in language teaching" (2000), Cambridge University Press
- Sun, S. J., "Measuring translation difficulty: Theoretical and methodological considerations", Across Languages and Cultures, 16(1), pp.29~54 (2015)[DOI]
- Taguchi, N., "Pragmatic development as a dynamic, complex process: General patterns and case histories", The Modern Language Journal, 95(4), pp.605~627 (2011)[DOI]
- Takimoto, M., "A corpus-based analysis of hedges and boosters in English academic articles", Indonesian Journal of Applied Linguistics, 5(1), pp.95~105 (2015)[DOI]
- Wible, D., "Bootstrapping in a language learning environment", Journal of Computer Assisted Learning, 19(1), pp.90~102 (2003)[DOI]
- Wood, R., "Assessment and testing: A survey of research" (1991), Cambridge University Press
- Xi, X., "What does corpus linguistics have to offer to language assessment?", Language Testing, 34(4), pp.565~577 (2017)[DOI]
- Xu, L., "Reliability and validity of WDCT in testing interlanguage pragmatic competence for EFL learners", Journal of Language Teaching and Research, 6(6), pp.1206~1215 (2015)[DOI]
- Zheng, L. X., "The corpus-based research into content validity of NETEM cloze test", Journal of Ningbo Institute of Education, 17(6), pp.47~50 (2015)
- Zhou, C. Y., "Analysis of English proofreading tasks from language testing perspective", Higher Education in Building Material Science, 19(2), pp.80~83 (2000)
