Predicting Second Language Writing Proficiency in Learner Texts Using Computational Tools
2
조회
0
다운로드
8
인용
0
참고문헌
키워드
초록
This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing samples graded by expert raters. The findings of this study show that an analysis of linguistic features can be used to significantly predict human judgments of the essays for the MELAB writing tasks. Furthermore, the findings indicate the relative contribution of a range of linguistic features in MELAB essays to overall second language (L2) writing proficiency scores. For instance, linguistic features associated with text length and lexical sophistication were found to be more predictive of writing quality in MELAB than those associated with cohesion and syntactic complexity. This study has important implications for defining writing proficiency at different levels of achievement in L2 academic writing as well as improving the current MELAB rating scale and rater training practices. Directions for future research are also discussed.
Abstract
This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing samples graded by expert raters. The findings of this study show that an analysis of linguistic features can be used to significantly predict human judgments of the essays for the MELAB writing tasks. Furthermore, the findings indicate the relative contribution of a range of linguistic features in MELAB essays to overall second language (L2) writing proficiency scores. For instance, linguistic features associated with text length and lexical sophistication were found to be more predictive of writing quality in MELAB than those associated with cohesion and syntactic complexity. This study has important implications for defining writing proficiency at different levels of achievement in L2 academic writing as well as improving the current MELAB rating scale and rater training practices. Directions for future research are also discussed.
참고문헌 (35)
- Baayen, R. H., "The CELEX lexical database (CD ROM)ROM)", Linguistic Data Consortium, ~ (1993)
- Teaching writing to second language learners : Insights from theory and research, "Barkaoui, K.", 35~48, pp.TESL Reporter, 40 (2007)
- Brill, E., "Transformation based error driven learning and natural language processing : A case study in part of speech tagging", Computational Linguisti cs, 21, pp.543~565 (1995)
- "British National Corpus, version 3" (2007)[URL]
- Brown, G. D. A., "A frequency count of 190,000 words in the London Lund Corpus of English Conversation", Behavior Research Methods, Instrumentation & Computers, 16, pp.502~532 (1984)
- Brysbaert, M., "Concreteness ratings for 40 thousand generally known English word lemmas", Behavior Research Methods, 46, pp.904~911 (2014)
- Buckwalter, J. K., "Emergen t biliteracy in Chinese and English", Journal of Second Language Writing, 11, pp.269~293 (2002)
- The MELAB 2013 Report, "CaMLA", Cambridge Michigan Language Assessment, ~ (2013)
- Chenoweth, N., "Fluency in writing : Generating text in L1 and L2", Written Communication, 18, pp.80~98 (2001)
- Beyond essay length: Evaluating e rater®’s performance on TOEFL essays, "Chodorow, M.", Educational Testing Service, ~ (2004)
- Connor, U., "Linguis tic", rhetorical measures of international persuasive student writing, 1990, pp.24, 67~87
- Crossley, S. A., "Linguistic microfeatures to predict L2 writing proficiency: A case study in Automated Writing Evaluation", Journal of Writing Assessment, 7 (2014)
- Crossley, S. A., "The Tool for the A utomatic Analysis of Text C ohesion(TASCO) : Automatic a ssessment of local, global, and text cohesion", Behavior Research Methods, 48, pp.1227~1237 (2016)
- Crossley, S. A., "Using automatic scoring models to detect changes in student writing in an intellig ent tutoring system", Proceedings of the 26th International Florida Artificial Intelligence Research Society (FLAIRS) Conference, pp.208~213 (2013), The AAAI Press
- Crossley, S. A., "Predicting the proficiency level of language learners using lexical indices", Language Testing, 29, pp.243~263 (2012)
- Enright, M. K., "Complementing human judgment of essays written by English language learners with e rater ® scoring", Language Testing, 27, pp.317~334 (2010)
- Fellbaum, C., "WordNet: An electronic lexical database" (1998), MIT Press
- Friginal, E., "Revisiting multiple profiles of learner compositions : A comparison of highly rat ed NS and NNS essays", Journal of Second Language Writing, 23, pp.1~16 (2014)
- Grant, L., "Using computer tagged linguistic features to describe L2 writing differences", Journal of Second Language Writing, 9, pp.123~145 (2000)
- Guo, L., "Predicting human judgments of essay quality in both integrated and independent second language writing samples : A comparison study", Assessing Writing, 18, pp.218~238 (2013)
- Kuperman, V., "Age o f acquisition ratings for 30thousand English words", Behavior Research Methods, 44, pp.978~990 (2012)
- Kyle, K., "Automatically assessing lexical sophistication : Indices, tools, findings, and application", TESOL Quarterly, 49, pp.757~786 (2015)
- Landauer, T. K., "Introduction to latent semantic analysis", Discourse Processes, 25, pp.259~284 (1998)
- Lee, J. J., "Breaking the rules? A corpus based comparison of informal features in L1 and L2 un dergraduate student writing", System, 80, pp.143~153 (2018)
- Louwerse, M. M., "Encyclopedia of linguistics", pp.216~218 (2005), Fitzroy Dearborn
- Marcus, M., "Building a large annotated corpus of English : The Penn Treebank", Computational Linguistics, 19, pp.313~330 (1993)
- McNamara, D. S., "Automated evaluation of text and discourse with Coh Metrix" (2014), Cambridge University Press
- Messick, S., "Meaning and values in test validation: The science and ethics of assessment", Educational Researcher, 18, pp.5~11 (1989)
- Reid, J., "A computer text analysis of four cohesion devices in English discourse by nati ve and nonnative writers", Journal of Second Language Writing, 1, pp.79~107 (1992)
- Reppen, R., "Variation in elementary student language: A multi dimensional perspective", doctoral, pp.1994
- Silva, T., "Toward an understanding of the distinct nature of L2 writing : The ESL research and its implications", TESOL Quarterly, 27, pp.657~677 (1993)
- Toglia, M. P., "Handbook of semantic word norms" (1978), Erlbaum
- Weigle, S. C., "English language learners and automated scoring of essays : Critical considerations", Assessing Writing, 18, pp.85~99 (2013)
- Whitten, I. A., "Data mining: Practical machine learning tools and techniques" (2011), Elsevier
- Wilson, M., "MRC psycholinguistic database: Machine usable dictionary, version 2.00", Behavior Research Methods, Instrumentation and Computers, 20 , 6 10, ~ (1988)