문법성과 어휘 응집성 기반의 영어 작문 평가 시스템
An English Essay Scoring System Based on Grammaticality and Lexical Cohesion
1
조회
0
다운로드
8
인용
0
참고문헌
키워드
초록
본 논문에서 우리는 문장의 문법성과 텍스트의 어휘 응집성 측정을 위주로 하는 영어 작문 자동 평가 시스템을 소개하려고 한다. 문법 검사를 위해서는 링크 파서를 사용하고 어휘 연쇄를 측정하기 위해서는 로제 시소러스를 사용한다. 자동 평가 시스템의 채점 신뢰도를 측정하기 위해서 자동 채점과 수동 채점의 결과를 통계적으로 비교한다. 카파 통계와 다국면 Rasch 모형에 따른 분석 결과 자동 채점은 수동 채점과 유사성이 크며 수동 채점과 비교해서 신뢰성에 특별한 문제가 없다는 결론을 내리게 된다. 본 연구의 가장 큰 의의는 다양한 종류의 기술과 도구를 바탕으로 신뢰할 만한 수준의 영작문 자동 평가 시스템을 개발했다는 것이다. 평가 대상이 문장 단위를 넘어 선 텍스트 단위이며, 단어나 문법 등의 형식적 측면만 검사하는 것이 아니라 내용적 측면도 평가한다.
Abstract
In this paper, we introduce an automatic system of scoring English essays. The system is comprised of three main components: a spelling checker, a grammar checker and a lexical cohesion checker. We have used such resources as WordNet, Link Grammar/Parser and Roget's thesaurus for these components. The usefulness of an automatic scoring system depends on its reliability. To measure reliability, we compared the results of automatic scoring with those of manual scoring, on the basis of the Kappa statistics and the Multi-facet Rasch Model. The statistical data obtained from the comparison showed that the scoring system is as reliable as professional human graders. This system deals with textual units rather than sentential units and checks not only formal properties of a text but also its contents.
참고문헌 (26)
- 김영택, "자연언어 처리" (2001), 생능출판사
- 김지은, "중학생 영작문 실력 향상을 위한 자동 문법 채점 시스템 구축", 한국콘텐츠학회 논문지, 7(5), pp.36~46 (2007)
- 시사영어사, "TOEIC Writing Test 공식 문제집" (2006), 시사영어사
- 이영식, "영어작문 평가의 채점 신뢰도에 대한 분석", 영어교육, 53(1), pp.179~200 (1998)
- 지은림, "many-facet Rasch 모형을 적용한 대입 논술고사 채점의 객관성 연구", 교육평가연구, 9(2), pp.5~22 (1996)
- 진경애, "영작문 자동 채점 시스템 개발 연구", 영어어문교육, 13(1), pp.235~259 (2007)
- Barzilay, R, "Using Lexical Chains for Text Summarization, Proceedings of the ACL Intelligent Scalable Text Summarization Workshop", ~ (1997)
- Tom Brehony, "Francophone Stylistic Grammar Checking (FSGC) Using Link Grammars", Computer Assisted Language Learning, 7(3), pp.257~269 (1994)
- Burstein, J., "Automated Essay Scoring for Nonnative English Speakers, Proceedings of the ACL99 Workshop on Computer-mediated Language Assessment and Evaluation of Natural Language Processing", ~ (1999)
- Engelhard, G, "The Measurement of Writing Ability with a Many-faceted Rasch Model", Applied Measurement in Education, 5(3), pp.171~191 (1992)
- Jarmasz, M, "Not as Easy as It Seems: Automating the Construction of Lexical Chains Using Roget's Thesaurus, Proceedings of the 16th Canadian Conference on Artificial Intelligence", ~ (2003)
- Kakkonen, T, "Automatic Assessment of the Content of Essays Based on Course Materials, Proceedings of the 2nd International Conference on Information Technology: Research and Education", ~ (2004)
- Landauer, T., "Automatic Scoring and Annotation of Essays with the Intelligent Essay Assessor, In Mark D. Shermis and Jill C. Burstein, editors, Automated Essay Scoring: A Cross-Disciplinary Perspective. Lawrence Erlbaum", ~ (2003)
- Landis, J, "The Measurement of Observer Agreement for Categorical Data", Biometrics, 33(1), pp.159~174 (1977)
- Londsale, D, "Automatic Rating of ESL essays, Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing Volume 2", ~ (2003)
- McCallum,A., "Bow: A Toolkit for Statistical Language Modeling, Text Retrieval, Classification and Clustering, http:", mccallum, pp.bow[DOI]
- McCallum, A, "A Comparison of Event Models for Naive Bayes Text Classification, Proceedings of the AAAI-98 Workshop on Learning for Text Classification", ~ (1998)
- Medelyan,O., "Computing Lexical Chains with Graph Clustering, Proceedings of the ACL 2007 Student Research Workshop", ~ (2007)
- Miltsakaki, E, "Evaluation of Text Coherence for Electronic Essay Scoring Systems", Natural Language Engineering, 10(1), pp.25~55 (2004)
- Morris, J, "Lexical Cohesion Computed by Thesaural Relations as an Indicator of the Structure of Text", Computational Linguistics, 17(1), pp.21~48 (1991)
- Page, E, "The Computer Moves into Essay Grading: Updating the Ancient Test", Phi Delta Kappa, 76, pp.561~565 (1995)
- Park, J. C., "An English Grammar Checker as a Writing Aid for Students of English as a Second Language, Proceedings of the Conference of Applied Natural Language Processing (ANLP)", ~ (1997)
- Rose, C., "A Hybrid Text Classification Approach for Analysis of Student Essays, Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing Volume 2", ~ (2003)
- Sleator, D, "Parsing English with a Link Grammar, Carnegie Mellon University Computer Science Technical Report CMU-CS-91-196", ~ (1991)
- Sutcliffe, R, "Using the Link Parser of Sleator and Temperley to Analyse a Software Manual Corpus, In R. Sutcliffe, H. Koch and A. McElligott, eds., Industrial Parsing of Software Manuals. Rodopi B.V", ~ (1994)
- Zhou, G, "Named Entity Recognition Using an HMM-based Chunk Tagger, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics", ~ (2002)