본문 바로가기
KERD
논문KCI 등재

랜덤포레스트 알고리듬을 활용한 한국어 에세이 자동채점 모델 개발 연구

A Study on the Development of Automated Korean Essay Scoring Model Using Random Forest Algorithm

3

조회

0

다운로드

7

인용

0

참고문헌

키워드

초록

In this study, Random Forest algorithm was applied using data of 402 Korean argumentative essays to develop an automatic scoring model for Korean essays. Recent studies on automatic scoring of essays are developing into studies using deep learning-based algorithms. However when automatic scoring is used in the field of education, deep learning-based algorithms have limitations because the interpretation and explanation of the scoring results must be possible. Therefore, in this study, we tried to develop a model that can interpret the scoring results by applying a machine learning-based algorithm that utilizes scoring features. In the essay, various parts of morpheme-based scoring qualities were derived, and the number of sentences was used. As a result, it was possible to derive a significant level of model performance. However, this study has a limitation in that it utilized basic quantitative feature such as parts of morpheme frequency and number of sentences. In future research, more in-depth grading feature will be developed to explore ways to provide more meaningful educational feedback through automated essay scoring.

Abstract

In this study, Random Forest algorithm was applied using data of 402 Korean argumentative essays to develop an automatic scoring model for Korean essays. Recent studies on automatic scoring of essays are developing into studies using deep learning-based algorithms. However when automatic scoring is used in the field of education, deep learning-based algorithms have limitations because the interpretation and explanation of the scoring results must be possible. Therefore, in this study, we tried to develop a model that can interpret the scoring results by applying a machine learning-based algorithm that utilizes scoring features. In the essay, various parts of morpheme-based scoring qualities were derived, and the number of sentences was used. As a result, it was possible to derive a significant level of model performance. However, this study has a limitation in that it utilized basic quantitative feature such as parts of morpheme frequency and number of sentences. In future research, more in-depth grading feature will be developed to explore ways to provide more meaningful educational feedback through automated essay scoring.

참고문헌 (41)

  1. Adadi, A., "Peeking inside the black-box: a survey on explainable artificial intelligence (XAI)", IEEE Access, 6, pp.52138~52160 (2018)
  2. 최숙기, "중등학생의 자료 통합적 글쓰기에 나타난 자료 글 차용하기 수행 양상 연구 - 인용과 환언을 중심으로-", 작문연구, (38), pp.201~245 (2018), 한국작문학회[DOI]
  3. 최용석, "트랜스포머와 BERT로 구현한 한국어 형태소 분석기의 성능 분석", 정보과학회논문지, 47(8), pp.730~741 (2020), 한국정보과학회[DOI]
  4. Methodological approaches to online scoring of essays, "Chung, G. K.", National Center for Research on Evaluation, Standards, and Student Testing, Graduate School of Education & Information Studies, University of California, ~, pp.Center for the Study of Evaluation Technical Report 461 (1997)
  5. Dascalu, M., "Modeling comprehension processes via automated analyses of dialogism", 39th Annual Meeting of the Cognitive Science Society (CogSci 2017) (2017), Cognitive Science Society
  6. Devlin, J., "Bert: Pre-training of deep bidirectional transformers for language understanding", ~ (2018)
  7. 하민수, "학습 도구로서의 서술형 평가 그리고 인공지능의 활용: WA3I 프로젝트 사례", 현장과학교육, 13(3), pp.271~282 (2019), 한국현장과학교육학회[DOI]
  8. Grammatical Structures Written at Three Grade Levels, "Hunt, K. W.", National Council of Teachers of English, ~, pp.Research Report No.3 (1965)
  9. Issacson, S., "CommUnication Skills and Classroom Success: Assessment Methodologoies for Language-Learning Disabled Students" (1985), College Hill Press
  10. Kumar, V., "Frontiers in education (Vol. 5)", pp.572367~ (2020), Frontiers Media SA
  11. Lantz, B., "Machine Learning with R" (2013), Packt Publishing Ltd
  12. 이용상, "한국어 쓰기 평가를 위한 자동채점의 가능성 탐색", 이중언어학, (86), pp.171~191 (2022), 이중언어학회[DOI]
  13. Lundberg, S. M., "Advances in Neural Information Processing Systems", pp.4765~4774 (2017), Curran Associates, Red Hook
  14. M. E. Peters, "Deep contextualized word representations", ~ (2018)
  15. Madnani, N., "Automated scoring: beyond natural language processing", Proceedings of the 27th International Conference on Computational Linguistics, pp.1099~1109 (2018), Association for Computational Linguistics
  16. Madnani, N., "Building better open-source tools to support fairness in automated scoring", Proceedings of the First (ACL) Workshop on Ethics in Natural Language Processing, pp.41~52 (2017), Association for Computational Linguistics
  17. Murdoch, W. J., "Definitions, methods, and applications in interpretable machine learning", Proceedings of the National Academy of Sciences, 116(44), pp.22071~22080 (2019)
  18. Murphy, K. P., "Machine Learning: A Probabilistic Perspective" (2012), MIT press
  19. Nam, M. J., "A study on exploring language qualifications for automatic scoring of Korean writing", Language Facts and Perspectives, 53, pp.9~32 (2022)
  20. Nguyen, H., "Argument mining for improving the automated scoring of persuasive essays", Proceedings of the AAAI Conference on Artificial Intelligence, pp.5892~5899 (2018)
  21. Refinement and trial application of auto-scoring program for short-answer questions for large-scale evaluation, "Noh, E. H.", Korea Institute of Curriculum and Evaluation, ~, pp.Korea Institute of Curriculum and Evaluation Research Report RRE 2013-5 (2013)
  22. Development oF Korean Self-Answer Questions Automatic Scoring Program and Verification of Practicality, "Noh, E. H.", Korea Institute of Curriculum and Evaluation, ~, pp.Research Report RRE 2014-6 (2014)
  23. A Study on Automatic Scoring of Short-Answer Questions for Large-Scale Evaluation, "Noh, E. H..", Korea Institute of Curriculum and Evaluation, ~, pp.Research Report RRE 2012-6 (2012)
  24. Advancement and Application of Automatic Scoring Program for Korean Sentence-Level Written Answer Questions, "Noh, E. H.", Korea Institute of Curriculum and Evaluation Research, ~, pp.Report RRE 2016-11 (2016)
  25. Development and Application of Automatic Scoring Program for Korean Sentence-level Self-answer Questions, "Noh, E. H.", Korea Institute of Curriculum and Evaluation, ~, pp.Research Report RRE 2015-9 (2015)
  26. Owens, R., "Language Development: An Introduction" (2012), Pearson
  27. Page, E. B., "The imminence of grading essays by computer", Phi Delta Kappan, 48, pp.238~243 (1966)
  28. Page, E. B., "Computer grading of student prose, using modern concepts and software", Journal of Experimental Education, 62, pp.127~142 (1994)
  29. Page, E. B., "Automated Essay Scoring: A Cross-disciplinary Perspective", pp.43~54 (2003), Lawrence Erlbaum Associates, Inc
  30. Design of Automatic Scoring Method for Computer-Based Written and Essay Evaluation (Ⅰ), "Park, J. I.", Korea Institute of Curriculum and Evaluation, ~, pp.Research Report RRE 2022-6 (2022)
  31. 박강윤, "한국어 에세이 문항 자동 채점을 위한 딥러닝 알고리듬 탐색", 교육평가연구, 35(3), pp.465~488 (2022), 한국교육평가학회
  32. Radford, A., "Improving language understanding by generative pre-training", ~ (2018)
  33. Rahimi, Z., "Assessing students’ use of evidence and organization in response-to-text writing: Using natural language processing for rubricbased automated scoring", International Journal of Artificial Intelligence in Education, 27, pp.694~728 (2017)
  34. "“Why should i trust you?”: Explaining the predictions of any classifier" (2016)[URL]
  35. 신동광, "채점자질의 적용이 영어 자동채점 모델의 성능에 미치는 영향", 교원교육, 38(6), pp.73~91 (2022), 교육연구원
  36. Taghipour, K., "A neural approach to automated essay scoring", Proceedings of the 2016 conference on empirical methods in natural language processing, pp.1882~1891 (2016), Association for Computational Linguistics
  37. Vaswani, A., "Attention is all you need", Advances in Neural Information Processing Systems, 30 (2017)
  38. West-Smith, P., "Trustworthy automated essay scoring without explicit construct validity", Proceedings of the 2018 AAAI Spring Symposium Series (2018), ACM
  39. Woods, B., "Formative essay feedback using predictive scoring models", Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.2071~2080 (2017), ACM
  40. Wu, Y., "Google's neural machine translation system: Bridging the gap between human and machine translation", ~ (2016)
  41. Zhao, S., "A memory-augmented neural model for automated grading", Proceedings of the Fourth ACM Conference on Learning, pp.189~192 (2017)

관련 논문AI추천

이 간행물의 다른 논문 (3건)