본문 바로가기
KERD
논문KCI 등재

한국어 학습자 쓰기 자동채점을 위한 어휘 다양도 지표의 자질 활용 가능성 탐색

Exploring the Possibility of Using the Feature of Lexical Diversity Indices for Automated Essay Evaluation of Korean Language Learners’ Writing

0

조회

0

다운로드

3

인용

0

참고문헌

키워드

초록

This study aimed to validate the validity of scoring qualities for evaluating vocabulary proficiency in automatic grading of Korean learners' writing and to analyze statistically how lexical diversity affects the evaluation scores of human evaluators. A total of 399 compositions written by Korean learners selected from the National Institute of Korean Language corpus were evaluated by Korean teachers, and vocabulary diversity was measured through 17 vocabulary diversity indices in the target compositions, which were used as scoring qualities. Finally, through correlation analysis and regression analysis, we explored whether the resulting vocabulary diversity values could be utilized as scoring qualities in automatic grading of Korean learners' writing. As a result, among the 17 vocabulary diversity indices excluding TTR, 16 indices all showed significant correlations with scores of 'linguistic diversity,' among which NDW showed a strong correlation. Furthermore, RTTR and CTTR showed moderate correlations. Also, the results of simple regression analysis for each index showed that NDW had the highest explanatory power at about 37%, followed by RTTR and CTTR at approximately 30%. This study examined scoring qualities for Korean automatic grading, verifying the applicability of qualities for evaluating vocabulary diversity in Korean learners' writing. Looking ahead, it is expected that research on various scoring qualities related to Korean learners' language use will continue, leading to comprehensive reviews of scoring qualities in automatic evaluation of Korean learners' writing.

Abstract

This study aimed to validate the validity of scoring qualities for evaluating vocabulary proficiency in automatic grading of Korean learners' writing and to analyze statistically how lexical diversity affects the evaluation scores of human evaluators. A total of 399 compositions written by Korean learners selected from the National Institute of Korean Language corpus were evaluated by Korean teachers, and vocabulary diversity was measured through 17 vocabulary diversity indices in the target compositions, which were used as scoring qualities. Finally, through correlation analysis and regression analysis, we explored whether the resulting vocabulary diversity values could be utilized as scoring qualities in automatic grading of Korean learners' writing. As a result, among the 17 vocabulary diversity indices excluding TTR, 16 indices all showed significant correlations with scores of 'linguistic diversity,' among which NDW showed a strong correlation. Furthermore, RTTR and CTTR showed moderate correlations. Also, the results of simple regression analysis for each index showed that NDW had the highest explanatory power at about 37%, followed by RTTR and CTTR at approximately 30%. This study examined scoring qualities for Korean automatic grading, verifying the applicability of qualities for evaluating vocabulary diversity in Korean learners' writing. Looking ahead, it is expected that research on various scoring qualities related to Korean learners' language use will continue, leading to comprehensive reviews of scoring qualities in automatic evaluation of Korean learners' writing.

참고문헌 (52)

  1. 강정희, "한국어 학습자의 어휘력 발달과 어휘 사용 양상 연구-쓰기 텍스트에 나타난 어휘 측정을 중심으로-", 이중언어학, 71, pp.37~64 (2018)
  2. 김승주, "채점 자질 설계를 통한 지도 학습 기반 작문 자동 채점의 타당도 확보 방안 탐색", 청람어문교육, 69, pp.265~295 (2019)
  3. 남미정, "한국어 쓰기 자동 채점을 위한 언어 자질 탐색 연구", 박사, pp.2023
  4. 남미정, "한국어 쓰기 자동 채점을 위한 언어 자질 탐색 연구", 언어사실과 관점, 57, pp.9~32 (2022)
  5. 남주연, "한국어 학습자의 문어 어휘 다양성 연구", 한국어 의미학, 55, pp.1~28 (2017)
  6. 박정은, "한국어 고급 학습자의 작문에 나타난 어휘의 다양성", 한국어교육, 25, pp.1~32 (2014)
  7. 배도용, "한국어 학습자의 쓰기에 나타난 어휘 다양도 및 어휘 밀도 연구", 언어과학, 19, pp.99~117 (2012)
  8. 서신웨, "한국어 중·고급 학습자의 문어 산출에 나타난 복잡성과 정확성 연구 : 서사와 논증 장르를 중심으로", 박사, pp.2022
  9. 안경화, "중간언어 어휘론 연구의 과제와 전망", 이중언어학, 23, pp.167~186 (2003)
  10. 원미진, "한국어 학습자의 쓰기에 나타난 어휘 풍요도 연구-숙달도 측정 도구로써 어휘 풍요도 측정 가능성을 중심으로-", 어문론총, 71, pp.33~55 (2017)
  11. 이용상, "한국어 쓰기 평가를 위한 자동채점의 가능성 탐색", 이중언어학, 86, pp.171~191 (2022)
  12. 이진, "한국어 학습자 쓰기 자동채점을 위한 자질 연구-내용 및 구조 관련 채점 자질을 중심으로-", 국제어문, 99, pp.433~471 (2023)
  13. 장지현, "머신 러닝 기법을 활용한 영어 에세이 자동채점 방안 연구", 박사, pp.2021
  14. 정진경, "언어권별 한국어 학습자의 어휘 풍요도 측정 연구-중국어권, 일본어권, 영어권 학습자 말뭉치의 작문 자료를 대상으로-", 국제어문, 91, pp.251~272 (2021)
  15. 조희련, "딥러닝 기반 언어모델을 이용한 한국어 학습자 쓰기 평가의 자동 점수 구간 분류-KoBERT와 KoGPT2를 중심으로", 한국언어문화학, 18, pp.217~241 (2021)
  16. 주우동, "중국인 한국어 학습자의 언어 발달 양상 연구 : 복잡성과 정확성을 중심으로", 박사, pp.2019
  17. 주월랑, "여성결혼이민자의 어휘 사용 연구-복잡성 측정을 중심으로-", 언어와 문화, 14, pp.223~248 (2018)
  18. 최숙기, "국어과 수행 평가의 평가자 신뢰도 보고 방안 탐색-고등학생 요약문 평가 결과를 중심으로", 작문연구, 14, pp.395~424 (2012)
  19. Burstein, J., "Automated essay scoring: A cross-disciplinary perspective", pp.113~121 (2003), Lawrence Erlbaum Associates
  20. Carroll, J. B., "Language and Thought" (1964), Prentice-Hall, Inc
  21. Chotlos, J. W., "IV. A statistical and comparative analysis of individual written language samples", Psychological Monographs, 56(2), pp.75~ (1944)
  22. MATTR user manual, "Covington, M. A.", ~ (2007)
  23. Covington, M. A., "Cutting the Gordian knot : The moving-average type–token ratio(MATTR)", Journal of quantitative linguistics, 17(2), pp.94~100 (2010)
  24. Daller, H., "Modelling and Assessing Vocabulary Knowledge" (2007), Cambridge University Press
  25. Dugast, D., "Sur quoi se fonde la notion d’étendue théoretique du vocabulaire", Le Français Moderne, 46, pp.25~32 (1978)
  26. Elliot, S., "Automated essay scoring: A cross-disciplinary perspective", pp.71~86 (2003), Lawrence Erlbaum Associates
  27. Guiraud, P., "Les Charactères Statistiques du Vocabulaire. Essai de méthodologie" (1954), Presses Universitaires de France
  28. Guiraud, P., "Proble`mes et me´thodes de la statistique linguistique" (1960), Presses Universitaires de France
  29. Hao, Y., "To predict L2 writing quality using lexical richness indices : An investigation of learners of Chinese as a Foreign Language", System, 118, pp.103123~ (2023)
  30. Herdan, G., "A new derivation and interpretation of Yule's ‘Characteristic’K", Zeitschrift für angewandte Mathematik und Physik ZAMP, 6, pp.332~339 (1955)
  31. Herdan, G., "Type-Token mathematics. A textbook of mathematicallinguistics" (1960), Mouton’s Gravenhage
  32. Herdan, G., "Quantitative linguistics" (1964), Butterworths
  33. Johnson, W., "Studies in language behavior : I. A program of research", Psychological Monographs, 56, pp.1~15 (1944)
  34. Landauer, T. K., "Automatic essay assessment", Assessment in education: Principles, policy & practice, 10(3), pp.295~308 (2003)
  35. Landis, J. R., "The measurement of observer agreement for categorical data", biometrics, pp.159~174 (1977)
  36. Leacock, C., "C-rater : Automated scoring of short-answer questions", Computers and the Humanities, 37, pp.389~405 (2003)
  37. Lee, J. H., "A BERT-Based Automatic Scoring Model of Korean Language Learners' Essay", Journal of Information Processing Systems, 18(2), pp.282~291 (2022)
  38. Lim, K., "Neural automated writing evaluation for Korean L2 writing", Natural Language Engineering, 29(5), pp.1341~1363 (2023)
  39. Maas, H. D., "Zusammenhang zwischen Wortschatzumfang und Länge eines Textes", Zeitschrift für Literaturwissenschaft und Linguisꠓtik, 8, pp.73~79 (1972)
  40. Malvern, D. D., "Lexical diversity and language development: Quantification and assessment" (2004), Palgrave Macmillan
  41. McCarthy, P. M., "An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (MTLD)", PhD, pp.2005
  42. McCarthy, P. M., "A theoretical and empirical evaluation of vocd", Language Testing, 24, pp.459~488 (2007)
  43. McCarthy, P. M., "MTLD, vocd-D, and HD-D : A validation study of sophisticated approaches to lexical diversity assessment", Behavior Research Methods, 42(2), pp.381~392 (2010)
  44. McNamara, D. S., "Linguistic features of writing quality", Written Communication, 27, pp.57~86 (2010)
  45. Miller, J., "Research on child language disorders: A decade of progress", pp.211~220 (1991), Pro-Ed
  46. Read, J., "어휘 평가" (2015), 글로벌콘텐츠
  47. Rudner, L. M., "Automated essay scoring using Bayes' theorem", The Journal of Technology, Learning and Assessment, 1(2), pp.1~22 (2002)
  48. Simpson, E. H., "Measurement of diversity", nature, 163(4148), pp.688~688 (1949)
  49. Šišková, Z., "Lexical richness in EFL students’ narratives", Language Studies Working Papers, 4(1), pp.26~36 (2012)
  50. Templin, M., "Certain language skills in children" (1957), University of Minnesota Press
  51. Tweedie, F. J., "How variable may a constant be? Measures of lexical richness in perspective", Computers & the Huꠓmanities, 32, pp.323~352 (1998)
  52. Yule, G. U., "The statistical study of literary vocabulary" (1944), Cambridge University Press

관련 논문AI추천

이 간행물의 다른 논문 (7건)