본문 바로가기
KERD
논문KCI 등재

한국 대학생 영작문에 대한 영어 원어민 교사와 ChatGPT의 피드백 비교 및 학습자 인식 연구

A Comparative Study of Feedback from Native English Teachers and ChatGPT on Korean University Students’ English Composition with Student Perceptions

2

조회

0

다운로드

1

인용

0

참고문헌

키워드

주제분류영어와문학

초록

This study examined the feedback provided to Korean undergraduate students on their English writing by native English-speaking teachers and ChatGPT. This study addressed two research questions: 1) What are the similarities and differences between feedback from native English speakers and ChatGPT? 2) How do Korean undergraduate students perceive feedback from native English teachers versus ChatGPT? Thirty university students wrote English essays on a designated topic and received feedback from both two native English teachers and ChatGPT in a blind manner. Analysis of the feedback revealed differences in content and length. Native English teachers tended to emphasize negative feedback, whereas ChatGPT provided both positive and negative feedback. Additionally, ChatGPT’s feedback was generally longer, including both general and specific comments. However, it also exhibited a tendency to repeat generic templates, which could compromise reliability and accuracy of feedback. Student perceptions of the feedback from different providers were further examined through a questionnaire. The findings indicate that students found feedback from native English teachers to be more comprehensible, helpful in identifying writing issues, and effective for enhancing their writing skills.

Abstract

This study examined the feedback provided to Korean undergraduate students on their English writing by native English-speaking teachers and ChatGPT. This study addressed two research questions: 1) What are the similarities and differences between feedback from native English speakers and ChatGPT? 2) How do Korean undergraduate students perceive feedback from native English teachers versus ChatGPT? Thirty university students wrote English essays on a designated topic and received feedback from both two native English teachers and ChatGPT in a blind manner. Analysis of the feedback revealed differences in content and length. Native English teachers tended to emphasize negative feedback, whereas ChatGPT provided both positive and negative feedback. Additionally, ChatGPT’s feedback was generally longer, including both general and specific comments. However, it also exhibited a tendency to repeat generic templates, which could compromise reliability and accuracy of feedback. Student perceptions of the feedback from different providers were further examined through a questionnaire. The findings indicate that students found feedback from native English teachers to be more comprehensible, helpful in identifying writing issues, and effective for enhancing their writing skills.

참고문헌 (35)

  1. Ahn, S., "The effect of different types of online feedback on the revisions of Korean EFL students in their L2 writing", Modern Studies in English Language & Literature, 67(2), pp.151~173 (2023)[DOI]
  2. Ben-Simon, A., "Toward more substantively meaningful automated essay scoring", Journal of Technology, Learning, and Assessment, 6(1), pp.1~47 (2007)
  3. Burkhart, C., "Assisting students’ writing with computer-based concept map feedback : A validation study of the CohViz feedback system", PLoS ONE, 15(6), pp.1~20 (2020)[DOI]
  4. Choe, Y., "Exploring perceptions of Korean pre-service English teachers on using ChatGPT in the English writing process", Journal of the Korea English Education Society, 22(2), pp.243~262 (2023)[DOI]
  5. Choi, J., "Effects of different types of written corrective feedback on Korean high school students’ writing", Foreign Language Education Research, 21, pp.21~40 (2017)
  6. Choi, Y., "Tomato TOEFL iBT" (2009), Neungyulgyoyuk
  7. Cohen, Y., "Validating human and automated scoring of essays against"True"scores", Applied Measurement in Education, 31(3), pp.241~250 (2018)[DOI]
  8. Ferris, D. R., "Written corrective feedback in second language acquisition and writing studies", Language Teaching, 45(4), pp.446~459 (2012)[DOI]
  9. Fleckenstein, J., "Automated feedback and writing : A multi-level meta-analysis of effects on students’ performance", Frontiers in Artificial Intelligence, 6, pp.1~11 (2023)[DOI]
  10. Guo, K., "To resist it or to embrace it? Examining ChatGPT’s potential to support teacher feedback in EFL writing", Education and Information Technologies, 29, pp.8435~8463 (2024)[DOI]
  11. Hwang, Y., "The emergence of generative AI and PROMPT literacy : Focusing on the use of ChatGPT and DALL-E for English education", Journal of the Korea English Education Society, 22(2), pp.263~288 (2023)[DOI]
  12. Hyland, K., "Feedback on second language students’ writing", Language Teaching, 39(2), pp.83~101 (2006)
  13. Jeon, J., "A comparative study on ChatGPT and native English teachers’ feedback on college students’ English writing", New Korean Journal of English Language and Literature, 66(1), pp.129~159 (2024)[DOI]
  14. Jung, Y. -S., "A study on the students’ need analysis on the types of feedback and ways of providing feedback in the process of English writing", The Journal of Modern British & American Language & Literature, 30(1), pp.31~56 (2012)
  15. Kang, E. Y., "A meta-analysis of short-and long-term effects of written corrective feedback on L2 grammatical accuracy in writing", Korean Journal of English Language and Linguistics, 22, pp.1269~1286 (2022)[DOI]
  16. Keith, T. Z, "Automated essay scoring: A cross-disciplinary perspective", pp.140~160 (2003), Lawrence Erlbaum Associates
  17. Kim, J., "Effects of rubric-referenced self-assessment training on Korean high school students’ English writing", English Teaching, 74(3), pp.79~111 (2019)[DOI]
  18. Koh, W. -Y., "Effective applications of automated writing feedback in process-based writing instruction", English Teaching, 72(3), pp.91~118 (2017)[DOI]
  19. Latifi, S., "Automated scoring of junior and senior high essays using Coh-Metrix features : Implications for large-scale language testing", Language Testing, 38(1), pp.62~85 (2021)[DOI]
  20. Lee, H., "A study of evaluation and feedback analysis of Korean English instructors, native English instructors, and ChatGPT on college students’ English writing", The Journal of Modern British & American Language & Literature, 41(4), pp.247~270 (2023)[DOI]
  21. Lee, Y. -J., "Students’ perceptions of the automated writing evaluation feedback in writing courses", Secondary English Education, 10(4), pp.143~164 (2017)
  22. Ngo, T. T. -N., "The effectiveness of automated writing evaluation in EFL", ESL writing : A three-level meta-analysis, 2024, pp.32(2), 727~744
  23. Oh, S., "A study on the case of using ChatGPT & learners’ perception in college liberal arts writing", Korean Journal of General Education, 17(3), pp.11~23 (2023)[DOI]
  24. Park, J., "An AI-based English grammar checker vs. human raters in evaluating EFL learners’ writing", Multimedia-Assisted Language Learning, 22(1), pp.112~131 (2019)[DOI]
  25. Park, J., "Analysis of high school students’ perception on instructional feedback from humans and non-human subject : Focused on feedback from teachers, peers, and artificial intelligence", Journal of Learner-Centered Curriculum and Instruction, 21(5), pp.733~745 (2021)[DOI]
  26. Patterson, N., "Computerized writing assessment : Technology gone wrong", Voices From the Middle, 13(2), pp.56~57 (2005)
  27. Shermis, M. D., "State-of-the-art automated essay scoring : Competition, results, and future directions from a United States demonstration", Assessing Writing, 20, pp.53~76 (2014)[DOI]
  28. Shin, D., "Utilizing ChatGPT in guided writing activities", Journal of the Korea English Education Society, 22(2), pp.197~217 (2023)[DOI]
  29. Shin, D., "Exploring the potential of using ChatGPT as a content-based English learning and teaching tool", Journal of the Korea English Education Society, 22(1), pp.171~192 (2023)[DOI]
  30. Shin, Y., "The effect of automated written feedback through Grammarly : Focusing on improving language-related writing quality of English", Multimedia-Assisted Language Learning, 25(2), pp.84~102 (2022)[DOI]
  31. Son, H., "Perceptions and attitudes of English learners towards the utilization of AI-based automated writing evaluation(AWE)programs and instructor feedback", English Language & Literature Teaching, 29(3), pp.91~112 (2023)[DOI]
  32. Stevenson, M., "A critical interpretative synthesis : The integration of automated writing evaluation into classroom writing instruction", Computers and Composition, 42, pp.1~16 (2016)[DOI]
  33. Stevenson, M., "The effects of computer-generated feedback on the quality of writing", Assessing Writing, 19, pp.51~65 (2014)[DOI]
  34. Warschauer, M., "Automated writing evaluation : Defining the classroom research agenda", Language Teaching Research, 10(2), pp.157~180 (2006)[DOI]
  35. Zhai, N., "The effectiveness of automated writing evaluation on writing quality : A meta-analysis", Journal of Educational Computing Research, 61(4), pp.875~900 (2023)[DOI]

관련 논문AI추천

이 간행물의 다른 논문 (6건)