영어교육에서 AI 활용의 한계는 어디쯤일까?: 자동 지문 생성 및 자동 문항 생성의 활용 가능성
What is the limit for using AI in English education?: Feasibility of using Automated Text Generation and Automated Item Generation.
0
조회
0
다운로드
17
인용
0
참고문헌
키워드
초록
This study explores the feasibility of using Automated Text Generation (ATG) and Automated Item Generation (AIG) in the field of English education. To this end, 60 pre-service teachers directly tried the two programs and participated in improving the quality of the test item through post-editing. The results suggest the following: First, among the participants who corrected the test items of the participants in this experiment, the pre-service teachers modified their test item 391 times in total. Text correction accounted for 75.19% of the modifications, and option correction accounted for 24.8%. In the program user satisfaction survey, 86% of the ATG program respondents showed a positive attitude. In addition, the fact that none of the respondents selected 'very dissatisfied' suggests that there was a high possibility of using ATG at school. On the other hand, the AIG program respondents demonstrated a relatively low level of satisfaction. Overall, 43% responded positively but 33% of the total respondents took a negative stance. Improvements for both programs collected through open-ended questions were also presented. It is, however, important to note that evaluation of whether or not a post-editing attempt was appropriate was beyond the scope of this study.
Abstract
This study explores the feasibility of using Automated Text Generation (ATG) and Automated Item Generation (AIG) in the field of English education. To this end, 60 pre-service teachers directly tried the two programs and participated in improving the quality of the test item through post-editing. The results suggest the following: First, among the participants who corrected the test items of the participants in this experiment, the pre-service teachers modified their test item 391 times in total. Text correction accounted for 75.19% of the modifications, and option correction accounted for 24.8%. In the program user satisfaction survey, 86% of the ATG program respondents showed a positive attitude. In addition, the fact that none of the respondents selected 'very dissatisfied' suggests that there was a high possibility of using ATG at school. On the other hand, the AIG program respondents demonstrated a relatively low level of satisfaction. Overall, 43% responded positively but 33% of the total respondents took a negative stance. Improvements for both programs collected through open-ended questions were also presented. It is, however, important to note that evaluation of whether or not a post-editing attempt was appropriate was beyond the scope of this study.
참고문헌 (26)
- 김재상, "인간과 인공지능 기기의 상호작용이 EFL 환경에서 초등학생들의 협업적 언어습득에 미치는 영향", 석사, pp.2017
- 권서경, "대학수학능력시험 영어영역 읽기 문항 난이도 요인 탐색: EBS 연계정책 변화를 중심으로", 영어교과교육, 21(2), pp.49~71 (2022), 한국영어교과교육학회
- 신동광, "영어 읽기평가 문항의 지문 어휘통제 과정에서 나타난 한국 예비교사들의 재기술(paraphrase) 양상 분석", 멀티미디어 언어교육, 23(3), pp.157~179 (2020), 한국멀티미디어언어교육학회[DOI]
- 신동광, "기계번역의 오류 양상과 사후수정(post-editing)이L2 쓰기 적절성에 미치는 영향", 영어교과교육, 21(1), pp.121~140 (2022), 한국영어교과교육학회
- 이용상, "원격교육 시대의 인공지능 활용 온라인 평가", 학습자중심교과교육연구, 20(14), pp.389~407 (2020), 학습자중심교과교육학회[DOI]
- Al-Yahya, M., "Ontology-based multiple choice question generation", The Scientific World Journal, 2014, pp.274949~ (2014)[DOI]
- Bejar, I., "Item generation for test development", pp.199~217 (2002), Erlbaum
- Bormuth, J., "On a theory of achievement test items" (1969), University of Chicago Press
- Brown, T. B., "Language models are few-shot learners. arXiv preprint", https:, arxiv.org, pp.abs, 2005.14165[DOI]
- Drasgow, F., "Educational measurement", pp.471~516 (2006), American Council on Education
- "OpenAI GPT-3, the most powerful language model:An overview" (2022)[URL]
- Gierl, M. J., "Automatic item generation, theory and practice" (2012), Routledge Chapman & Hall
- Heilman, M., "Good question! Statistical ranking for question generation", Proceedings of Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp.609~617 (2010), Association for Computational Linguistics
- Automatic novel writing:A status report, "Klein, S.", University of Wisconsin, Computer Science Department, ~, pp.Technical Report 186 (1973)
- Lebowitz, M., "Creating a story-telling universe", Proceedings of the Eighth International Joint Conference on Artificial Intelligence, pp.63~65 (1983), Morgan Kaufmann Inc
- Liu, M., "G-Asks : An intelligent automatic question generation system for academic writing support", Dialogue and Discourse, 3(2), pp.101~124 (2012)[DOI]
- (2018)[URL]
- Meehan, J. R., "TALE-SPIN: An interactive program that writes stories", Proceedings of the Fifth International Joint Conference on Artificial Intelligence, pp.91~98 (1977), Morgan Kaufmann Inc
- Millman, J., "Computer-assisted writing of achievement test items : toward a future technology", Journal of Educational Measurement, 26(2), pp.177~190 (1989)[DOI]
- Ras, E., "A hybrid engineering process for semi-automatic item generation", Paper presented at the Communications in Computer and Information Science (2017)
- Turner, S. R., "Minstrel: A computer model of creativity and storytelling", doctor, pp.1993
- Vinu, E. V., "A novel approach to generate MCQs from domain ontology : Considering DL semantics and open-world assumption", Journal of Web Semantics, 34, pp.40~54 (2015)[DOI]
- von Davier, M., "Automated item generation with Recurrent Neural Networks", Psychometrika, 83(4), pp.847~857 (2018)[DOI]
- Wachob, P., "Methods and materials for motivation and learner autonomy", Reflections on English Language Teaching, 5(1), pp.93~122 (2006)
- (2020)[URL]
- Zeng, V., "A survey on Machine Reading Comprehension : Tasks, evaluation metrics and benchmark datasets", Applied Sciences, 10, pp.7640~ (2020)[DOI]