인공지능 딥 러닝 기법을 이용한 영어 시험의 성적 예측과 활용
Prediction of English test results using artificial neural network.
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study researches the possibility of applying a deep learning-based English assessment score prediction model in an assessment setting to predict scores of unperformed English assessments. A reading difficulty index and a corpus index were calculated as attributes connected to each test type for each English test type. A deep learning score prediction model was run to learn from the test. The model's validity was determined by comparing the predicted model's performance on a new trial with the scores obtained from the actual test. The results show a high level of prediction accuracy, and out of the 10,380 participants' scores from the TOEIC English assessment taken as a part of a university liberal arts class over three years, for 782 test-takers, there was only a 0.07- point difference in the average scores across 3 test types. Furthermore, there was only a 1.5 and 1.3 point difference between the top 30% and bottom 30% groups. It was also confirmed that the results were valid even when the individual test takers were replaced with a different group. It is expected that this prediction model of English test results can be used as an additional indicator for developing an English test. This study proposes that the valid result predictions of unperformed English assessments using the deep learning-based English assessment prediction model can be used as a new supporting indicator.
Abstract
This study researches the possibility of applying a deep learning-based English assessment score prediction model in an assessment setting to predict scores of unperformed English assessments. A reading difficulty index and a corpus index were calculated as attributes connected to each test type for each English test type. A deep learning score prediction model was run to learn from the test. The model's validity was determined by comparing the predicted model's performance on a new trial with the scores obtained from the actual test. The results show a high level of prediction accuracy, and out of the 10,380 participants' scores from the TOEIC English assessment taken as a part of a university liberal arts class over three years, for 782 test-takers, there was only a 0.07- point difference in the average scores across 3 test types. Furthermore, there was only a 1.5 and 1.3 point difference between the top 30% and bottom 30% groups. It was also confirmed that the results were valid even when the individual test takers were replaced with a different group. It is expected that this prediction model of English test results can be used as an additional indicator for developing an English test. This study proposes that the valid result predictions of unperformed English assessments using the deep learning-based English assessment prediction model can be used as a new supporting indicator.
참고문헌 (31)
- 강흥립, "한ㆍ일 간의 대입 수능 영어 비교 연구", 아시아문화연구, 6, pp.141~163 (2002)
- 김충배, "영어학 영어교육의 제문제" (2001), 한국문화사
- 남보우, "수능시험 선택과목 점수조정에 대한 측정학 및 경영과학 비교", 한국경영과학회 학술대회논문집, pp.164~168 (2010)
- 이상하, "문항 예측정답률과 문항반응이론을 이용한 원점수 분포 추정", 교육평가연구, 25(3), pp.459~477 (2012), 한국교육평가학회
- 민경찬, "수능 시험의 이해와 활용", 대학교육, 110, pp.7~15 (2001)
- 박정, "문항반응이론에 의한 2005 대학수학능력시험 영역별 선택과목 점수조정의 타당성 검증", 교육학연구, 43(2), pp.135~154 (2005), 한국교육학회
- 성태제, "타당도와 신뢰도" (2002), 학지사
- 최인철, "외국어능력 검정시험 개발 모델", 어학연구, 29(3), pp.383~425 (1993)
- Abadi, M., "Tensorflow: A system for large-scale machine learning", Paper presented at the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI-16) (2016)
- Allen, M. J., "Introduction to measurement theory", pp.1979, Brooks
- Bachman, L. F., "Fundamental considerations in language testing" (1990), Oxford University Press
- Choi, I-C., "Content and construct validation of a criterion-referenced English proficiency test", English Teaching, 48, pp.311~348 (1994)
- Cronbach, L. J., "Coefficient alpha and the internal structure of tests", Psychometrika, 16(3), pp.297~334 (1951)
- Drum, P. A., "The effects of surface structure variables on performance in reading comprehension tests", Reading Research Quarterly, 16, pp.486~514 (1981)
- Entin, E. B., "Some inter-relationships of readability, cloze and multiple choice scores on a reading comprehension test", Journal of Reading Behavior, 10(4), pp.417~436 (1978)
- Flesch, R., "How to write in plain English: A book for lawyers and consumers" (1979), Harper
- Freedle, R., "The prediction of TOEFL reading item difficulty : Implications for construct validity", Language Testing, 10(2), pp.133~170 (1993)
- Goodfellow, I., "Deep learning" (2016), MIT press
- Gunning, R., "The technique of clear writing" (1952), McGraw-Hill
- Hochreiter, S., "Long short-term memory", Neural Computation, 9(8), pp.1735~1780 (1997)
- Holland, P. W., "Classical test theory as a first-order item response theory : Application to true-score prediction from a possibly nonparallel test", Psychometrika, 68(1), pp.123~149 (2003)
- Jin, K-A, "The prediction of English item difficulty in college scholastic ability test", English Teaching, 59, pp.267~278 (2004)
- Derivation of new readability formulas for navy enlisted personnel, "Kincaid, J. P.", Navy Research Branch, ~ (1975)
- Kuder, G. F., "The theory of the estimation of test reliability", Psychometrika, 2(3), pp.151~160 (1937)
- Lee, Y., "Validation study of TOEIC based on test usefulness", doctor, pp.2020
- Lord, F. M., "Statistical theories of mental test scores" (1968), Addison-Wesley
- Rumelhart, D. E., "Learning representations by back-propagating errors", Nature, 323(6088), pp.533~536 (1986)
- Scott, M, "WordSmith Tools" (1996), Oxford University Press
- Liu, W., "An improved back-propagation neural network for the prediction of college students’ English performance", International Journal of Emerging Technologies in Learning, 14(16), pp.130~142 (2019)
- Williams, R. J., "Back-propagation: Theory, architectures and applications" (1992), Lawrence Erlbaum Associates
- Zhang, W., "Parallel distributed processing model with local space-invariant interconnections and its optical architecture", Applied Optics, 29(32), pp.4790~4797 (1990)