자동채점 기반 영어 말하기 시험 현황 분석
Analysis of English automated speaking scoring tests
0
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
This study investigated the feasibility of adopting an automatic scoring system (ASS) in a domestic English-speaking education context. Scope, test items, assessment criteria, scoring methods, and reporting strategies of six overseas English-speaking tests utilizing ASSs were examined. Moreover, a comparative analysis was conducted to identify disparities between ASS-based and non-ASS-based speaking tests. Findings were: 1) some ASS-based tests utilized ASS technology throughout the assessment, while others adopted a hybrid scoring system involving human raters; 2) compared to non-ASS-based tests, ASS-based tests used more test items targeting low-level skills such as sound and forms but fewer test items targeting conversation and discourse level skills; 3) pronunciation, fluency, and vocabulary were widely employed as evaluation criteria with sparse use of organization, content, and task completion in most ASS-based tests; 4) differences were minimal in assessment criteria application and score calculation between ASS-based and non-ASS-based tests; and 5) some ASS-based tests provided criteria-specific results and feedback with total scores and proficiency levels.
Abstract
This study investigated the feasibility of adopting an automatic scoring system (ASS) in a domestic English-speaking education context. Scope, test items, assessment criteria, scoring methods, and reporting strategies of six overseas English-speaking tests utilizing ASSs were examined. Moreover, a comparative analysis was conducted to identify disparities between ASS-based and non-ASS-based speaking tests. Findings were: 1) some ASS-based tests utilized ASS technology throughout the assessment, while others adopted a hybrid scoring system involving human raters; 2) compared to non-ASS-based tests, ASS-based tests used more test items targeting low-level skills such as sound and forms but fewer test items targeting conversation and discourse level skills; 3) pronunciation, fluency, and vocabulary were widely employed as evaluation criteria with sparse use of organization, content, and task completion in most ASS-based tests; 4) differences were minimal in assessment criteria application and score calculation between ASS-based and non-ASS-based tests; and 5) some ASS-based tests provided criteria-specific results and feedback with total scores and proficiency levels.
참고문헌 (18)
- Bachman, L. F., "Language testing reconsidered", pp.41~71 (2007), University of Ottawa Press
- Carr, N., "Designing and analyzing language tests" (2011), Oxford University Press
- 채정관, "국가영어능력평가시험(NEAT) 정책의 실패 원인에 대한 시차적 접근", 교육정치학연구, 22(3), pp.1~32 (2015), 한국교육정치학회
- "Automated scoring of nonnative speech using the SpeechRater v. 5.0 Engine. (ETS Research Report No. RR-18-10)" (2018)[URL]
- 최원경, "AI 챗봇을 활용한 초등영어 과정중심 말하기 평가: 가능성과 한계", 초등영어교육, 26(1), pp.131~152 (2020), 한국초등영어교육학회[DOI]
- 최원경, "초등학생 대상 AI 챗봇 자동발음평가 적용 사례 연구", 영어평가, 16(2), pp.167~185 (2021), 한국영어평가학회[DOI]
- 추성엽, "영어 상호작용 촉진을 위한 과업 기반 AI 챗봇 활용 및 학생 발화 분석", 초등영어교육, 25(2), pp.27~52 (2019), 한국초등영어교육학회[DOI]
- "TOEFL iBT® Test" (2023)[URL]
- Khabbazbashi, N., "Challenges in language testing around the world", pp.333~344 (2021), Springer
- 김나영, "Effects of Voice Chat on EFL Learners’ Speaking Ability according to Proficiency Levels", 멀티미디어 언어교육, 19(4), pp.63~88 (2016), 한국멀티미디어언어교육학회[DOI]
- 김성우, "인공지능기술 활용 대화형 영어 학습 앱을 적용한 초등영어 말하기 수업 분석: 학습자 몰입 및 수업에 대한 학생과 교사의 인식을 중심으로", 초등영어교육, 26(3), pp.177~202 (2020), 한국초등영어교육학회[DOI]
- Lee, M., "School English speaking performance assessment support system utilizing ICT. (ORM 2015-50-21)" (2015), Korean Institute for Curriculum and Evaluation
- Ockey, G., "New and not so new methods for assessing oral communication", Language Value, 7(10), pp.1~12 (2015)
- 송은주, "A Study on the Development of AI-based Assessment Model for English Speaking Skills in Primary School", 영어평가, 15(2), pp.53~76 (2020), 한국영어평가학회[DOI]
- Wang, X., "Automated speaking assessment: Using language technologies to score spontaneous speech", pp.138~156 (2019), Routledge
- Wang, Y., "Towards automatic assessment of spontaneous spoken English", Speech Communication, 104, pp.47~56 (2018)
- "Linguaskill building a validity argument for the speaking test" (2020)[URL]
- Xu, J., "Assessing L2 English speaking using automated scoring technology: Examining automarker reliability", Assessment in Education: Principles, Policy & Practice, 28(4), pp.411~436 (2021)