국가수준 학업성취도 평가의 시행 매체 변화가 학업성취도에 미치는 영향
A study on mode effects of National Assessment of Educational Achievement(NAEA)
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
본 연구는 고등학교 2학년 학생을 대상으로 시행된 2020년 국가수준 학업성취도 평가에서 수집된지필평가(paper-based test, 이하 PBT)와 컴퓨터 기반 평가(computer-based test, 이하 CBT) 데이터를 활용하여 시행 매체에 따른 학업성취도를 교과와 문항 유형별로 비교·분석하였다. 먼저, 신뢰도 분석 결과, 국어, 수학, 영어 교과에서 두 시행방식의 안정적인 신뢰도 지수가 산출되었다. 구인동등성 확인 결과, 국어와 영어 교과에서 매체 간의 구인동등성이 의심되는 결과가 나타났다. 매체효과 확인 시 학교수준의 특성을 통제하기 위해 PBT와 CBT가 모두 시행된 학교를 대상으로 다층모형을 적용하여 분석하였고, 모형적합도가 높은 무선절편 모형의 회귀 계수를 해석하였다. 그 결과 국어 교과와 영어 교과의 서답형 문항 유형에서 분명한 매체효과가 발견되었으며 CBT의 점수가 유의하게 더 높았다. 다음으로 성별, 지역 규모, 학교성별유형과 같은 하위집단별 매체효과의 차이를 분석하였다. 그결과 수학과 영어 교과에서 남학생들이 CBT에서 여학생보다 높은 점수를 얻는 경향이 있었고, 여학생들은 매체에 영향을 받지 않는 것으로 나타났다.
Abstract
This study was conducted using paper-based test (PBT) and computer-based test (CBT) data collected from the 2020 national level academic achievement evaluation for high school sophomores. First, according to the reliability analysis results, stable reliability indices were calculated for the two implementation methods in Korean, math, and English subjects. As a result of examining construct equivalence, it was found that the construct equivalence between the two modes in Korean and English subjects was suspected. In order to control the school-level characteristics, a multi-level model was applied to the schools where both PBT and CBT were implemented, and the regression coefficients of the random intercept model showing greater model-fit were interpreted from the analysis results. As a result, a clear mode effect was found in the type of construct item in Korean and English subjects, and the CBT score was significantly higher. Next, differences in mode effects for each subgroup such as gender, regional size, and school gender were analyzed. As a result, it was found that male students tend to score favorable on CBT in mathematics and English subjects, and female students' achievement was not affected by the change in testing mode.
참고문헌 (14)
- 교육부, "한 아이도 놓치지 않고 기초학력 책임진다", 교육부, ~ (2019)
- 교육부, "2019년 국가수준 학업성취도 평가 결과 발표", 교육부, ~ (2019)
- 국가수준 학업성취도 평가의 컴퓨터 기반 평가 체제 구축 방안, "구남욱", 한국교육과정평가원, ~, pp.연구보고 RRE 2021-1 (2021)
- 국제 컴퓨터·정보 소양 연구: ICILS 2018 결과 분석, "박상욱", 한국교육과정평가원, ~, pp.연구보고 RRE 2019-9 (2019)
- Canz, T., "Presentation-mode effects in large-scale writing assessments", Assessing Writing, 45, pp.100470~ (2020)
- Fishbein B, "The TIMSS 2019 item equivalence study: Examining mode effects for computer-based assessment and implications for measuring trends", Large-scale Assessments in Education, 6(11) (2018)
- Jerrim, J., "PISA 2015: How big is the ‘mode effect’ and what has been done about it?", Oxford Review of Education, 44(4), pp.476~493 (2018)
- "2017 NAEP Transition to Digitally Based Assessments in Mathematics and Reading at Grades 4 and 8: Mode Evaluation Study" (2020)[URL]
- "Mode Comparability Study Based on Spring 2015 Operational Test Data" (2016)[URL]
- Proctor, T. P., "Comparability of performance on the SAT Suite of Assessments across pencil‐and‐paper and computer‐based modes of administration", College Board, ~ (2019)
- "Spring 2015 Digital Devices Comparability Research Study" (2016)[URL]
- Three Studies of Comparability between Paper-Based and Computer-Based Testing for the ACT, "Steedle, J.", ACT, Inc, ~ (2020)
- Equally Prepared for Life?: How 15-Year-Old Boys and Girls Perform in School, "Organisation for Economic Cooperation and Development", OECD, ~ (2009)