대단위 쓰기 평가 설계를 위한 평가 체제 검토 -SAT essay와 GRE Analytical Writing을 중심으로-
Examination of Assessment Systems for Large-Scale Writing Assessment Design -focused on the SAT Essay and GRE Analytical Writing-
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
대단위 쓰기 평가는 교실 쓰기 평가와 대별되어 특정 규모 이상의 학습자를 대상으로 하는 직접 평가 방식의 쓰기 평가를 말한다. 서논술형 평가에 대한 사회적 요구가 강해지는 가운데, 타당도와 신뢰도, 공정성을 확보한 대단위 쓰기 평가를 개발하기 위한 체계적 연구가 선행되어야 한다. 이 연구의 목적은 평가틀의 관점에서 대학 이상 기관의 선발 시험에서 시행한 대단위 쓰기 평가 체제를 검토하여 대단위 쓰기 평가의 체제 개발의 시사점을 도출하는 것이다. 이를 위해 미국의 SAT와 GRE의 쓰기 평가 체제와 개발 과정을 검토하여 시사점에 대해 논의하였다. 대단위 쓰기 평가의 설계를 위해 증거 중심 설계와 평가 공학적 접근의 활용 방안, 다양한 심리측정학적 접근 방법을 활용한 대단위 쓰기 평가 설계 연구 방안을 제시하였고, 공정성 확보를 위한 실질적 대안을 논의하였다.
Abstract
Large-scale writing assessments differentiate from classroom writing assessments as they are a direct assessment method targeting a significant number of learners. With the increasing societal demand for essay type assessments, systematic research must precede the development of large-scale writing assessments, ensuring validity, reliability, and fairness. This study aims to review the assessment systems of large-scale writing assessments administered in college-level selection exams, extracting implications for the development of a large-scale writing assessment framework. In pursuit of this objective, the study examined and discussed the implications of the writing assessment systems and development processes of the United States' SAT and GRE. For the design of the large-scale writing assessments, this study proposed research methods utilizing evidence-centered design, assessment engineering approaches, and various psychometric approaches. It also discussed practical alternatives to secure fairness.
참고문헌 (52)
- 교육부, "2028 수능 국·수·탐 선택과목 없이 통합 평가 학업포기 내모는 내신 9등급제, 2025부터 5등급 체제로", 2023. 10. 10.자 보도자료, ~ (2023)
- 이영진, "대단위 작문 평가를 위한 문항 개발과 채점 방법 연구", 박사, 국어교육전공, pp.2012
- 장성민, "대학수학능력시험 “국어” 작문 영역 평가의 타당성 검토 - 선다형 점수와 수행형 점수의 상관관계를 중심으로 -", 작문연구, (29), pp.163~209 (2016), 한국작문학회
- 정혜진, "국어 교과 ‘비판적·창의적 사고 역량’ 측정도구 개발 및 타당화에 대한 탐색적 연구", 교육평가연구, 33(2), pp.511~532 (2020), 한국교육평가학회
- 채정관, "국가영어능력평가시험(NEAT) 정책의 실패 원인에 대한 시차적 접근", 교육정치학연구, 22(3), pp.1~32 (2015), 한국교육정치학회
- 대학수학능력시험의 성과와 발전 방향, "한국교육과정평가원", 한국교육과정평가원, ~, pp.연구자료 ORM 2020-21-5 (2020)
- Almond, R. G., "Enhancing the design and delivery of assessment systems: A four-process architecture", Journal of Technology, Learning, and Assessment, 1(5) (2002)
- American Educational Research Association, "Standards for educational and psychological testing", American Educational Research Association, ~ (2014)
- Bejar, I. I., "Item Generation for Test Development", pp.199~217 (2002), Erlbaum
- Brentani, E., "Unidimensionality in the Rasch model : how to detect and interpret", Statistica, 67(3), pp.253~261 (2007)
- Bridgeman, B., "Comparability of essay question variants", Assessing Writing, 16(4), pp.237~255 (2011)
- College Board, "Test specifications for the redesigned SAT", pp.24~38 (2014), College Board
- "Test Specifications for the Redesigned SAT" (2015)[URL]
- College Board, "SAT® suite of assessments technical manual: Characteristics of the SAT", ~ (2017)
- College Board, "SAT® suite of assessments technical manual Appendixes", ~ (2017)
- College board, "Helping Higher Education Understand the SAT Essay", ~ (2018)
- Fairness on the SAT Suite of Assessments, "College Board", College Board, ~ (2021)
- ETS, "Introduction to the Analytical Writing measure", ETS, ~ (2013)
- Dana Gioia, "Meanwhile: When reading wanes, it's time to worry", The New York Times, ~ (2005)
- Embretson, S., "Handb. Statist. Psychometr. 26", pp.747~768 (2006)
- ETS, "Overview of the Analytical Writing Section", ETS, ~ (2010)
- ETS, "GRE® Guide to the Use of Scores", ETS, ~ (2022)
- ETS, "GRE® Guide to the Use of Scores", ETS, ~ (2023)
- Gierl, M. J., "Automatic Item Generation : Theory and Practice" (2013), Routledge
- Kim, S., "Investigating the effectiveness of equating designs for constructed response tests in large scale assessment", Journal of Educational Measurement, 47, pp.186~201 (2010)
- Performance assessment: The state of the art (SCOPE Student Performance Assessment Series), "Lane, S.", Stanford University, Stanford Center for Opportunity Policy in Education, ~ (2010)
- Linacre, J. M., "Many-faceted Rasch measurement", Doctor, pp.1989
- Luecht, R. M., "Automatic item generation: Theory and practice", pp.59~76 (2013)
- McNamara, T. F., "Language assessment as social practice : Challenges for research", Language testing, 18(4), pp.333~349 (2001)
- McNamara, T. F., "‘Interaction’ in second language performance assessment:whose performance?", Applied Linguistics, 18, pp.446~466 (1997)
- Messick, S., "Educational measurement", pp.13~103 (1989), Macmillan
- Mislevy, R. J., "Implications of evidence-centered design for educational testing", Educational measurement : issues and practice, 25(4), pp.6~20 (2006)
- A brief introduction to evidence-centered design, "Mislevy, R. J.", ETS, i~29, pp.ETS Research Report Series, 2003(1) (2003)
- Mislevy, R. J., "Focus article : On the structure of educational assessments", Measurement : Interdisciplinary research and perspectives, 1(1), pp.3~62 (2003)
- Newton, S., "Utilizing evidence-centered design to develop assessments: A high school introductory computer science course", Frontiers in Education, 6, pp.695376~ (2021), Frontiers Media SA
- Correlates of satisfaction with graduate school applicants’ performance on the GRE Writing measure, "Powers, D. E.", Educational Testing Service, ~, pp.Research Report No. RR-96-24 (1997)
- Robin, F., "2.3 Evaluating and Maintaining the Psychometric and Scoring Characteristics of the Revised Analytical Writing Measure. A Compendium of Studies", ~ (2014)
- Identifying the writing tasks important for academic success at the undergraduate and graduate levels, "Rosenfeld, M.", ETS, i~91, pp.ETS Research Report Series, 2004(2) (2004)
- Psychometric Evaluation of the New GRE® Writing Assessment, "Schaeffer, G. A.", ETS, i~53, pp.ETS Research Report Series, 2001(1) (2001)
- Sehmitt, A. P., "Differential item functioning for minority examinees on the SAT", Journal of Educational Measurement, 27(1), pp.67~81 (1990)
- Uto, M., "A multidimensional generalized many-facet Rasch model for rubric-based performance assessment", Behaviormetrika, 48(2), pp.425~457 (2021)
- Uto, M., "A generalized many-facet Rasch model and its Bayesian estimation using Hamiltonian Monte Carlo", Behaviormetrika, 47, pp.469~496 (2020)
- The Research Foundation for the GRE revised General Test: A Compendium of Studies, "Wendler, Cathy", Educational Testing Service, ~ (2014)
- Yu, C. H., "Assessing unidimensionality: A comparison of Rasch modeling, parallel analysis, and TETRAD", Practical Assessment, Research, and Evaluation, 12(1), pp.14~ (2007)
- Zenisky, A. L., "Detection of differential item functioning in large-scale state assessments : A study evaluating a two-stage approach", Educational and Psychological Measurement, 63(1), pp.51~64 (2003)
- Zhao, J. C., "Timing configuration and psychometric properties of the redesigned Analytical Writing measure of the GRE General Test", Paper presented at the annual meeting of the National Council on Measurement in Education (2006)
- Zieky, M. J., "Looking Back", pp.115~127 (2011), Springer
- "Shorter GRE® Test Coming September 2023" (2023)[URL]
- "Overview of the Analytical Writing Measure"[URL]
- "Analytical Writing―Analyze an Issue, Pool of Issue Topics" (2023)[URL]
- "Analytical Writing―Analyze an Argument, Pool of Argument Topics" (2023)[URL]
- "SAT Essay Scoring" (2023)[URL]