평가자 일관성에 따른 설명문 평가 예시문 선정의 차이 연구
A Study on the differences of the anchor papers according the rater consistence
1
조회
0
다운로드
14
인용
0
참고문헌
키워드
초록
이 연구는 예비국어교사들의 중학생 설명문 평가 자료를 분석하여 예비국어교사 평가자들을 일관성 여부에 따라서 평가 예시문을 산출하고, 집단 간 평가 예시문 선정의 차이를 조사하는 것에 목적이 있다. 그 결과 다음과 같은 결론을 얻었다. 첫째, 적합한 일관성을 보이는 예비국어교사는 국어교사들보다 더 엄격한 평가를 하였지만 예비국어교사 내에서 부적합 및 과적합 일관성을 보이는 집단은 일관성 적합 집단보다 전반적으로 더 관대한 평가를 하였다. 둘째, 일관성 적합 집단과 일관성 부적합 및 과적합 집단에서 선정된 평가 예시문 중에서 최저점에 해당하는 예시문과 최고점에 해당하는 예시문에는 큰 차이가 없지만 최저점과 최고점 사이에 해당하는 글의 경우에는 상당한 차이가 있음을 알 수 있었다. 이러한 결과를 통하여 이 연구에서는 향후 평가 예시문의 선정 시에는 평가 일관성이 적합하다고 검증된 평가자 집단의 자료만을 활용 할 것을 제안한다.
Abstract
The aim of this paper is to select anchor papers for middle school students' expository writing assessment according to the consistence of rater and analysis the differences of anchor papers. The finding which can be drawn from this analysis are these. First, the conformity consistence group show higher severity than the group that teacher of Korean in previous study. And the unconformity consistence group show higher severity than the conformity consistence group. Second, there isn't striking differences in the highest grade and lowest grade but, there is a broad distinction in the mean of the top 30%, the average of whole, and the mean of the bottom 30%. Our conclusion could be stated thus: the researcher of anchor papers selection need to look more closely at this rater consistency to overcome shortage of exist assessment that had reliability low in writing assessment.
참고문헌 (26)
- 박영민, "국어교사의 설명문 평가에 대한 모평균 추정과 평가 예시문 선정", 우리어문연구, (36), pp.293~326 (2010), 우리어문학회
- 박영민, "중학생 논설문 평가의 모평균 추정과 평가 예시문 선정", 국어교육, (131), pp.437~461 (2010), 한국어교육학회
- 박영민, "Rasch 모형을 활용한 국어교사의 쓰기 평가 특성 분석－중학생 설명문 쓰기 평가를 중심으로－", 국어교육학연구, (37), pp.367~391 (2010), 국어교육학회
- 박종임, "Rasch 모형을 활용한 국어교사의 채점 일관성 변화 양상 및 원인 분석 -중학생 서사문 채점을 중심으로-", 우리어문연구, 39(39), pp.301~335 (2011), 우리어문학회
- Brennan, R. L., "Generalizability of performance assessments", Educational Measurement : Issures and Practice, 14(4), pp.9~12 (1995)
- Culham, "Reproducible forms for the writing traits classroom" (2010), Scholatic Inc
- 10 steps to district performance assessment, "Driscoll, L. A.", Memphis City Schools, Department of Research, Standards, and Accountability, ~, pp.ERIC Document Reproduction Service No. ED406396 (1996)
- Freedman, S. W., "Influences on evaluators of expository essays: Beyond the text", Research in the Teaching of English, 15, pp.245~255 (1981)
- Hughes. A., "Testing for Language Teachers" (1989), Cambridge University Press
- Huot, B. A., "The literature of direct writing assessment: Major concerns and prevailing trends", Review of Educational Research, 60, pp.237~263 (1990)
- Huot, B. A., "The influence of holistic scoring procedures on reading and rating student essays, In Validating holistic scoring for writing assessment: Theoretical and empirical foundations", pp.206~236 (1993), Hampton Press
- Linacre, J. M., "Objectivity for judge-mediated certification examinations", Paper presented at annual meeting of the American Education Research Association (1989)
- Linacre, J. M., "Many-faceted Rasch measurement" (1989), MESA Press
- Linacre, J. M., "A User's Guide to Facets: Rasch measurement computer program(version 3.0)" (1996), MESA Press
- Lunz, M, E., "Judge consistency and severity across grading periods", Ecaluation and the Health Professions, 13, pp.425~444 (1990)
- McNamara, T, F., "Exploring rater characteristics with Rasch techniques", In Selected papers of the 13th Language Testing Research Colloquium (LTRC), pp.345~498 (1994)
- McNamara, T, F., "Item Response Theory and the validation of an ESP test for health professionals", Language Testing, 7(1), pp.52~75 (1990)
- Messick, S., "Standards of validity and the validity of standards in performance assessment", Educational Measurement : Issues and Practice, 21(2), pp.15~19 (1995)
- Morrow, K., "Communicative language testing; revolution or evolution?, In The Communicative Approach to Language Teaching", pp.143~158 (1979), Oxford University Press
- Writing proficiency examination guide2000-2001, "Nevada Department of Education", ~, pp.ERIC Document Reproduction Service No.ED447485 (2000)
- Novak, J. R., "Establishing validity for performance-based assessment: An illustration for collections of writing", Journal of Educational Research, 89, pp.220~233 (1996)
- Roid, G. H., "Patterns of writing skills derived from cluster analysis of direct-writing assessments", Applied Measurement in Education, 7, pp.159~170 (1994)
- Sharon, E. O., "The critical role of anchor paper selection in writing assessment", Applied measurement in education, 22, pp.255~271 (2009)
- Grade10 anchor set annotations from the spring 2001 Washington Assessment of StudentLearning in Writing (with) presentation guide for principals, "Washington Office of the State Superintendent of Public Instruction", ~, pp.ERiC Document Reproduction Service No. ED466812 (2001)
- Weigle, S. C., "Using FACETS to model rater training effects", Paper presented at the 16th annual Language Testing Research Colloquium (1994)
- Wolfe, E. W., "Cognitive differences in proficient and nonproficient essay scores", Written Communication, 15, pp.465~492 (1998)