인지진단모형을 활용한 논증적 글쓰기 평가를 위한 Q행렬 타당화 연구
Validation study of Q-matrix for Argumentative Writing Assessment Using Cognitive Diagnostic Model
0
조회
0
다운로드
7
인용
0
참고문헌
키워드
초록
본 연구의 목적은 인지진단모형을 활용한 논증적 글쓰기 평가를 실행하기 위해 논증적 글쓰기 평가를 위해 개발된 EDD 체크리스트를 바탕으로 Q행렬을 구안하고 타당화하는 데 있다. 이를 위해 총 6인의 현직 국어 교사가 EDD 체크리스트를 사용하여 중1부터 고1 학생 총 402명의 논증적 글쓰기 과제를 대상으로 이분 점수에 의한 채점을 실시하였다. 채점을 실시한 6인의 국어 교사들은 연구진에 의해 도출된 논증적 글쓰기 과제 수행에 요구되는 인지 요소들을 EDD 평가기준별로 할당하였고, Q행렬 초안을 작성하였다. 6인 평가자가 도출한 Q행렬을 토대로 연구진은 검토를 통해 Q행렬 초안을 확정하였고, 이를 바탕으로 하여 ① MCMC 기반 알고리즘에 의한 사후 분포의 수렴 여부, ② 문항 모수치의 양호도 ③ Fusion 모형의 적합도 등 세 가지 지표를 활용하여 Q행렬 최종안을 도출하였다. Q행렬 초안의 수정 과정에서는 (1)인지요소의 통합, (2)일부 인지요소의 삭제. (3)12번 평가기준의 삭제가 검토되었으며 연구 결과를 토대로 인지 요소1(논증적 글쓰기 과제 맥락 고려 및 자료 이해), 인지요소2(자료 활용 및 내용 생성), 인지요소3(논증 구성의 타당성과 일관성), 인지요소4(반론 고려 및 논증 정교화), 인지요소5(표현의 유창성 및 정확성) 등 총 5개 인지요소를 토대로 한 Q행렬을 최종 확정하였다.
Abstract
The purpose of this study is to devise and validate the Q-matrix based on the EDD checklist in order to perform the argumentative writing assessment using the cognitive diagnostic model. Based on the Q-matrix derived by the six-person evaluator, the researchers confirmed the draft Q-matrix based on the review, and based on this, the final model of the Q-matrix was derived by examining ① whether or not the post-distribution by the MCMC-based algorithm converged, ② the quality of item parameter ③ model fit indicators of the Fusion model. In the process of revising the draft Q matrix, (1) integration of attributes, and (2) deletion of some attributes. (3) the deletion of EDD checklist criteria No. 12 was reviewed and based on the results of the study, attribute 1 (consideration of the context of argumentative writing tasks and understanding of source text), attribute 2 (use of source text and generation of content), attribute 3 (validity of argument composition and consistency), attribute 4 (consideration of counter-arguments and elaboration of arguments), and attribute 5 (fluent and correctness of expression) were finally confirmed.
참고문헌 (66)
- 김지효, "진단분류모형을 활용한 진로결정자기효능감 측정", 進路敎育硏究, 27(2), pp.23~40 (2014), 한국진로교육학회
- 김태호, "인지진단평가를 활용한 독해 능력 프로파일 분석과 맞춤형 읽기교육 연구 : 일반계 고등학생을 중심으로", 박사, pp.2019
- 김태호, "인지진단모형을 활용한 일반계 고등학생의 비문학 독해 프로파일 분석", 국어교육학연구, 52(2), pp.213~248 (2017), 국어교육학회[DOI]
- 김태호, "인지진단모형을 활용한 대학수학능력시험 국어 영역 독서 문항의 Q행렬 타당화 연구-2018학년도 대학수학능력시험 대비 6월 모의평가를 중심으로-", 청람어문교육, (63), pp.201~232 (2017), 청람어문교육학회[DOI]
- 김현숙, "대학생 글쓰기에서장르 기반 문법 지식 교육의 필요성 - 논설문을 중심으로 -", 작문연구, (29), pp.211~239 (2016), 한국작문학회
- 박종임, "논증적 글쓰기 인지진단평가를 위한 EDD(Empirically-derived and Descriptor-based Diagnostic) 체크리스트 개발 연구", 작문연구, (41), pp.131~160 (2019), 한국작문학회[DOI]
- 이상빈, "인공신경망(artificial neural network)을 활용한 일반계 고등학생의 비문학 독해 평가에서의 인지진단모형 적용 방안", 독서연구, (46), pp.107~145 (2018), 한국독서학회[DOI]
- 최숙기, "인지진단모형의 국어 교육적 접근 방안 탐색", 청람어문교육, (62), pp.41~66 (2017), 청람어문교육학회[DOI]
- 최숙기, "중등학생의 자료 통합적 글쓰기에 나타난 자료 글 차용하기 수행 양상 연구 - 인용과 환언을 중심으로-", 작문연구, (38), pp.201~245 (2018), 한국작문학회[DOI]
- {ENTER} ing the time series {SPACE}: Uncovering the writing process through keystroke analyses, "Allen, L. K.", International Educational Data Mining Society, ~ (2016)
- Bachman, L. F., "Fundamental considerations in language testing" (1990), Oxford university press
- Bachman, L. F., "Language testing in practice: Designing and developing useful language tests (Vol. 1)" (1996), Oxford University Press
- Banerjee, J., "Assessing and reporting performances on pre-sessional EAP courses : Developing a final assessment checklist and investigating its validity", Journal of English for Academic Purposes, 5(1), pp.50~69 (2006)
- Buck, G., "Application of the rule-space procedure to language testing : Examining attributes of a free response listening test", Language Testing, 15(2), pp.119~157 (1998)
- De La Paz, S., "Adolescents’ disciplinary use of evidence, argumentative strategies, and organizational structure in writing about historical controversies", Written Communication, 29(4), pp.412~454 (2012)
- Deane, P., "Using writing process and product features to assess writing quality and explore how those features relate to other literacy tasks", ETS Research Report Series, 2014(1), pp.1~23 (2014)
- Cognitive models of writing : Writing proficiency as a complex integrated skill, "Deane, P.", ETS, ~, pp.ETS Research Report Series 2008(2) (2008)
- Exploring the feasibility of using writing process features to assess text production skills, "Deane, P.", ETS, 1~16, pp.ETS Research Report Series 2015(2) (2015)
- DiBello, L. V., "Handbook of statistics: Psychometrics(Vol. 26)", pp.970~1030 (2007), Elsevier
- Douglas, D., "Assessing languages for specific purposes" (2000), Cambridege University Press
- Dunsmuir, S., "Children's writing and the use of information and communications technology", Educational Psychology in Practice, 19(3), pp.170~187 (2003)
- Effatpanah, F., "Diagnosing EFL learners’ writing ability : a diagnostic classification modeling analysis", Language Testing in Asia, 9(1), pp.12~ (2019)
- Mark Felton, "Arguing to Agree", Written Communication, 32(3), pp.317~331 (2015), SAGE Publications[DOI]
- Fernández-i-Marın, X., "Ggmcmc : Analysis of MCMC samples and bayesian inference", Journal of Statistical Software, 70(9), pp.1~20 (2016)
- Ferretti, R. P., "Knowledge of persuasion and writing goals predict the quality of children’s persuasive writing", Reading and Writing, 32(6), pp.1411~1430 (2019)
- Fischer, G. H., "The linear logistic test model as an instrument in educational research", Acta Psychological, 37(6), pp.359~374 (1973)
- Grabe, W., "Theory and practice of writing" (1996), Longman
- Grabe, W., "Theory and practice of writing : An applied linguistic perspective" (2014), Routledge
- Hartz, S. M., "A Bayesian framework for the unified model for assessing cognitive abilities : Blending theory with practicality", Doctor, pp.2002
- Jang, E. E., "Demystifying a Q-matrix for making diagnostic inferences about L2 reading skills", Language Assessment Quarterly, 6(3), pp.210~238 (2009)
- Kim, Y. H., "An argument-based validity inquiry into the empirically-derived descriptor-based diagnostic (EDD) assessment in ESL academic writing", Doctor, pp.2010
- Kim, Y. H., "Diagnosing EAP writing ability using the reduced reparameterized unified model", Language Testing, 28(4), pp.509~541 (2011)
- Knoch, U., "Rating scales for diagnostic assessment of writing : What should they look like and where should the criteria come from?", Assessing Writing, 16(2), pp.81~96 (2011)
- Kristopher Kyle, "The relationship between lexical sophistication and independent and source-based writing", Journal of Second Language Writing, 34, pp.12~24 (2016), Elsevier BV[DOI]
- Li, H., "Constructing and validating a Q-matrix for cognitive diagnostic analyses of a reading test", Educational Assessment, 18(1), pp.1~25 (2013)
- Yongyan Li, "Two first-year students’ strategies for writing from sources: Patchwriting or plagiarism?", Journal of Second Language Writing, 21(2), pp.165~180 (2012), Elsevier BV[DOI]
- MacArthur, C. A., "Handbook of writing research", pp.24~40 (2015), The Guilford Press
- Sharon McCulloch, "Investigating the reading-to-write processes and source use of L2 postgraduate students in real-life academic tasks: An exploratory study", Journal of English for Academic Purposes, 12(2), pp.136~147 (2013), Elsevier BV[DOI]
- Ekaterina Midgette, "The effects of content and audience awareness goals for revision on the persuasive essays of fifth- and eighth-grade students", Reading and Writing, 21(1-2), pp.131~151 (2008), Springer Science and Business Media LLC[DOI]
- Monte-Sano, C., "Reading, thinking, and writing about history : Teaching argument writing to diverse learners in the Common Core classroom, grades 6-12" (2014), Teachers College Press
- Moon, R., "Vocabulary: Description, acquisition and pedagogy", pp.40~63 (1997), Cambridge University Press
- Moon, R., "Fixed expressions and idioms in English : A corpus-based approach" (1998), Oxford University Press
- Nussbaum, E. M., "Using argumentation vee diagrams(AVDs)for promoting argument-counterargument integration in reflective writing", Journal of Educational Psychology, 100(3), pp.549~565 (2008)
- Nussbaum, E. M., "Promoting argument-counterargument integration in students' writing", The Journal of Experimental Education, 76(1), pp.59~92 (2007)
- Plakans, L., "The role of reading strategies in L2 writing tasks", Journal of English for Academic Purposes, 8, pp.252~266 (2009)
- Plakans, L., "Discourse synthesis in integrated second language assessment", Language Testing, 26, pp.1~27 (2009)
- Plakans, L., "A close investigation into source use in integrated second language writing tasks", Assessing Writing, 17(1), pp.18~34 (2012)
- Plakans, L., "Using multiple texts in an integrated writing assessment : source text use as a predictor of score", Journal of Second Language Writing, 22, pp.217~230 (2013)
- Rouet, J. F., "Text relevance and learning from text", pp.19~52 (2011), Information Age
- Roussos, L. A., "Cognitive diagnostic assessment for education: Theory and applications", pp.275~318 (2007), Cambridge University Press
- Templin, J., "Diagnostic measurement : Theory, methods, and applications" (2010), Guilford Press
- Sawaki, Y., "Q-matrix construction : Defining the link between constructs and test items in large-scale reading and listening comprehension assessments", Language Assessment Quarterly, 6(3), pp.190~209 (2009)
- Shahsavar, Z., "Diagnosing English learners’ writing skills : A cognitive diagnostic modeling study", Cogent Education, 6(1), pp.1608007~ (2019)
- Struthers, L., "Assessing cohesion in children’s writing : Development of a checklist", Assessing Writing, 18(3), pp.187~201 (2013)
- Van Waes, L., "Handbook of writing research", pp.410~426 (2016), Guilford Press
- Van Der Heide, J., "Instructional chains as a method for examining the teaching and learning of argumentative writing in classrooms", Written Communication, 30(3), pp.300~329 (2013)
- Weigle, S. C., "Source text borrowing in an integrated reading", writing assessment, 2012, pp.21(2), 118~133
- Williams, G. J., "Narrative writing, reading and cognitive processes in middle childhood : What are the links?", Learning and Individual Differences, 28, pp.142~150 (2013)
- Williams, J., "논증의 탄생" (2008), 홍문관
- Wolfe, C. R., "The use of other side information : Explaining the myside bias in argumentation", 46th annual meeting of the Psychonomic Society (2005)
- Wolfe, C. R., "Argumentation schema and the myside bias in written argumentation", Written Communication, 26(2), pp.183~209 (2009)
- Christopher R. Wolfe, "Individual Differences in the “Myside Bias” in Reasoning and Written Argumentation", Written Communication, 29(4), pp.477~501 (2012), SAGE Publications[DOI]
- Christopher R. Wolfe, "The locus of the myside bias in written argumentation", Thinking & Reasoning, 14(1), pp.1~27 (2008), Informa UK Limited[DOI]
- Christopher R. Wolfe, "Argumentation Schema and the Myside Bias in Written Argumentation", Written Communication, 26(2), pp.183~209 (2009), SAGE Publications[DOI]
- Xie, Q., "Diagnosing university students’ academic writing in English : Is cognitive diagnostic modelling the way forward?", Educational Psychology, 37(1), pp.26~47 (2017)
- Yeh, S. S., "Empowering education : Teaching argumentative writing to cultural minority middle-school students", Research in the Teaching of English, 33(1), pp.49~83 (1998)
