Investigating Learners’ Involvement in Process-Oriented Web Novel Assessment
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study examined the following aspects when learners actively participated in process-oriented assessment as raters, test-takers, and rubric developers: the assessment area to receive the highest weight in the learner-generated rubric, the reliability between self-assessment and expert assessment, the achievement levels of students in different assessment areas, and learners' perceptions of the process-oriented assessment. 25 college students with upper-intermediate to advanced writing proficiency participated in a workshop on web novel writing, rater training, the creation of analytical scoring rubrics, and the process of self-assessment based on their own rubrics. Analysis of the scoring rubrics, comparison of self-assessment and expert assessment, and the results from open-ended survey questions revealed that learners identified creativity as the area with the highest point, and there was a very high reliability between self- and expert assessment. Students showed relatively low achievement levels in the area of setting, particularly in character development and background description. While students noted psychological comfort and motivation as key advantages of the process-oriented web novel assessment, they also highlighted reliability issues as the main disadvantage. The study concluded with the implication that process-oriented assessment should be tailored to the specific assessment environment.
Abstract
This study examined the following aspects when learners actively participated in process-oriented assessment as raters, test-takers, and rubric developers: the assessment area to receive the highest weight in the learner-generated rubric, the reliability between self-assessment and expert assessment, the achievement levels of students in different assessment areas, and learners' perceptions of the process-oriented assessment. 25 college students with upper-intermediate to advanced writing proficiency participated in a workshop on web novel writing, rater training, the creation of analytical scoring rubrics, and the process of self-assessment based on their own rubrics. Analysis of the scoring rubrics, comparison of self-assessment and expert assessment, and the results from open-ended survey questions revealed that learners identified creativity as the area with the highest point, and there was a very high reliability between self- and expert assessment. Students showed relatively low achievement levels in the area of setting, particularly in character development and background description. While students noted psychological comfort and motivation as key advantages of the process-oriented web novel assessment, they also highlighted reliability issues as the main disadvantage. The study concluded with the implication that process-oriented assessment should be tailored to the specific assessment environment.
참고문헌 (29)
- An, B. Y., "A study regarding the perception of pre-service teachers on process-based and result-based assessments", Korean Journal of Teacher Education, 39(2), pp.5~22 (2023)
- Andrade, H. L., "Putting rubrics to the test:The effect of a model, criteria generation, and rubricreferenced self‐assessment on elementary school students'writing", Educational Measurement : Issues and Practice, 27(2), pp.3~13 (2008)[DOI]
- Becker, A., "Student-generated scoring rubrics: Examining their formative value for improving ESL students' writing performance", Assessing Writing, 29, pp.15~24 (2016)[DOI]
- Cho, Y., "Examining a process-oriented writing assessment in a large-scale ESL testing contest", Doctor, pp.2001
- Cho, Y., "Assessing writing: Are we bound by only one method?", Assessing Writing, 8(3), pp.165~191 (2003)[DOI]
- Choi, W., "Process-oriented speaking assessment of primary English using AI chatbots: Possibilities and limitations", Primary English Education, 26(1), pp.131~152 (2020)[DOI]
- Chong, S. W., "Three paradigms of classroom assessment:Implications for written feedback research", Language Assessment Quarterly, 15(4), pp.330~347 (2018)[DOI]
- Dann, R., "Assessment as learning: Blurring the boundaries of assessment and learning for theory, policy and practice", Assessment in Education: Principles, Policy & Practice, 21(2), pp.149~166 (2014)[DOI]
- Earl, L. M., "Assessment as learning: Using classroom assessment to maximize student learning" (2013), Corwin press
- 황성삼, "Discourse analysis of EFL interview test at secondary school : a focus on oral fluency", 박사, 영어교육, pp.2010
- Jo, S., "Identifying reader's internal needs and characteristics using keywords from Korean web novels", Journal of the Korea Institute of Information and Communication Engineering, 24(2), pp.158~165 (2020)
- Kim, M., "A study on the methodology of web novel creation", The Studies of Korean Literature, 77, pp.105~135 (2020)[DOI]
- Lee, H., "Validity of self-and peer-ratings in an EFL essay-writing test", English Teaching, 60(3), pp.195~219 (2005)
- Lee, H., "Assessing creativity in various speaking tasks: A perspective from pre-service English teachers", Modern Studies in English Language & Literature, 68(4), pp.87~117 (2024)[DOI]
- Lee, H., "Assessing the creativity of English dramas through university students’ learner-generated rubrics", English 21, 32(2), pp.349~372 (2019)
- Lee, H. K., "Classroom assessment of English oral proficiency : Focusing on self-, peer-, and teacher assessment", English Language Teaching, 21(4), pp.241~264 (2009)[DOI]
- Lee, I., "Assessment for learning: Integrating assessment, teaching, and learning in the ESL", EFL writing classroom, 2007, pp.64(1), 199~213
- Lee, Y., "Establishing theoretical foundation for narrative assessment in English language teaching", The Journal of Teaching English Literature, 27(3), pp.11~31 (2023)
- Lee, Y. J., "The process-oriented ESL writing assessment:Promises and challenges", Journal of Second Language Writing, 15(4), pp.307~330 (2006)[DOI]
- Mozaffari, H., "An analytical rubric for assessing creativity in creative writing", Theory & Practice in Language Studies, 3(12), pp.2214~2219 (2013)[DOI]
- Nawas, A., "Grading anxiety with self and peer-assessment: A mixed-method study in an Indonesian EFL context", Issues in Educational Research, 30(1), pp.224~244 (2010)
- Park, J., "Developing procedure-based assessment model of English and its application to elementary school students’ learning of listening and speaking", Teacher Education Research, 56(4), pp.367~384 (2017)[DOI]
- Park, S. M., "The unconventional and conservative characteristics of web novel narratives", Journal of Korean Literary Criticism, 75, pp.35~60 (2022)
- Shim, K. N., "Investigating a working model of process-oriented assessment of primary English", Primary English Education, 24(4), pp.119~143 (2018)
- Shim, K. N., "A study of the development of AI mediated assessment tasks and their application into practice in primary English classrooms", English Language Assessment, 15(1), pp.41~61 (2020)[DOI]
- Spiro, J., "Testing the untestable in language education", pp.165~190 (2010), Multilingual Matters
- White, E., "Student perspectives of peer assessment for learning in a public speaking course", Asia EFL Journal, 33(1), pp.1~36 (2009)
- Xu, Y., "Teacher assessment knowledge and practice:A narrative inquiry of a Chinese college EFL teacher's experience", TESOL Quarterly, 43(3), pp.492~513 (2009)[DOI]
- Yi, Y., "Questions arising from the assessment of EFL narrative writing", ELT Journal, 67(1), pp.70~79 (2013)[DOI]