Evaluating the Validity of Classroom Assessments in the Secondary EFL Curriculum
2
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
This study investigated the validity of instructionally relevant classroom assessments embedded within a secondary English as a Foreign Language (EFL) curriculum. The study was conducted at a high school in Korea, including 148 12th graders learning EFL. For the validity analysis, two sets of achievement tests were chosen (i.e., multiple-choice listening tests for the midterm and final exams). The test items were qualitatively analyzed in relation to the national curriculum and instructional materials, and student test performance was also statistically analyzed. Results revealed that the two achievement tests were found to tap all curricular learning goals, albeit in an imbalanced manner due to the varied degree of coverage of the learning goals (cognitive validity). While the tests covered the target instructional contents, the degree of coverage and representativeness was again limited and inconsistent with the instructions (instructional validity). A statistical analysis of the test performance provided evidence of test validity but found a lack of consistency between the students’ listening ability and item difficulty levels (inferential validity). These findings suggest the types and forms of evidence required for test development and a validity argument within a learning-focused classroom assessment context.
Abstract
This study investigated the validity of instructionally relevant classroom assessments embedded within a secondary English as a Foreign Language (EFL) curriculum. The study was conducted at a high school in Korea, including 148 12th graders learning EFL. For the validity analysis, two sets of achievement tests were chosen (i.e., multiple-choice listening tests for the midterm and final exams). The test items were qualitatively analyzed in relation to the national curriculum and instructional materials, and student test performance was also statistically analyzed. Results revealed that the two achievement tests were found to tap all curricular learning goals, albeit in an imbalanced manner due to the varied degree of coverage of the learning goals (cognitive validity). While the tests covered the target instructional contents, the degree of coverage and representativeness was again limited and inconsistent with the instructions (instructional validity). A statistical analysis of the test performance provided evidence of test validity but found a lack of consistency between the students’ listening ability and item difficulty levels (inferential validity). These findings suggest the types and forms of evidence required for test development and a validity argument within a learning-focused classroom assessment context.
참고문헌 (35)
- Ahn, B-K., "High school English conversation" (2013), Chunjae Education Inc
- Bachman, L. F., "Statistical analyses for language assessment" (2004), Cambridge University Press
- Bachman, L. F., "Building and supporting a case for test use", Language Assessment Quarterly, 2, pp.1~34 (2005)
- Bachman, L. F., "Language testing in practice" (1996), Oxford University Press
- Bond, T. G., "Applying the Rasch model" (2015), Routledge
- Brindley, G., "Outcomes-based assessment in practice : Some examples and emerging insights", Language Testing, 18, pp.393~407 (2001)
- Brown, J. D., "Testing in language programs: A comprehensive guide to English language assessment" (2005), McGraw-Hill
- Butler, Y. G., "The effects of self-assessment among young learners of English", Language Testing, 27, pp.5~31 (2010)
- Cheng, L., "ESL", EFL instructors’ classroom assessment practice: Purposes, methods, and procedures, 2004, pp.21, 360~389
- 전유아, "Lexical Threshold of L2 Reading in the Korean CSAT", 영미연구, 31, pp.341~376 (2014), 영미연구소
- Colby-Kelly, C., "Teachers College Columbia University Roundtable in Second Language Studies" (2014)
- Colby-Kelly, C., "AFL research in the L2 classroom and evidence of usefulness : Taking formative assessment to the next level", Canadian Modern Language Review, 64, pp.9~38 (2007)
- Eckes, T., "Introduction to many-facet Rasch measurement: Analyzing and evaluating ratermediated assessments" (2015), Peter Lang
- Educational Broadcasting System, "Special lecture for College Scholastic Ability Test: English listening" (2017), Educational Broadcasting System
- Fisher, W. P. Jr, "Reliability statistics", Rasch Measurement: Transactions of the Rasch Measurement SIG, 6, pp.238~ (1992)
- Green, T., "Learning oriented language assessment for the classroom: A primer", Workshop presented at the 37th Language Testing Research Colloquium (2015)
- Hughes, A., "Testing for language teachers" (2003), Cambridge University Press
- Kane, M., "Educational measurement", pp.17~64 (2006), Greenwood Publishing
- Kane, M., "Validating score interpretations and uses", Language Testing, 29, pp.3~17 (2012)
- Kim, A-Y., "The effectiveness of instructor feedback for learning-oriented language assessment: Using an integrated reading-to-write task for English for academic purposes", Assessing Writing, 32, pp.57~71 (2017)
- Research on the development of achievement goals and levels for the revised English national curriculum, "Korea Institute for Curriculum and Evaluation", Korea Institute for Curriculum and Evaluation, ~ (2012)
- Kwon, S. K., "Educational assessment in the Republic of Korea: Lights and shadows of high-stake exam-based education system", Assessment in Education: Principles, Policy & Practice, 24, pp.60~77 (2017)
- Leung, C., "Developing formative teacher assessment : Knowledge, practice and change", Language Assessment Quarterly, 1, pp.19~41 (2004)
- Leung, C., "Expanding horizons and unresolved conundrums : Language testing and assessment", TESOL Quarterly, 40, pp.211~234 (2006)
- Leung, C., "Formative assessment in language education policies : Emerging lessons from Wales and Scotland", Annual Review of Applied Linguistics, 29, pp.64~79 (2009)
- Ministry of Education, Science, and Technology, "English National Curriculum", Ministry of Education, Science, and Technology, ~ (2009)
- Monitoring sources of variability within the Test of Spoken English Assessment System, "Myford, C. M.", Educational Testing Service, ~, pp.Research Report No. 65 (2000)
- Pellegrino, J. W., "A framework for conceptualizing and evaluating the validity of instructionally relevant assessments", Educational Psychologist, 51(1), pp.59~81 (2016)
- Purpura, J. E., "Second and foreign language assessment", Modern Language Journal, 100, pp.190~208 (2016)
- Rea-Dickins, P., "Currents and eddies in the discourse assessment : A learning-focused interpretation", International Journal of Applied Linguistics, 16, pp.163~188 (2006)
- Rea-Dickins, P., "Snares or silver bullets : Disentangling the construct of formative assessment", Language Testing, 17, pp.215~244 (2000)
- Tsagari, D., "Teachers College Columbia University Roundtable in Second Language Studies" (2014)
- Turner, E., "Routledge handbook of language testing", pp.65~78 (2012), Routledge, Taylor & Francis Group
- Turner, C. E., "Handbook of second language assessment", pp.255~273 (2016), De Gruyter Mouton
- Yin, M., "Understanding classroom language assessment through teacher thinking research", Language Assessment Quarterly, 7, pp.175~194 (2010)