Reliability of a Repeated Testing Design: a Handwritten and Computerized ESL Writing Test
0
조회
0
다운로드
0
인용
0
참고문헌
초록
Abstract
This study examined the psychometric reliability of a testing design combined with a handwritten and a computerized essay test in an ESL placement context. That is, this study attempted to enlighten the medium (handwritten/computerized) effect on second language writers and raters from the viewpoints of testers. 42 international students took an essay-writing test twice: a handwritten test and a computerized test. Each of students' essays was scored in terms of 5 holistic/analytic features, that is, organization, linguistic expression, source use, content, and holistic feature. This study analyzed the rating data using descriptive statistics, G-theory, and intra-class correlation. As a result, the study found medium effect was conspicuous in an essay-writing test. The series of D-studies showed that a single handwritten test with a holistic judgment is a most reliable test. This study ended with the conclusion that testing professionals and/or teachers need to consider individual variability in test implementation.
참고문헌 (33)
- "Dostudents get higher scores on their word-processed papers? A study of bias in scoring hand-written versus word-processed papers" (1990), Unpublished manuscript
- "Language testing in practice" (1996), Oxford: University Press
- "Comparability of scores on word-processed and handwritten essays on the graduate management admission test" (1998)
- "The effect of rater variables in the development of an occupation-specifi language performance test", 12, pp.1~15 (1995)
- "A comparison of the effects of analytic and holistic composition in the context of composition tests", 11, pp.207~241 (2000)
- Collier, R, "When computer writers compose by hand", 12, pp.47~59 (1995)
- "The interpretation of tasks by writers and readers in holistically rated direct assessment of writing Reading in the composition classroom", pp.141~160 (1993)
- "Handwriting legibility and word-processing in assessing rater reliability" (2001)
- "Expertise in evaluating second language compositions", 7, pp.31~51 (1990)
- "Reflections on academic discourse How it relates to freshmen and our colleagues", pp.135~155 (1991)
- Ferris, D, "Teaching ESL composition: Purpose, process and practice" (1998), Mahwa, NJ: Lawrence Erlbaum Associates
- Second language writing, "Research insight for the classroom", pp.69~87 (1990), Cambridge University Press
- Hamp-Lyons, L., "Rating nonnative writing: The trouble with holistic scoring.", 29, pp.759~762 (1995)
- Hayes, J, "A new framework for understanding cognition and affect in writing" (1996), NJ: Lawrence Erlbaum Associates
- Hayes, "Identifying the organization of writing processes Cognitive processes in writing", pp.31~50 (1980), Lawrence Erlbaum Associates
- Huot, B, "Toward a new theory of writing assessment", 47, pp.549~66 (1996)
- Kellogg, "The psychology of writing" (1994), Oxford University Press
- Kroll, "What does time buy? ESL student performance on home versus class compositions Research insights for the classroom University of Cambridge Press", pp.140~154 (1990)
- Lee, H. K, "Examining the impact of word processing on an ESL placement assessment" (2002)
- "Rater characteristics and rater bias: Implications for training.", 12, pp.54~71 (1995)
- "A study of decision-making behavior of composition markers", pp.92~114 (1996), Cambridge: University of Cambridge Press
- "The computer and the non-native writer" (1996), Creskill, NJ: Hampton Press
- "Will they think less of my handwritten essay if others word process theirs? Effects on essay scores of intermingling handwritten and word processed essays Journal of Educational Measurement", pp.220~233 (1994)
- "Professor's reactions to the academic writing of nonnative speaking students", 22, pp.69~90 (1988)
- "Do English and ESL faculty differ in evaluating the essays of native English-speaking and ESL students?", 5, pp.163~182 (1996)
- "An investigation of writing features which have predictive evidence of validity for EPT placement levels" (1999)
- "Taking on testing", pp.93~121 (1987)
- "The effect of typing versus handwriting on the holistic scoring of essays Journal of Research and Development in Education", pp.24~29 (1992)
- "The effect of ESL errors on holistic scores assigned by English composition faculty", pp.53~69 (1993)
- "The influence of sentence-level and rhetorical feature Journal of Second Language Writing", pp.3~17 (1993)
- "Validation of a new holistic rating scale using Rasch multi-facet analysis , Validation in language testing", pp.39~57 (1996), Clevedon, Avon: Multilingual Matters
- "Effects of training on raters of ESL compositions"
- "Assessing writing" (2002), Cambridge: University Press
