본문 바로가기
KERD
논문KCI 등재

Reliability of a Repeated Testing Design: a Handwritten and Computerized ESL Writing Test

0

조회

0

다운로드

0

인용

0

참고문헌

주제분류영어와문학

초록

Abstract

This study examined the psychometric reliability of a testing design combined with a handwritten and a computerized essay test in an ESL placement context. That is, this study attempted to enlighten the medium (handwritten/computerized) effect on second language writers and raters from the viewpoints of testers. 42 international students took an essay-writing test twice: a handwritten test and a computerized test. Each of students' essays was scored in terms of 5 holistic/analytic features, that is, organization, linguistic expression, source use, content, and holistic feature. This study analyzed the rating data using descriptive statistics, G-theory, and intra-class correlation. As a result, the study found medium effect was conspicuous in an essay-writing test. The series of D-studies showed that a single handwritten test with a holistic judgment is a most reliable test. This study ended with the conclusion that testing professionals and/or teachers need to consider individual variability in test implementation.

참고문헌 (33)

  1. "Dostudents get higher scores on their word-processed papers? A study of bias in scoring hand-written versus word-processed papers" (1990), Unpublished manuscript
  2. "Language testing in practice" (1996), Oxford: University Press
  3. "Comparability of scores on word-processed and handwritten essays on the graduate management admission test" (1998)
  4. "The effect of rater variables in the development of an occupation-specifi language performance test", 12, pp.1~15 (1995)
  5. "A comparison of the effects of analytic and holistic composition in the context of composition tests", 11, pp.207~241 (2000)
  6. Collier, R, "When computer writers compose by hand", 12, pp.47~59 (1995)
  7. "The interpretation of tasks by writers and readers in holistically rated direct assessment of writing Reading in the composition classroom", pp.141~160 (1993)
  8. "Handwriting legibility and word-processing in assessing rater reliability" (2001)
  9. "Expertise in evaluating second language compositions", 7, pp.31~51 (1990)
  10. "Reflections on academic discourse How it relates to freshmen and our colleagues", pp.135~155 (1991)
  11. Ferris, D, "Teaching ESL composition: Purpose, process and practice" (1998), Mahwa, NJ: Lawrence Erlbaum Associates
  12. Second language writing, "Research insight for the classroom", pp.69~87 (1990), Cambridge University Press
  13. Hamp-Lyons, L., "Rating nonnative writing: The trouble with holistic scoring.", 29, pp.759~762 (1995)
  14. Hayes, J, "A new framework for understanding cognition and affect in writing" (1996), NJ: Lawrence Erlbaum Associates
  15. Hayes, "Identifying the organization of writing processes Cognitive processes in writing", pp.31~50 (1980), Lawrence Erlbaum Associates
  16. Huot, B, "Toward a new theory of writing assessment", 47, pp.549~66 (1996)
  17. Kellogg, "The psychology of writing" (1994), Oxford University Press
  18. Kroll, "What does time buy? ESL student performance on home versus class compositions Research insights for the classroom University of Cambridge Press", pp.140~154 (1990)
  19. Lee, H. K, "Examining the impact of word processing on an ESL placement assessment" (2002)
  20. "Rater characteristics and rater bias: Implications for training.", 12, pp.54~71 (1995)
  21. "A study of decision-making behavior of composition markers", pp.92~114 (1996), Cambridge: University of Cambridge Press
  22. "The computer and the non-native writer" (1996), Creskill, NJ: Hampton Press
  23. "Will they think less of my handwritten essay if others word process theirs? Effects on essay scores of intermingling handwritten and word processed essays Journal of Educational Measurement", pp.220~233 (1994)
  24. "Professor's reactions to the academic writing of nonnative speaking students", 22, pp.69~90 (1988)
  25. "Do English and ESL faculty differ in evaluating the essays of native English-speaking and ESL students?", 5, pp.163~182 (1996)
  26. "An investigation of writing features which have predictive evidence of validity for EPT placement levels" (1999)
  27. "Taking on testing", pp.93~121 (1987)
  28. "The effect of typing versus handwriting on the holistic scoring of essays Journal of Research and Development in Education", pp.24~29 (1992)
  29. "The effect of ESL errors on holistic scores assigned by English composition faculty", pp.53~69 (1993)
  30. "The influence of sentence-level and rhetorical feature Journal of Second Language Writing", pp.3~17 (1993)
  31. "Validation of a new holistic rating scale using Rasch multi-facet analysis , Validation in language testing", pp.39~57 (1996), Clevedon, Avon: Multilingual Matters
  32. "Effects of training on raters of ESL compositions"
  33. "Assessing writing" (2002), Cambridge: University Press

관련 논문AI추천

이 간행물의 다른 논문 (14건)