본문 바로가기
KERD
논문KCI 등재

Examining the Effects of Peer Review in EFL Writing: A Coh-Metrix Analysis

2

조회

0

다운로드

2

인용

0

참고문헌

키워드

초록

This study investigated the effectiveness of peer feedback in EFL writing classes. A total of 285 original editorials by 103 Korean learners of English were compared with revised drafts reviewed by classmates. Coh-Metrix, an automated tool offering a wide range of measures on cohesion and language was used for analyses. Fourteen indices of the Coh-Metrix system were selected to observe variances in lexical (i.e., TTR, frequency, age of acquisition, concreteness, familiar- ity, meaningfulness, polysemy), syntactic (i.e., number of words before the main verb, NP/VP density), and discourse (i.e., argument/content word overlap, LSA) aspects between the two sets of writings. There were significant improvements in the lexical aspect as positive modifications were noted in revised drafts. In addition, selective enhancements were observed in the syntactic aspect. Although the number of words before the main verb increased in the revised drafts, NP/VP density resulted in no difference between the two corpora. For the discourse aspect, no significant difference was found. Overall, the implementation of peer review led to positive outcomes in specific areas. However, results also revealed overt limitations of the practice in EFL settings. These findings suggest some educational implications in EFL writing pedagogy.

Abstract

This study investigated the effectiveness of peer feedback in EFL writing classes. A total of 285 original editorials by 103 Korean learners of English were compared with revised drafts reviewed by classmates. Coh-Metrix, an automated tool offering a wide range of measures on cohesion and language was used for analyses. Fourteen indices of the Coh-Metrix system were selected to observe variances in lexical (i.e., TTR, frequency, age of acquisition, concreteness, familiar- ity, meaningfulness, polysemy), syntactic (i.e., number of words before the main verb, NP/VP density), and discourse (i.e., argument/content word overlap, LSA) aspects between the two sets of writings. There were significant improvements in the lexical aspect as positive modifications were noted in revised drafts. In addition, selective enhancements were observed in the syntactic aspect. Although the number of words before the main verb increased in the revised drafts, NP/VP density resulted in no difference between the two corpora. For the discourse aspect, no significant difference was found. Overall, the implementation of peer review led to positive outcomes in specific areas. However, results also revealed overt limitations of the practice in EFL settings. These findings suggest some educational implications in EFL writing pedagogy.

참고문헌 (76)

  1. 안병규, "A Coh-Metrix Analysis of Elementary School English Textbooks", 영어영문학21, 28(3), pp.435~460 (2015), 21세기영어영문학회[DOI]
  2. 안수진, "한국인 대학생과 영어 원어민 대학생의 영어 설명문, 논설문의 코메트릭스 분석을 통한 비교", 새한영어영문학, 60(3), pp.177~205 (2018), 새한영어영문학회[DOI]
  3. Arapoff, N., "Discover and transform : A method of teaching writing to foreign students", TESOL Quarterly, 3(4), pp.297~304 (1969)
  4. "Linguistic Data Consortium, University of Pennsylvania", 1995
  5. Bangert-Drowns, R., "The instructional effect of feedback in testlike events", Review of Educational Research, 61(2), pp.213~238 (1991)
  6. Beck, I., "Bringing words to life: Robust vocabulary development" (2002), Guilford Press
  7. Berlin, J., "Rhetoric and ideology in the writing class", College English, 50(5), pp.477~494 (1988)
  8. Bialystok, E., "Symbolic representation and attentional control in pragmatic competence", Interlanguage Pragmatics, 3(1), pp.43~57 (1993)
  9. M. Ali Bolgün, "A Contrastive Rhetoric Analysis of English and Hindi Editorials", Journal of Pan-Pacific Association of Applied Linguistics, 21(2), pp.15~39 (2017), 범태평양 응용언어학회[DOI]
  10. Bouton, L. F., "Conversational implicature in the second language: Learned slowly when not deliberately taught", Journal of Pragmatics, 22(2), pp.157~167 (1994)
  11. 장지연, "Coh-Metrix를 통한 2017-2018 수능 영어 독해 지문 비교: 기초 산출치, 이독성, 용이성을 중심으로", Foreign Languages Education, 25(4), pp.81~106 (2018), 한국외국어교육학회[DOI]
  12. Chaudron, C., "Evaluating writing : Effects of feedback on revision", RELC Journal, 15(2), pp.1~14 (1984)
  13. Cho, Kwangsu, "Scaffolded writing and rewriting in the discipline", Computers and Education, 48(3), pp.409~426 (2007)
  14. 최정은, "A Comparative Discourse Analysis of English and Korean Editorials", 영어학, 15(2), pp.231~257 (2015), 한국영어학회[DOI]
  15. Cohen, A., "Developing the ability to perform speech acts", Studies in Second Language Acquisition, 18(2), pp.253~267 (1996)
  16. Coltheart, M., "The MRC psycholinguistic database", The Quarterly Journal of Experimental Psychology Section A, 33(4), pp.497~505 (1981)
  17. Connor, U., "Contrastive rhetoric: Cross-cultural aspects of second-language writing" (1996), Cambridge University Press
  18. Crossley, S. A., "Computational assessment of lexical differences in L1 and L2 writing", Journal of Second Language Writing, 18(2), pp.119~135 (2009)
  19. Crossley, S. A., "Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing", International Journal of Continuing Engineering Education and Life Long Learning, 21(2-3), pp.170~191 (2011)
  20. Crossley, S. A., "Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners", Journal of Second Language Writing, 26, pp.66~79 (2014)
  21. Edelsky, C., "Writing in a bilingual program : The relation of L1 and L2 texts", TESOL Quarterly, 16(2), pp.211~228 (1982)
  22. Elbow, P., "Writing without teachers" (1973), Oxford University Press
  23. Fox, T., "The social uses of writing: Politics and Pedagogy" (1990), Ablex
  24. Gernsbacher, M. A., "Language comprehension as structure building" (1990), Erlbaum
  25. Gielen, S., "Improving the effectiveness of peer feedback for learning", Learning and Instruction, 20(4), pp.304~315 (2010)
  26. Gilhooly, K. J., "Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1, 944 words", Behavior Research Methods & Instrumentation, 12(4), pp.395~427 (1980)
  27. Graesser, A. C., "Discourse cohesion in text and tutorial dialogue", Information Design Journal, 15(3), pp.199~213 (2007)
  28. Graesser, A. C., "Coh-Metrix: Analysis of text on cohesion and language", Behavior Research Methods, Instruments, & Computers, 36(2), pp.193~202 (2004)
  29. Haberlandt, K. F., "Component processes in text comprehension and some of their interactions", Journal of Experimental Psychology : General, 114(3), pp.357~374 (1985)
  30. Hattie, J., "The power of feedback", Review of Educational Research, 77(1), pp.81~112 (2007)
  31. House, J., "Developing pragmatic fluency in English as a foreign language : Routines and metapragmatic awareness", Studies in Second Language Acquisition, 18(2), pp.225~252 (1996)
  32. Jacobs, S. E., "Linguistics and literacy series: 3" (1982), Center for Applied Linguistics
  33. 장은영, "신문의 ‘보도기사’와 ‘사설’에 쓰인 영어 관계대명사의 담화기능 비교", 영어영문학연구, 56(3), pp.365~382 (2014), 한국중앙영어영문학회[DOI]
  34. 전문기, "Coh-Metrix를 이용한 중학교 1학년과 2학년 개정 영어교과서 읽기 자료의 코퍼스 언어학적 연계성 분석", 언어과학연구, (56), pp.201~218 (2011), 언어과학회
  35. 전문기, "고등학교 개정 영어교과서 읽기 자료의 연계성 분석", Journal of The Korean Data Analysis Society, 16(2), pp.925~938 (2014), 한국자료분석학회
  36. 전문기, "자동화된 언어분석 프로그램을 활용한 중학교 영어교과서의 학년 간 연계성 분석", 현대영어교육, 16(1), pp.195~218 (2015), 현대영어교육학회
  37. 전문기, "코메트릭스(Coh-Metrix)를 이용한 중학교 1학년 개정 영어 교과서의 코퍼스 언어학적 비교 분석", 영어교육연구, 21(4), pp.265~292 (2009), 팬코리아영어교육학회[DOI]
  38. Jeong, Mikyung, "A comparative study on the English argumentative essays of Korean and native English university students with Coh-Metrix: Focused on lexical, syntactic, and discourse features", doctoral, pp.2015
  39. 정미경, "Coh-Metrix를 통한 한국인 EFL학습자와 영어 모국어 화자의 논설문 코퍼스에 나타난 언어적 특성 분석", 언어학 연구, (33), pp.373~395 (2014), 한국중원언어학회[DOI]
  40. Jung, Philip-Yoongoo, "Comparing the difficulty of classroom English in teacher-guidebooks by levels using Coh-Metrix", English Teaching
  41. Just, M. A., "The psychology of reading and language comprehension" (1987), Allyn & Bacon
  42. 김철규, "코퍼스에 기반한 영국 신문의 논설문과 한국 대학생의 영어 논설문에 사용된 메타담화(metadiscourse) 비교 연구", 담화와 인지, 16(3), pp.65~88 (2009), 담화·인지언어학회[DOI]
  43. 김정렬, "Coh-Metrix를 통한 초·중등 영어교과서 연계성 분석", 영어교육, 67(2), pp.319~341 (2012), 한국영어교육학회[DOI]
  44. Kim, Minjeong, "The effects of the assessor and assessee’s roles on preservice teachers’ metacognitive awareness, performance, and attitude in a technology-related design task", Doctoral, pp.2005
  45. 김소정, "Coh-Metrix를 활용한 초등학교 6학년 영어학습자의 영작문 분석 연구", 현대영어교육, 17(3), pp.263~287 (2016), 현대영어교육학회
  46. Kluger, A. N., "The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory", Psychological Bulletin, 119(2), pp.254~284 (1996)
  47. Landauer, T. K., "A solution to Plato’s problem : The latent semantic analysis theory of acquisition, induction, and representation of knowledge", Psychological Review, 104(2), pp.211~240 (1997)
  48. Landauer, T. K., "Handbook of latent semantic analysis" (2007), Lawrence Erlbaum
  49. 이지영, "A Coh-Metrix Analysis of Lexical, Syntactic and Discourse Aspects in the Newspaper Articles of Korean and British University Students", 현대영어교육, 19(4), pp.17~26 (2018), 현대영어교육학회[DOI]
  50. 이지영, "A Coh-Metrix Analysis of the English Editorials of Korean, American, and British University Students", 현대영어교육, 21(3), pp.15~27 (2020), 현대영어교육학회[DOI]
  51. 이승환, "코메트릭스(Coh-Metrix)를 이용한 고등학교 영어1과 영어2 교과서 문어 텍스트의 코퍼스 언어학적 난이도 분석", 외국어교육연구, 27(2), pp.131~148 (2013), 외국어교육연구소[DOI]
  52. Ljung, M., "Studies in English language and teaching", pp.75~83 (1997), Rodopi
  53. McNamara, D. S., "Automated evaluation of text and discourse with Coh-Metrix" (2014), Cambridge University Press
  54. McNamara, D. S., "Suppressing irrelevant information: Knowledge activation or inhibition?", Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(2), pp.465~182 (2004)
  55. Mendonca, C. O., "Peer review negotiations : Revision activities in ESL writing instruction", TESOL Quarterly, 28(4), pp.745~769 (1994)
  56. Mory, E. H., "Handbook of research on educational communications and technology", pp.745~783 (2004), Lawrence Erlbaum
  57. Narciss, S., "Feedback strategies for interactive learning tasks", Handbook of Research on Educational Communications and Technology, 3, pp.125~144 (2008)
  58. Nelson, G. L., "Peer response groups: Do L2 writers use peer comments in revising their drafts?", TESOL Quarterly, 27(1), pp.135~142 (1993)
  59. Orsmond, P., "Moving feedback forward : Theory to practice", Assessment & Evaluation in Higher Education, 38(2), pp.240~252 (2013)
  60. Perdue, V. A., "Writing as an act of power: Basic writing pedagogy as social practice (Composition, literacy, rhetoric)", doctoral, pp.1984
  61. Perfetti, C., "Reading ability : Lexical quality to comprehension", Scientific Studies of Reading, 11(4), pp.357~383 (2007)
  62. Pierson, H., "Peer and teacher correction: A comparison of the effects of two methods of teaching composition in grade nine English classes", doctoral, pp.1967
  63. Purves, A. C., "The teacher as reader : An anatomy", College English, 46(3), pp.259~265 (1984)
  64. Rollinson, P., "Using peer feedback in the ESL writing class", ELT journal, 59(1), pp.23~30 (2005)
  65. 류지수, "Coh-Metrix 시스템을 이용한 고등학교 영어교과서의 듣기대본 복잡도 분석", 영어학, 20, pp.363~383 (2020), 한국영어학회[DOI]
  66. Santos, T., "Ideology in composition : L1 and ESL", Journal of Second Language Writing, 1(1), pp.1~15 (1992)
  67. Schmidt, R., "Interlanguage pragmatics", pp.21~42 (1993), Oxford University Press
  68. Silva, T., "Toward an understanding of the distinct nature of L2 writing: The ESL research and its implications", TESOL Quarterly, 27(4), pp.657~677 (1993)
  69. Sluijsmans, D. M. A., "Peer assessment training in teacher education : Effects on performance and perceptions", Assessment and Evaluation in Higher Education, 27(5), pp.443~454 (2002)
  70. 성일호, "이독성공식과 Coh-Metrix를 활용한우리나라 고등학교 영어교과서 이독성 분석", 영어영문학연구, 40(4), pp.299~320 (2014), 대한영어영문학회[DOI]
  71. Toglia, M. P., "Handbook of semantic word norms" (1978), Erlbaum
  72. Topping, K., "Peer assessment between students in colleges and universities", Review of Educational Research, 68(3), pp.249~276 (1998)
  73. Tsui, A. B. M., "Do secondary L2 writers benefit from peer comments?", Journal of Second Language Writing, 9(2), pp.147~170 (2000)
  74. Van Steendam, E., "The effect of instruction type and dyadic or individual emulation on the quality of higher-order peer feedback in EFL", Learning and Instruction, 20(4), pp.316~327 (2010)
  75. Villamil, O. S., "Peer revision in the L2 classroom : Social-cognitive activities, mediating strategies, and aspects of social behavior", Journal of Second Language Writing, 5(1), pp.51~75 (1996)
  76. Zhang, S., "Reexamining the affective advantage of peer feedback in the ESL writing class", Journal of Second Language Writing, 4(3), pp.209~222 (1995)

관련 논문AI추천

이 간행물의 다른 논문 (8건)