An Investigation into the Equivalence of Three Pictures for Creative Story Writing: ‘Dog Owners’, ‘Lost Dog’, and ‘Overslept’
0
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
Alternate pictures that are proven to be equivalent are in high demand to assess creative thinking and language skills. This study aimed to investigate the equivalence of three pictures (‘Dog owners,’ ‘Lost Dog,’ and ‘Overslept’) recently developed for use in a creative writing task. Middle school students (N=183) wrote a story in English based on one of the three prompts distributed randomly. Four writing features (fluency, syntactic complexity, lexical diversity, and temporality) were analyzed with Coh-Metrix and MANCOVA. The three prompts were largely equivalent in their capacity to detect differences among writers in all the features of writing. The difficulty levels of the three prompts, however, were not necessarily the same. Two prompts, Dog Owners and Lost Dog, were verified as equivalent prompts, and therefore, they are recommended as alternate forms to assess creative language skills in repeated measurements. The Overslept prompt had greater facility in eliciting diverse words and more temporal connectives in composing stories. The differential difficulty shown among the prompts suggests that the validity of using different picture versions in repeated assessment remains questionable unless those versions undergo equivalence verification.
Abstract
Alternate pictures that are proven to be equivalent are in high demand to assess creative thinking and language skills. This study aimed to investigate the equivalence of three pictures (‘Dog owners,’ ‘Lost Dog,’ and ‘Overslept’) recently developed for use in a creative writing task. Middle school students (N=183) wrote a story in English based on one of the three prompts distributed randomly. Four writing features (fluency, syntactic complexity, lexical diversity, and temporality) were analyzed with Coh-Metrix and MANCOVA. The three prompts were largely equivalent in their capacity to detect differences among writers in all the features of writing. The difficulty levels of the three prompts, however, were not necessarily the same. Two prompts, Dog Owners and Lost Dog, were verified as equivalent prompts, and therefore, they are recommended as alternate forms to assess creative language skills in repeated measurements. The Overslept prompt had greater facility in eliciting diverse words and more temporal connectives in composing stories. The differential difficulty shown among the prompts suggests that the validity of using different picture versions in repeated assessment remains questionable unless those versions undergo equivalence verification.
참고문헌 (55)
- Ackerman, J. M., "Reading, writing, and knowing: The role of disciplinary knowledge in comprehension and composing", Research in the Teaching of English, 25(2), pp.133~178 (1991)
- American Educational Research Association, "Standards for educational and psychological testing" (1999), American Educational Research Association, American Psychological Association, & National Council on Measurement in Education
- Bachman, L. F., "Fundamental considerations in language testing" (1990), Oxford University Press
- Bae, J., "Development of English skills need not suffer as a result of immersion: Grades 1 and 2 writing assessment in a Korean", English Two‐Way Immersion Program, 2007, pp.57(2), 299~332
- 배정옥, "어린이용 이야기 쓰기 그림 검사의 동형 확인: ‘피자’와 ‘놀이동산’", 교육평가연구, 27(1), pp.209~229 (2014), 한국교육평가학회
- Bae, J., "The validation of parallel test forms : ‘Mountain’ and ‘beach’ picture series for assessment of language skills", Language Testing, 28(2), pp.155~177 (2011)
- Bae, J., "On the role of content in writing assessment", Language Assessment Quarterly, 13(4), pp.302~328 (2016)
- Bae, J., "The effect of relative simplicity or complexity of picture prompt on language and creativity performances in writing", Paper presented at the 2012 Annual International Conference of the Korea English Language Testing Association (2012)
- Baker, E. L., "Studies in measurement and methodology. Work unit 1: Design and use of tests" (1979), University of California at Los Angeles
- Berman, R. A., "Relating events in narrative: A crosslinguistic developmental study" (1994), Erlbaum
- Cleaver, B., "Children's responses to silhouette illustrations in picture books" (1993), International Visual Literacy Association
- Cliffordson, C., "Effects of practice and intellectual growth on performance on the Swedish Scholastic Aptitude Test (SweSAT)", European Journal of Psychological Assessment, 20(3), pp.192~204 (2004)
- Crisp, V., "Can a picture ruin a thousand words? The effects of visual resources in exam questions", Educational Research, 48(2), pp.139~154 (2006)
- Cronbach, L. J., "Test ‘reliability’ : Its meaning and determination", Psychometrika, 12(1), pp.1~16 (1947)
- Cycowicz, Y. M., "Picture naming by young children : Norms for name agreement, familiarity, and visual complexity", Journal of Experimental Child Psychology, 65(2), pp.171~237 (1997)
- "Testing group difference using T-tests, ANOVA, and nonparametric measures" (2006)[URL]
- Department for Education and Employment, "The National Literacy Strategy: a framework for teaching" (1998), Department for Education and Employment
- Duff, K., "Evidence-based indicators of neuropsychological change in the individual patient:Relevant concepts and methods", Archives of Clinical Neuropsychology, 27(3), pp.248~261 (2012)
- Duran, N. D., "Using temporal cohesion to predict temporal coherence in narrative and expository texts", Behavior Research Methods, 39(2), pp.212~223 (2007)
- Fillmore, C., "Linguistics in the morning calm", pp.111~137 (1982), Hanshin
- Graesser, A. C., "Coh-Metrix: Analysis of text on cohesion and language", Behavior Research Methods, Instruments, & Computers, 36(2), pp.193~202 (2004)
- Greenhoot, A. D., "Do illustrations enhance preschoolers’ memories for stories? Age-related change in the picture facilitation effect", Journal of Experimental Child Psychology, 99(4), pp.271~287 (2008)
- Halliday, M. A. K., "Cohesion in English" (1976), Longman
- Hausknecht, J. P., "Retesting in selection : A meta-analysis of coaching and practice effects for tests of cognitive ability", Journal of Applied Psychology, 92(2), pp.373~385 (2007)
- 정주은, "The Influence of Picture Prompt Variation on Writing Performance: ‘Series’ vs. ‘Imagine Before and After’", 영어교육연구, 25(2), pp.27~46 (2013), 팬코리아영어교육학회
- 김하은, "Positive, negative, and nil effects of connectives in written stories: Analysis by proficiency groups", 언어연구, 32, pp.105~124 (2015), 언어정보연구소
- Kintsch, E., "Comprehension theory as a guide for the design of thoughtful questions", Topics in Language Disorders, 25(1), pp.51~64 (2005)
- Koizumi, R., "Effects of text length on lexical diversity measures : Using short texts with less than 200 tokens", System, 40(4), pp.554~564 (2012)
- Laufer, B., "Vocabulary size and use : Lexical richness in L2 written production", Applied Linguistics, 16(3), pp.307~322 (1995)
- 이신동, "영재학생의 영어 독해력 관련 변인 구조모형 분석", 영재와 영재교육, 12(1), pp.53~80 (2013), 한국영재교육학회[DOI]
- 이경화, "동화활용 언어프로그램이 유아의 창의성 향상에 미치는 효과", 영재와 영재교육, 7(1), pp.29~46 (2008), 한국영재교육학회[DOI]
- Liu, Y., "Analyzing RM ANOVA related data using SPSS 10", Measurement in Physical and Exercise Science, 6(1), pp.43~60 (2002)
- Lomax, R. G., "Statistical concepts: A second course for education and the behavioral sciences" (2001), Lawrence Erlbaum Publishers
- McCaffrey, R. J., "Practitioner's guide to evaluating change with neuropsychological assessment instruments", pp.2000, Kluwer Academic
- McCarthy, P. M., "vocd : A theoretical and empirical evaluation", Language Testing, 24(4), pp.459~488 (2007)
- McCarthy, P. M., "MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment", Behavior research methods, 42(2), pp.381~392 (2010)
- McCutchen, D., "Domain knowledge and linguistic knowledge in the development of writing ability", Journal of Memory and Language, 25(4), pp.431~444 (1986)
- McCutchen, D., "A capacity theory of writing : Working memory in composition", Educational Psychology Review, 8(3), pp.299~325 (1996)
- McCutchen, D., "Knowledge, processing, and working memory: Implications for a Theory of Writing", Educational Psychologist, 35(1), pp.13~23 (2000)
- McDonald, R. P., "Test theory: A unified treatment" (1999), Lawrence Erlbaum
- McNamara, D. S., "Linguistic features of writing quality", Written Communication, 27(1), pp.57~86 (2010)
- Meyers, L. S., "Applied multivariate research: design and interpretation" (2006), Sage Publications
- Oswald, F. L., "Extending a practical method for developing alternate test forms using independent sets of items", Organizational Research Methods, 8(2), pp.149~164 (2005)
- Pearce, W. M., "Does the choice of stimulus affect the complexity of children's oral narratives?", Advances in Speech-Language Pathology, 5(2), pp.95~103 (2003)
- Pena, E. D., "Dynamic assessment of school-age children's narrative ability: An experimental investigation of classification accuracy", Journal of Speech, Language, and Hearing Research, 49(5), pp.1037~1057 (2006)
- Peterson, C., "A longitudinal analysis of young children’s cohesion and noun specification in narratives", Journal of Child Language, 18(2), pp.397~415 (1991)
- Quereshi, M. Y., "Absence of parallel forms for the traditional individual intelligence tests", Current Psychology, 22(2), pp.149~154 (2003)
- Raymond, M. R., "Retest effects on identical and parallel forms in certification and licensure testing", Personnel Psychology, 60(2), pp.367~396 (2007)
- Schneider, P., "Story presentation effects on children’s retell content", American Journal of Speech-Language Pathology, 14(1), pp.52~60 (2005)
- Schweizer, M. L., "The effect of content, style, and color of picture prompts on narrative writing:An analysis of fifth and Eighth grade students’ writing", doctor, pp.1999
- Shapiro, L. R., "Tell me a make-believe story : Coherence and cohesion in young children's picture-elicited narratives", Developmental Psychology, 27(6), pp.960~974 (1991)
- Skehan, P., "A framework for the implementation of task-based instruction", Applied Linguistics, 17(1), pp.38~62 (1996)
- Weir, C. J., "Establishing test form and individual task comparability : Case study of a semi-direct speaking test", Language Testing, 23(2), pp.167~197 (2006)
- Wolfe-Quintero, K., "Second language development in writing:Measures of fluency, accuracy and complexity" (1998), University of Hawaii at Manoa
- Young, K. M., "Writing from primary documents : A way of knowing in history", Written Communication, 15(1), pp.25~68 (1998)