Legitimate or Illegitimate Uses of Test Scores in Contexts Unrelated to Test Purposes
1
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
Scores on the Test of English for International Communication Listening and Reading (TOEIC LR) have been used for rolling-based university admissions in South Korea. Because this test is not designed for that purpose by the testing organization, there may be a need to explore the validity of TOEIC LR score use for admission purposes. Accordingly, this study investigated how admissions officers in South Korea perceived TOEIC LR scores and their uses and why they used the scores for admission purposes. We adopted a sequential mixed-method design (Creswell & Clark, 2011) involving three phases of data collection: (1) analysis of application handbooks to identify the questionnaire survey target, (2) a survey of faculty (n = 20) involved in admissions, and (3) semi-structured interviews with selected participants (n = 5) to delve into their questionnaire responses. The findings were that (a) almost half of the universities in Korea were implementing special admissions and accepted the scores on the recognized English tests; (b) use of the test lacks validity because most of the admissions officers did not view the test scores as an indicator of English proficiency and academic aptitude; (c) approximately half of the admissions officers viewed the scores as a reflection of an applicant’s effort, test-taking skills learned at language institutes, and former residence in English-speaking countries; and (d) the TOEIC LR scores were used due to a lack of other English tests as well as social pressure such as test fees and testing site availability.
Abstract
Scores on the Test of English for International Communication Listening and Reading (TOEIC LR) have been used for rolling-based university admissions in South Korea. Because this test is not designed for that purpose by the testing organization, there may be a need to explore the validity of TOEIC LR score use for admission purposes. Accordingly, this study investigated how admissions officers in South Korea perceived TOEIC LR scores and their uses and why they used the scores for admission purposes. We adopted a sequential mixed-method design (Creswell & Clark, 2011) involving three phases of data collection: (1) analysis of application handbooks to identify the questionnaire survey target, (2) a survey of faculty (n = 20) involved in admissions, and (3) semi-structured interviews with selected participants (n = 5) to delve into their questionnaire responses. The findings were that (a) almost half of the universities in Korea were implementing special admissions and accepted the scores on the recognized English tests; (b) use of the test lacks validity because most of the admissions officers did not view the test scores as an indicator of English proficiency and academic aptitude; (c) approximately half of the admissions officers viewed the scores as a reflection of an applicant’s effort, test-taking skills learned at language institutes, and former residence in English-speaking countries; and (d) the TOEIC LR scores were used due to a lack of other English tests as well as social pressure such as test fees and testing site availability.
참고문헌 (38)
- Akiyama,T, "Introducing speaking tests into a Japanese Senior High School entrance examination", doctor, pp.2004
- American Educational Research Association, "Standards for educational and psychological testing" (2014), American Educational Research Association
- Bachman, L. F., "Building and supporting a case for test use", Language Assessment Quarterly, 2, pp.1~34 (2005)
- Bachman, L. F., "Language testing in practice: Designing and developing useful language tests" (1996), Oxford University Press
- Bachman, L. F., "Language assessment in practice" (2010), Oxford University Press
- Booth, D. K., "Exploring the washback of the TOEIC in South Korea: A sociocultural perspective on student test activity", doctor, pp.2012
- Creswell, J., "Designing and conducting mixed methods research" (2011), Sage
- "User guide listening and reading" (2013)[URL]
- "Sin Thoik Silhayng (New version of the TOEIC LR implemented)" (2016)[URL]
- Elder, C., "What does test bias have to do with fairness?", Language Testing, 14(3), pp.261~277 (1997)
- Glenn Fulcher, "TEST USE AND POLITICAL PHILOSOPHY", Annual Review of Applied Linguistics, 29, pp.3~ (2009), Cambridge University Press (CUP)[DOI]
- Fulcher, G., "Re-examining language testing: A philosophical and social inquiry" (2015), Routledge
- A comparative investigation into understandings and uses of the TOEFL iBT® Test, the International English Language Testing Service (Academic) Test, and the Pearson Test of English for Graduate Admissions in the United States and Australia: A case study of two university contexts, "Ginther, A.", Educational Testing Service, ~, pp.TOEFL iBT Research Report No. 24, ETS Research Report No. RR-14-44 (2014)
- Hirai, M., "Correlation between STEP BULATS speaking and TOEIC® scores", Proceedings of the 8th Annual JALT Pan-SIG Conference (2009)
- Kane, M. T., "Educational measurement", pp.2006, American Council on Education
- Kane, M. T., "Validating the interpretations and uses of test scores", Journal of Educational Measurement, 50(1), pp.1~73 (2013)
- "Silmwu Yengewa Tongttelecin Thoik Silhyoseng nonlan … Kiep Pwulsin Phayngpay (Companies’ distrust of the TOEIC LR due to lack of authentic business English)" (2006)[URL]
- "Naynyenpwute tayhakcengwuen kamchuk … chwoyhawuykulwup ‘thwoychwul’ (University enrollment will be cut from next year, ‘weeding out’ lowest ranking universities)" (2013)[URL]
- Lynch, B. K., "Rethinking assessment from a critical perspective", Language Testing, 18, pp.351~372 (2001)
- McNamara, T., "21st Century shibboleth : Language tests, identity and intergroup conflict", Language Policy, 4(4), pp.351~370 (2005)
- McNamara, T., "Language testing: The social dimension" (2006), Blackwell Publishing
- Mehrens, W. A., "The consequences of consequential validity", Educational Measurement : Issues and Practices, 16(2), pp.16~18 (1997)
- Menken, K., "Encyclopedia of language and education, 2nd edition, volume 7: Language testing and assessment", pp.401~414 (2008), Springer
- Messick, S., "Educational measurement", pp.13~103 (1989), American Council on Education & Macmillan
- Ministry of Education, Culture, Sports, Science and Technology, "Chugakuko Gakushu Shidoyoryo. (The course of study for junior high schools)" (1998), Kairyudo
- Ministry of Education, Culture, Sports, Science and Technology, "Kotogako Gakushu Shidoyouyou-Kaisetu (The course of study for senior high schools)" (1999), Kairyudo
- Mislevy, R. J., "Design and analysis in task–based language assessment", Language Testing, 19(4), pp.477~496 (2002)
- Mislevy, R. J., "On the structure of assessment arguments", Measurement : Interdisciplinary Research and Perspectives, 1(1), pp.3~62 (2003)
- "Pakkwuyn Thoik Cekunghalya Sinsupheyk kacchwulya Swuhemsayng ‘icwungko’ (Double whammy for test-takers as they adjust to new TOEIC LR)" (2016)[URL]
- O’Loughlin, K., "The interpretation and use of proficiency test scores in university selection: How valid and ethical are they?", Language Assessment Quarterly, 8, pp.146~160 (2011)
- O’Loughlin, K., "Developing the assessment literacy of university proficiency test users", Language Testing, 30(3), pp.363~380 (2013)
- Popham, W. J., "Consequential validity : Right concern–Wrong concept", Educational Measurement : Issues and Practices, 16(2), pp.9~13 (1997)
- "The case for a comprehensive, four skills assessment of English language proficiency (TOEIC Compendium TC-10-12)" (2010)[URL]
- Powers, D. E., "The incremental contribution of TOEIC® Listening, Reading, Speaking, and Writing tests to predicting performance on real-life English language tasks", Language Testing, 32(2), pp.151~167 (2015)
- The redesigned TOEIC® (listening and reading) test: Relations to test-taker perceptions of proficiency in English, "Powers, D. E", ~, pp.ETS Research Report RR-08-56 (2008)
- "Background and Goals of the TOEIC® Listening and Reading Test Redesign Project (TOEIC Compendium TC-10-02)" (2010)[URL]
- Shohamy, E., "The power of tests: A critical perspective on the uses of language tests" (2001), Pearson
- Zumbo, B. D., "Consequences, side effects and the ecology of testing:Keys to considering assessment ‘in vivo’", Keynote address, the annual meeting of the Association for Educational Assessment– Europe (AEA-Europe) (2015)