What Do International Business Professionals Say about the TOEIC?
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
In today’s globalized world, the English language is regarded as a necessity to raise the international competitiveness of domestic industries for countries like South Korea. To assess employees’ qualifications in South Korea for international business, international companies have used the Test of English for International Communication (TOEIC) over the past 30 years. However, few empirical studies have investigated how thoroughly the TOEIC achieves its designated purposes in international business workplaces. The purpose of this study is to examine to what extent the intended meanings and uses of TOEIC scores and consequences of TOEIC use are achieved in the Korean context. Drawing on an argument-based validation framework using the stakeholder approach, the extent to which the TOEIC’s designated purposes have been achieved was investigated by collecting various validity related evidence through interviews with employers (n = 6) and employees (n = 10) in international business workplaces. The overall findings represent the complexity of relationships between stakeholders’ understanding and uses of TOEIC scores that are affected by contextual factors in South Korea. The findings call for including contextual factors in validation frameworks and studies. Validation researchers may need to pay more attention to contextual factors and collect validity evidence in collaboration with relevant stakeholders.
Abstract
In today’s globalized world, the English language is regarded as a necessity to raise the international competitiveness of domestic industries for countries like South Korea. To assess employees’ qualifications in South Korea for international business, international companies have used the Test of English for International Communication (TOEIC) over the past 30 years. However, few empirical studies have investigated how thoroughly the TOEIC achieves its designated purposes in international business workplaces. The purpose of this study is to examine to what extent the intended meanings and uses of TOEIC scores and consequences of TOEIC use are achieved in the Korean context. Drawing on an argument-based validation framework using the stakeholder approach, the extent to which the TOEIC’s designated purposes have been achieved was investigated by collecting various validity related evidence through interviews with employers (n = 6) and employees (n = 10) in international business workplaces. The overall findings represent the complexity of relationships between stakeholders’ understanding and uses of TOEIC scores that are affected by contextual factors in South Korea. The findings call for including contextual factors in validation frameworks and studies. Validation researchers may need to pay more attention to contextual factors and collect validity evidence in collaboration with relevant stakeholders.
참고문헌 (60)
- Allan, A., "Development and validation of a scale to measure test-wiseness in EFL", ESL reading test takers, 1992, pp.9, 101~122
- Amer, A. A., "Teaching EFL students to use a test-taking strategy", Language Testing, 10, pp.71~77 (1993)
- Bachman, L. F., "Language assessment in practice:Developing language assessments and justifying their use in the real world" (2010), Oxford University Press
- Booth, D. K., "Exploring the washback of the TOEIC in South Korea: A sociocultural perspective on student test activity", doctoral, pp.2012
- Braun, V., "Using thematic analysis in psychology", Qualitative Research in Psychology, 3, pp.77~101 (2006)
- 차경애, "A study of Korean university students’ English proficiency based on TOEIC test", Foreign Languages Education, 13(1), pp.275~295 (2006), 한국외국어교육학회
- Chapelle, C. A., "Building a validity argument for the Test of English as a Foreign Language" (2008), Routledge
- Cohen, A. D., "Interfaces between second language acquisition and language testing research", pp.90~111 (1998), Cambridge University Press
- Cohen, A. D., "‘I want to go back to the text’: Response strategies on the reading subtest of the new TOEFL", Language Testing, 24, pp.209~250 (2007)
- Crocker, L., "Teaching for the test: Validity, fairness, and moral action", Educational Measurement: Issues and Practice, 22, pp.5~11 (2003)
- Educational Testing Service, "Examinee handbook listening & reading" (2012), Educational Testing Service
- "User guide listening and reading" (2013)[URL]
- Elder, C., "What does test bias have to do with fairness?", Language Testing, 14, pp.261~277 (1997)
- Gatewood, R. D., "Human resource selection" (2008), Southwestern
- A comparative investigation into understandings and uses of the TOEFL iBT® Test, the International English Language Testing Service (Academic) test, and the Pearson Test of English for graduate admissions in the United States and Australia: A case study of two university contexts, "Ginther, A.", Educational Testing Service, ~, pp.TOEFL iBT research report no. 24, ETS research report no. RR-14-44 (2014)
- Hamp-Lyons, L., "Social, professional and individual responsibility in language testing", System, 28, pp.579~591 (2000)
- 황요한, "블렌디드 러닝이 토익 성취도에 미치는 영향: 대학 신입생 교양영어 수업 분석을 통해", 영어교과교육, 18(4), pp.233~248 (2019), 한국영어교과교육학회[DOI]
- Im, G. H., "Stakeholder voices: Validity argument for score meaning and use of the Test of English for International Communication (TOEIC) in international business workplaces in South Korea", doctoral, pp.2019
- 임관혁, "Legitimate or Illegitimate Uses of Test Scores in Contexts Unrelated to Test Purposes", 영어교육, 72(2), pp.71~99 (2017), 한국영어교육학회[DOI]
- Gwan-Hyeok Im, "Critical review of validation models and practices in language testing: their limitations and future directions for validation research", Language Testing in Asia, 9(14) (2019), Springer Science and Business Media LLC[DOI]
- 주미진, "의사소통 중심 교수법을 이용한 토익수업의 효과성 연구", 언어과학, 16(2), pp.83~102 (2009), 한국언어과학회
- Jung, Y. S., "The effect of TOEIC listening strategy instruction on students’ TOEIC listening scores", English Teaching, 53(4), pp.135~153 (1998)
- Kane, M. T., "An argument-based approach to validity", Psychological Bulletin, 112, pp.527~535 (1992)
- Kane, M. T., "Educational measurement", pp.2006, American Council on Education
- Kane, M. T., "Validating the interpretations and uses of test scores", Journal of Educational Measurement, 50, pp.1~73 (2013)
- 김용석, "토익읽기 선다형 문제의 구인타당성과 문항난이도 예측 연구", 영어교육연구, 15(4), pp.191~214 (2003), 팬코리아영어교육학회
- "YBM/Korea TOEIC® committee newsletter" (2013)[URL]
- 이효웅, "TOEIC의 신뢰도와 타당도", 영어교육, 58(4), pp.27~42 (2003), 한국영어교육학회
- Lee, I. S., "The effectiveness of TOEIC scores on English oral proficiency", Modern English Education, 7(1), pp.33~52 (2006)
- 이종복, "토익 성적 향상에 따른 영어 학습동기와 학습전략의 관계성 연구", 영어교과교육, 19(4), pp.205~221 (2020), 한국영어교과교육학회[DOI]
- Lee, S. J., "A validation study on TOEIC", Korean Journal of Psychology, 5(2), pp.49~58 (1999)
- Liu, J., "The research foundation for the TOEIC tests: A compendium of studies (Vol. 2)", pp.2.1~2.25 (2013)
- Louhiala-Salminen, L., "Professional communication in a global business context : The notion of global communicative competence", IEEE Transactions of Professional Communication, 54, pp.244~262 (2011)
- Ma, J., "TESOL Encyclopedia of English language teaching" (2018)
- McNamara, T., "Language testing: The social dimension" (2006), Blackwell Publishing
- Messick, S., "Educational measurement", pp.13~103 (1989), American Council on Education &Macmillan
- Mislevy, R. J., "On the structure of assessment arguments", Measurement : Interdisciplinary Research and Perspectives, 1, pp.3~62 (2003)
- Moss, P. A., "Validity in educational assessment", Review of Research in Education, 30, pp.109~162 (2006)
- 문채영, "Construct Validation of Picture Description Items on the TOEIC Listening Section", 응용언어학, 25(3), pp.391~417 (2009), 한국응용언어학회
- Munhwa Broadcasting Corporation, (2016), Munhwa
- "Broadcasting Corporation"[URL]
- 오희정, "토익 듣기 전략과 성취도의 관계", 영어교육연구, 23(3), pp.151~169 (2011), 팬코리아영어교육학회[DOI]
- O’Loughlin, K., "The interpretation and use of proficiency test scores in university selection: How valid and ethical are they?", Language Assessment Quarterly, 8, pp.146~160 (2011)
- O’Loughlin, K., "Developing the assessment literacy of university proficiency test users", Language Testing, 30, pp.363~380 (2013)
- Patton, M. Q., "Qualitative research and evaluation methods" (2002), Sage
- Potočnik, C., "International human resource management", pp.67~92 (2014), Cambridge University Press
- Powers, D. E., "The research foundation for the TOEIC tests: A compendium of studies (Vol. 2)", pp.1.1~1.7 (2013)
- The Redesigned TOEIC® (Listening and Reading) Test: Relations to Test-Taker Perceptions of Proficiency in English, "Powers, D. E.", ETS, ~, pp.ETS Research Rep. No. RR-08-56 (2008)
- Powers, D. E., "The incremental contribution of TOEIC®listening, reading, speaking, and writing tests to predicting performance on real-life English language tasks", Language Testing, 32, pp.151~167 (2015)
- (2018)[URL]
- "South Korea’s economic development, 1948–1996" (2017)[URL]
- Shepard, L. A., "The role of assessment in a learning culture", Educational Researcher, 29, pp.4~14 (2000)
- 신동일, "정책도구로 사용되는 언어시험: 국내 영어시험 중심으로", 영어교육, 66(4), pp.281~305 (2011), 한국영어교육학회[DOI]
- Shin, M., "Applying latent class analysis and survial analysis to classifying 4-year university students by job preference and investigating length of job searching period with examination of its determinants", The 11th KEEP (Korean education & employment panel) conference, pp.391~413 (2016)
- Shohamy, E., "The power of tests: A critical perspective on the uses of language tests" (2001), Pearson
- Shohamy, E., "Language tests as language policy tools", Assessment in Education, 14, pp.117~130 (2007)
- Sun, Y., "Context, construct, and consequences: Washback of the college English test in China", doctoral, pp.2016
- Tavakoli, E., "Test-wiseness strategies in PBTs and IBTs:The case of EFL test takers, who benefits more?", Procedia—Social and Behavioral Sciences, 98, pp.1876~1884 (2014)
- Validity of global self-ratings of ESL speaking proficiency based on an FSI, "ILR-Referenced scale", Wilson, K. M., 1999, ETS Research Rep. RR-99-13
- Zumbo, B. D., "Consequences, side effects and the ecology of testing: Keys to considering assessment ‘in vivo’", the annual meeting of the Association for Educational Assessment–Europe (AEA–Europe) (2015)