교수자 직관 평가에 대한 타당성 고찰 -한일순차통역시험에 대한 동료평가 사례 연구를 중심으로
Considerations on the Validity of Intuition-Based Evaluations Made by Interpretation Instructors.
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This case study aims to verify the reliability of intuitive assessments conducted by some instructors during comprehensive evaluations, such as midterms, finals, and graduation exams in interpreting and translation graduate programs. Specifically, this research focuses on validating the legitimacy of such intuitive assessments through peer evaluation of final exams, with the additional objective of deepening students' understanding of intuitive assessment. In market-based interpretation settings, the primary assessors of quality are the users (audience or clients). In education settings, that role is assumed by instructors and students. While students typically undergo evaluation to enhance their interpreting abilities, they also participate directly in peer assessment to improve metacognitive skills related to interpreting assessment. Quality assessment of interpretation often relies on accuracy, clarity, and fidelity; however, the overlap and ambiguity between these criteria pose a challenge to the practicality of quality assessment. Therefore, comprehensive assessments in interpreter training programs often adopt a holistic approach. This comprehensive and holistic assessment approach depends on intuition, a sophisticated cognitive structure developed through extensive learning and experience. However, assessments based on instructors' intuition may raise concerns about validity. The results of this case study indicate that the average rankings from the first round of intuitive assessments by 20 peer evaluators largely corresponded with those from the second round of scaled assessments. Through this peer evaluation case study, the validity of intuitive assessments in interpreting has been partially corroborated.
Abstract
This case study aims to verify the reliability of intuitive assessments conducted by some instructors during comprehensive evaluations, such as midterms, finals, and graduation exams in interpreting and translation graduate programs. Specifically, this research focuses on validating the legitimacy of such intuitive assessments through peer evaluation of final exams, with the additional objective of deepening students' understanding of intuitive assessment. In market-based interpretation settings, the primary assessors of quality are the users (audience or clients). In education settings, that role is assumed by instructors and students. While students typically undergo evaluation to enhance their interpreting abilities, they also participate directly in peer assessment to improve metacognitive skills related to interpreting assessment. Quality assessment of interpretation often relies on accuracy, clarity, and fidelity; however, the overlap and ambiguity between these criteria pose a challenge to the practicality of quality assessment. Therefore, comprehensive assessments in interpreter training programs often adopt a holistic approach. This comprehensive and holistic assessment approach depends on intuition, a sophisticated cognitive structure developed through extensive learning and experience. However, assessments based on instructors' intuition may raise concerns about validity. The results of this case study indicate that the average rankings from the first round of intuitive assessments by 20 peer evaluators largely corresponded with those from the second round of scaled assessments. Through this peer evaluation case study, the validity of intuitive assessments in interpreting has been partially corroborated.
참고문헌 (48)
- 곽중철, "통번역대학원의 통역 수업 모형-협력학습 중심의 영한순차통역 강의 연구", 통번역학연구, 16(2), pp.85~114 (2012)
- 박헌일, "청중의 통역품질 평가에 대한 고찰", 통번역학연구, 23(1), pp.45~64 (2019)
- 이대현, "직관에 관한 연구 역사와 수학교육적 의미 고찰", 한국학교수학회논문집, 11(3), pp.363~376 (2008)
- 최문선, "TV통역 품질에 대한 사용자 평가 : 생방송 동시통역을 중심으로", 통번역교육연구, 16(3), pp.195~224 (2018)
- Pöchhacker, F., "통역학 입문" (2009), 이화여자대학교출판부
- Agor, W. H., "The logic of intuition : How top executives make important decisions", Journal of Advanced Research in Management, 7(3), pp.216~227 (1998)
- Anand, V., "An organizational memory approach to information management", Academy of Management Review, 23(4), pp.796~809 (1998)
- Atkinson, R. K., "Learning from examples : instructional principles from the worked examples research", Review of Educational Research, 70(2), pp.181~214 (2000)
- Bennett, R. E., "Formative assessment: a critical review. Assessment in Education", Principles, Policy & Practice, 18(1), pp.5~25 (2011)
- Black, P., "Assessment and classroom learning. Assessment in Education", Principles, Policy & Practice, 5(1), pp.7~74 (1998)
- Burke, L. A., "Taking the mystery out of intuitive decision making", Academy of Management Perspectives, 13(4), pp.91~99 (1999)
- Burke, L. A., "Instructor intuition in the educational setting", Academy of Management Learning &Education, 5(2), pp.169~181 (2006)
- Bühler, H., "Linguistic(semantic)and extralinguistic(pragmatic)criteria for the evaluation of conference interpretation and interpreters", Multilingua, 5(4), pp.231~235 (1986)
- Claxton, G., "The intuitive practitioner: on the value of not always knowing what one is doing" (2000), Open University Press
- Davison, C., "Inside teacher assessment decision-making : From judgement gestalts to assessment pathways", Frontiers in Education, 6, pp.636257~ (2021)
- Dubrovsky, R., "Curriculum review at the ISIT (Mexico City)", pp.61~69 (1990)
- Fischbein, E., "Intuition in science and mathematics: An educational approach" (1987), D. Reidel Publishing Company
- Fletcher, A., "How voice and choice influence giving feedback : Insights from contemporary performance management", Journal of Organizational Behavior, 33(1), pp.122~139 (2012)
- Havnes, A., "Formative assessment and feedback : Making learning visible", Studies in Educational Evaluation, 38(1), pp.21~27 (2012)
- Hogarth, R. M., "Educating Intuition" (2001), University of Chicago Press
- Kahane, L. H., "Anti-abortion activities and the market for abortion services—protest as a disincentive", American Journal of Economics and Sociology, 59(3), pp.463~485 (2000)
- Kahneman, D., "Thinking, fast and slow. Penguin" (2011), Farrar, Straus and Giroux
- Klein, G., "Decision Making in Action: Models and Methods", pp.138~147 (1993), Ablex Publishing Corporation
- Klein, G., "Sources of power: How people make decisions" (1998), MIT Press
- Klein, G., "Intuition at work: Why developing your gut instincts will make you better at what you do" (2002), Doubleday
- Kurz, I., "Conference interpretation : Expectations of different user groups", The Interpreters' Newsletter, 5, pp.13~21 (1993)
- Kurz, I., "Conference interpreting: quality in the ears of the user", Meta, Translators' Journal, 46(2), pp.394~409 (2001)
- Leonard, D., "The role of tacit knowledge in group innovation", California Management Review, 40(3), pp.112~132 (1998)
- Linn, R. L., "Measurement and assessment in teaching" (2000), Prentice-Hal
- Longley, P. E., "Language, Interpretation and Communication" (1978), Plenum
- Ormerod, R. J., "Rational inference : Deductive, inductive, and probabilistic thinking", Journal of the Operational Research Society, 61(8), pp.1207~1223 (2010)
- Polanyi, M., "The Tacit Dimension" (1966), University of Chicago Press
- Polanyi, M., "Life's irreducible structure", Science, 160(3834), pp.1308~1312 (1968)
- Rakoczy, K., "The interplay between student evaluation and instructional quality : Moderation by classroom characteristics", Learning and Instruction, 18(5), pp.429~442 (2008)
- Reber, A. S., "Implicit learning and tacit knowledge: An essay on the cognitive unconscious" (1993), Oxford University Press
- Reinholz, D., "The assessment cycle : A model for learning through peer assessment", Assessment & Evaluation in Higher Education, 41(2), pp.301~315 (2016)
- Roberts, A. R., "Crisis intervention handbook:Assessment, treatment, and research" (2000), Oxford University Press
- Sadler-Smith, E., "The intuitive executive : Understanding and applying 'gut feel' in decision-making", Academy of Management Executive, 18(4), pp.76~91 (2004)
- Shirley, D., "Intuition : A review of the literature", Psychological Reports, 79(2), pp.563~584 (1996)
- Simon, Herbert A., "Administrative behavior: A study of decision-making processes in administrative organizations" (1965), Macmillan
- Simon, Herbert A., "Making management decisions : The role of intuition and emotion", The Academy of Management Executive, 1(1), pp.57~64 (1987)
- Simon, Herbert A., "What is an explanation of behavior?", Psychological Science, 3(3), pp.150~161 (1992)
- Pöchhacker, F., "Quality assessment in conference and community interpreting", Meta: Journal des Traducteurs. Meta. Translators' Journal, 46(2), pp.410~425 (2001)
- Van der Kleij, F. M., "Integrating data-based decision making, assessment for learning, and diagnostic testing in formative assessment. Assessment in Education", Principles, Policy & Practice, 22(3), pp.300~329 (2015)
- Viezzi, M., "Interpretation quality assessment: from theory to practice", pp.171~186 (2007)
- Wadensjö, C., "Interpreting as interaction" (1998), Longman
- Wally, S., "Personal and structural determinants of the pace of strategic decision making", Academy of Management Journal, 37(4), pp.932~956 (1994)
- "Understanding the role of intuition in teaching" (2013)[URL]