한국무용 실기고사의 심사자간 신뢰도
Inter-judges Reliability of Korean Dance Performance Assessment
0
조회
0
다운로드
12
인용
0
참고문헌
키워드
초록
Traditionally, dance performance was assessed in term of holistic rating method which describes overall score instead of several discrete evaluation domains. This method, however, can be influenced by the judge's severity and inconsistency unlike the dance performance measures. The trimmed mean has been used in dance performance assessment as a summing method of judges' assessment results. The trimmed mean, however, may not account for the severity and leniency in the assessment of the remaining judges who are not involved in summing process. The purpose of the study was to examine the inter-judges' reliability of Korean dance performance assessment collected from three years. Five experts who worked at the University more than 10 years were selected for rating students' dance performances for three years from 2004 to 2006. Intraclass correlation coefficient (ICC) was calculated to examine the internal consistency of five judges' assessment. The range of ICCs obtained from dance performance assessment for three years was .840 to .851. The range of ICCs when a pertinent judge was deleted was .780 to .917 In conclusions, judges' dance performance ratings based on holistic rating method was moderately high, but not enough for requirement for competition judgment (ICC=.900) and appeared inconsistent assessment among judges. It is suggested that analytic rating method is recommended when assessing dance performance for the purpose of college entrance examination and national level dance competition.
Abstract
Traditionally, dance performance was assessed in term of holistic rating method which describes overall score instead of several discrete evaluation domains. This method, however, can be influenced by the judge's severity and inconsistency unlike the dance performance measures. The trimmed mean has been used in dance performance assessment as a summing method of judges' assessment results. The trimmed mean, however, may not account for the severity and leniency in the assessment of the remaining judges who are not involved in summing process. The purpose of the study was to examine the inter-judges' reliability of Korean dance performance assessment collected from three years. Five experts who worked at the University more than 10 years were selected for rating students' dance performances for three years from 2004 to 2006. Intraclass correlation coefficient (ICC) was calculated to examine the internal consistency of five judges' assessment. The range of ICCs obtained from dance performance assessment for three years was .840 to .851. The range of ICCs when a pertinent judge was deleted was .780 to .917 In conclusions, judges' dance performance ratings based on holistic rating method was moderately high, but not enough for requirement for competition judgment (ICC=.900) and appeared inconsistent assessment among judges. It is suggested that analytic rating method is recommended when assessing dance performance for the purpose of college entrance examination and national level dance competition.
참고문헌 (19)
- 강상조, "예,체능계 입시실기평가의 공정성: Rasch 모형의 적용", 40(1), pp.614~629 (2001)
- 강상조, "무용실기고사의 객관도", (16) (1997), 한국체육대학
- 최윤선, "심사자의 엄격성과 전공유형이 무용연기력 판정에 미치는 효과" (1998)
- 최윤선, "심사자의 엄격성과 전공유형이 무용연기력 판정에 미치는 효과", 43(1), pp.647~660 (2004)
- 황규자, "동아콩쿠르 발레연기 심사결과의 객관도" (1997)
- "A facet-factorial approach to the construction of rating scales to measure complex behaviors Journal of Educational Measurement", pp.145~151 (1973)
- Baumgartner, T. A, "Measurement for evaluation in physical education and exercise science" (2006), Boston: McGraw-Hill
- and Johnson, "The process-performance paradox in expert judgment How can experts know so much and predict so badly Toward a general theory of expertise", Cambridge University Press
- Fiske, "The effect of training procedure in musical performance evaluation on judge reliability" (1978)
- Jackson, "A comparison of alpha and the intraclass reliability coefficients Research Quarterly for Exercise and Sport", pp.568~571 (1980)
- Kang, S. J, "Comparison of holistic and analytic rating" (2007)
- "Approaches on the Assessment of Dance Performance Research Quarterly for Exercise and Sport"
- Landis, "The measurement of observer agreement for categorical data", pp.159~174 (1977)
- Looney, M, "Objective measurement of Figure skating performance", 1, pp.143~163 (1997)
- Lunz, "Measuring the impact of judge severity of examination scores Applied Measurement in Education", pp.331~345 (1990)
- Raymond, "Correcting performance rating errors in oral examinations", pp.100~122 (1991)
- Safrit, M. J, "Introduction to measurement in physical education and exercise science" (1995), St. Louis: Mosby
- Shrout, "Uses in assessing rater reliability", pp.420~428 (1979)
- Tasse, "Test-retest and inter-rater reliability of the Quebec Adaptive Behavior Scale Unpublished paper presented at the Annual Conference of the American Association of Mental Retardation" (1994)