객관구조화진료시험에서 평가자 평가와 수험자의 자기평가 간의 차이
Differences between Scores Assessed by Examiners and Examinees on Objective Structured Clinical Examination
2
조회
0
다운로드
7
인용
0
참고문헌
키워드
초록
Purpose: Self-assessment is important for learning. But, the reliability of self-assessment has long been questioned. We investigated this problem in an objective structured clinical examination (OSCE) setting. Methods: Forty third-year nursing students who participated in the OSCE were given the same checklist that was used for assessment by the examiners after they finished the OSCE. Then, the scores that were calculated by the examiners and examinees were compared, measuring the average, correlation, difference in score between examinee and examiner, difference in score according to the examinee's level, and Z-score. Results: Scores that were made by the examinees were significantly higher than those of the examiners (average 72.34 vs. 64.03), between which there was a loose positive correlation (r=0.34). The correlation coefficient increased (r=0.62) when a difference in score (examinee-examiner) was compared with the examiner score. When examinees were divided into 3 groups according to their scores, the difference tended to become less as the scores of examinee rose. When the Z-score was compared with the level of the examinee, examinees who had higher scores evaluated themselves lower than the examiners, and examinees with lower scores assessed themselves higher than the examiners. Conclusion: According to comparison of averages or correlation analysis, self- assessment by the examinee appeared to be less reliable. But when data were analyzed using difference in score, examinee levels, and Z-score, a certain tendency developed. This pattern may be due to the heterogeneity of the examinee group. Selection of appropriate analysis methods might be important to estabilish the reliability of self-assessment skills.
Abstract
Purpose: Self-assessment is important for learning. But, the reliability of self-assessment has long been questioned. We investigated this problem in an objective structured clinical examination (OSCE) setting. Methods: Forty third-year nursing students who participated in the OSCE were given the same checklist that was used for assessment by the examiners after they finished the OSCE. Then, the scores that were calculated by the examiners and examinees were compared, measuring the average, correlation, difference in score between examinee and examiner, difference in score according to the examinee's level, and Z-score. Results: Scores that were made by the examinees were significantly higher than those of the examiners (average 72.34 vs. 64.03), between which there was a loose positive correlation (r=0.34). The correlation coefficient increased (r=0.62) when a difference in score (examinee-examiner) was compared with the examiner score. When examinees were divided into 3 groups according to their scores, the difference tended to become less as the scores of examinee rose. When the Z-score was compared with the level of the examinee, examinees who had higher scores evaluated themselves lower than the examiners, and examinees with lower scores assessed themselves higher than the examiners. Conclusion: According to comparison of averages or correlation analysis, self- assessment by the examinee appeared to be less reliable. But when data were analyzed using difference in score, examinee levels, and Z-score, a certain tendency developed. This pattern may be due to the heterogeneity of the examinee group. Selection of appropriate analysis methods might be important to estabilish the reliability of self-assessment skills.
참고문헌 (20)
- Gordon MJ, "A review of the validity and accuracy of self-assessments in health professions training", Acad Med, 66, pp.762~769 (1991)
- Das M, "Self and tutor evaluations in problem-based learning tutorials: is there a relationship?", Med Educ, 32, pp.411~418 (1998)
- Morton JB, "Correlations between staff, peer and self assessments of fourth-year students in surgery", Med Educ, 11, pp.167~170 (1977)
- Palmer PB, "Effect of videotape replay on the quality and accuracy of student selfevaluation", Phys Ther, 65, pp.497~501 (1985)
- Nathan RG, "Using an in-training examination to assess and promote the self-evaluation skills of residents", Acad Med, 67, pp.613~ (1992)
- Lind DS, "Competency-based student self- assessment on a surgery rotation", J Surg Res, 105, pp.31~34 (2002)
- Cho MS, "A comparative study between self evaluation and superior evaluation in the nursing performance appraisal", 석사, pp.1994
- Regehr G, "Measuring self-assessment skills: an innovative relative ranking model", Acad Med, 71, pp.S52~S54 (1996)
- Mattheos N, "The interactive examination: assessing students' self-assessment ability", Med Educ, 38, pp.378~389 (2004)
- Arnold L, "Self- evaluation in undergraduate medical education: a longitudinal perspective", J Med Educ, 60, pp.21~28 (1985)
- Antonelli MA, "Accuracy of second-year medical students' self-assessment of clinical skills", Acad Med, 72, pp.S63~S65 (1997)
- Kruger J, "Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments", J Pers Soc Psychol, 77, pp.1121~1134 (1999)
- Evans AW, "The influence of self-deception and impression management on surgeons' self-assessment scores", Med Educ, 36, pp.1095~ (2002)
- Hwang JK, "School education and educational evaluation. 2nd ed", pp.527~528 (2006), Educational Science Publications
- Coutts L, "Predictors of student self- assessment accuracy during a clinical performance exam: comparisons between over-estimators and under-estimators of SP-evaluated performance", Acad Med, 74, pp.S128~S130 (1999)
- Woolliscroft JO, "Medical students' clinical self-assessments: comparisons with external measures of performance and the students' self-assessments of overall performance and effort", Acad Med, 68, pp.285~294 (1993)
- Hodges B, "Difficulties in recognizing one’s own incompetence: novice physicians who are unskilled and unaware of it", Acad Med, 76, pp.S87~S89 (2001)
- 허정원, "자기주도학습을 도입한 임상실습에서 의과대학생이 설정한 학습목표, 방법 및 의견과 자기평가 점수의 분석", 한국의학교육, 20(2), pp.123~132 (2008), 한국의학교육학회
- Violato C, "Self and peer assessment of pediatricians, psychiatrists and medicine specialists: implications for self-directed learning", Adv Health Sci Educ Theory Pract, 11, pp.235~244 (2006)
- Gruppen LD, "Woolliscroft JO. Medical students' self-assessments and their allocations of learning time", Acad M, 75, pp.374~379 (2000)