부산 ․경남 컨소시엄에서 임상수행평가의 시행 경험
Experience of clinical skills assessment in the Busan-Gyeongnam Consortium
2
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
Purpose: The purpose of this study is to judge the quality of clinical skills assessment in Busan-Gyeongnam Consortium. Methods: Fourth grade medical school students (n=350 in 2012 and n=419 in 2013) in the Busan-Gyeongnam Consortium were included in the study. The examination was consisted of 6 clinical performance examination (CPX) and 6 objective structured clinical examination (OSCE) stations. The students were divided into groups to take the exam in 4 sites during 3 days. The overall reliability was estimated by Cronbach α coefficient across the stations and the case reliability was by α across checklist items. Analysis of variance and between-group variation were used to evaluate the variation of examinee performance across different days and sites. Results: The mean total CPX/OSCE score was 67.0 points. The overall α across-stations was 0.66 in 2012 and 0.61 in 2013. The α across-items within a station was 0.54 to 0.86 in CPX, 0.51 to 0.92 in OSCE. There was no significant increase in scores between the different days. The mean scores over sites were different in 30 out of 48 stations but between-group variances were under 30%, except 2 cases. Conclusion: The overall reliability was below 0.70 and standardization of exam sites was unclear. To improve the quality of exam,case development, item design, training of standardized patients and assessors, and standardization of sites are necessary. Above of all, we need to develop the well-organized matrix to measure the quality of the exam.
Abstract
Purpose: The purpose of this study is to judge the quality of clinical skills assessment in Busan-Gyeongnam Consortium. Methods: Fourth grade medical school students (n=350 in 2012 and n=419 in 2013) in the Busan-Gyeongnam Consortium were included in the study. The examination was consisted of 6 clinical performance examination (CPX) and 6 objective structured clinical examination (OSCE) stations. The students were divided into groups to take the exam in 4 sites during 3 days. The overall reliability was estimated by Cronbach α coefficient across the stations and the case reliability was by α across checklist items. Analysis of variance and between-group variation were used to evaluate the variation of examinee performance across different days and sites. Results: The mean total CPX/OSCE score was 67.0 points. The overall α across-stations was 0.66 in 2012 and 0.61 in 2013. The α across-items within a station was 0.54 to 0.86 in CPX, 0.51 to 0.92 in OSCE. There was no significant increase in scores between the different days. The mean scores over sites were different in 30 out of 48 stations but between-group variances were under 30%, except 2 cases. Conclusion: The overall reliability was below 0.70 and standardization of exam sites was unclear. To improve the quality of exam,case development, item design, training of standardized patients and assessors, and standardization of sites are necessary. Above of all, we need to develop the well-organized matrix to measure the quality of the exam.
참고문헌 (16)
- 박훈기, "의사국가시험에서 실기시험의 도입이 의학교육에 미친 영향", 대한의사협회지, 55(2), pp.116~123 (2012), 대한의사협회
- Holmboe ES, "The role of assessment in competency-based medical education", Med Teach, 32, pp.676~682 (2010)
- Pell G, "How to measure the quality of the OSCE: a review of metrics - AMEE guide no. 49", Med Teach, 32, pp.802~811 (2010)
- Newble D, "Techniques for measuring clinical competence: objective structured clinical examinations", Med Educ, 38, pp.199~203 (2004)
- Brannick MT, "A systematic review of the reliability of objective structured clinical examination scores", Med Educ, 45, pp.1181~1189 (2011)
- 임미경, "임상수행능력평가(CPX)에서 학교효과가 신뢰도에 미치는 영향", Korean Journal of Medical Education, 22(3), pp.215~223 (2010), 한국의학교육학회
- Schoonheim-Klein M, "On the reliability of a dental OSCE, using SEM: effect of different days", Eur J Dent Educ, 12, pp.131~137 (2008)
- 한재진, "시험 시기에 따른 임상수행능력평가 점수 비교", Korean Journal of Medical Education, 19(1), pp.31~38 (2007), 한국의학교육학회
- 박훈기, "진료수행시험 (CPX: Clinical Performance Examination)에서 정보공유가 시험성적에 미치는 영향", Korean Journal of Medical Education, 17(2), pp.185~196 (2005), 한국의학교육학회
- Fuller R, "Longitudinal interrelationships of OSCE station level analyses, quality improvement and overall reliability", Med Teach, 35, pp.515~517 (2013)
- Auewarakul C, "Item analysis to improve reliability for an internal medicine undergraduate OSCE", Adv Health Sci Educ Theory Pract, 10, pp.105~113 (2005)
- Wilkinson TJ, "Standard setting in an objective structured clinical examination: use of global ratings of borderline performance to determine the passing score", Med Educ, 35, pp.1043~1049 (2001)
- Hodges B, "Analytic global OSCE ratings are sensitive to level of training", Med Educ, 37, pp.1012~1016 (2003)
- Hodges B, "Evaluating communication skills in the OSCE format: reliability and generalizability", Med Educ, 30, pp.38~43 (1996)
- Reznick RK, "Large-scale high-stakes testing with an OSCE: report from the Medical Council of Canada", Acad Med, 71(1 Suppl), pp.S19~S21 (1996)
- Dauphinee WD, "Using the judgments of physician examiners in setting the standards for a national multi-center high stakes OSCE", Adv Health Sci Educ Theory Pract, 2, pp.201~211 (1997)