A study on evaluator factors affecting physician-patient interaction scores in clinical performance examinations: a single medical school experience
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Background: This study is an analysis of evaluator factors affecting physician-patient interaction (PPI) scores in clinical performance examination (CPX). The purpose of this study was to investigate possible ways to increase the reliability of the CPX evaluation. Methods: The six-item Yeungnam University Scale (YUS), four-item analytic global rating scale (AGRS), and one-item holistic rating scale (HRS) were used to evaluate student performance in PPI. A total of 72 fourth-year students from Yeungnam University College of Medicine in Korea participated in the evaluation with 32 faculty and 16 standardized patient (SP) raters. The study then examined the differences in scores between types of scale, raters (SP vs. faculty), faculty specialty, evaluation experience, and level of fatigue as time passes. Results: There were significant differences between faculty and SP scores in all three scales and a significant correlation among raters’ scores. Scores given by raters on items related to their specialty were lower than those given by raters on items out of their specialty. On the YUS and AGRS, there were significant differences based on the faculty’s evaluation experience; scores by raters who had three to ten previous evaluation experiences were lower than others’ scores. There were also significant differences among SP raters on all scales. The correlation between the YUS and AGRS/HRS declined significantly according to the length of evaluation time. Conclusion: In CPX, PPI score reliability was found to be significantly affected by the evaluator factors as well as the type of scale.
Abstract
Background: This study is an analysis of evaluator factors affecting physician-patient interaction (PPI) scores in clinical performance examination (CPX). The purpose of this study was to investigate possible ways to increase the reliability of the CPX evaluation. Methods: The six-item Yeungnam University Scale (YUS), four-item analytic global rating scale (AGRS), and one-item holistic rating scale (HRS) were used to evaluate student performance in PPI. A total of 72 fourth-year students from Yeungnam University College of Medicine in Korea participated in the evaluation with 32 faculty and 16 standardized patient (SP) raters. The study then examined the differences in scores between types of scale, raters (SP vs. faculty), faculty specialty, evaluation experience, and level of fatigue as time passes. Results: There were significant differences between faculty and SP scores in all three scales and a significant correlation among raters’ scores. Scores given by raters on items related to their specialty were lower than those given by raters on items out of their specialty. On the YUS and AGRS, there were significant differences based on the faculty’s evaluation experience; scores by raters who had three to ten previous evaluation experiences were lower than others’ scores. There were also significant differences among SP raters on all scales. The correlation between the YUS and AGRS/HRS declined significantly according to the length of evaluation time. Conclusion: In CPX, PPI score reliability was found to be significantly affected by the evaluator factors as well as the type of scale.
참고문헌 (30)
- Elliot DL, "Evaluation of physical examination skills : reliability of faculty observers and patient instructors", JAMA, 258, pp.3405~3408 (1987)
- Tamblyn RM, "The accuracy of standardized patient presentation", Med Educ, 25, pp.100~109 (1991)
- Vu NV, "Standardized(simulated)patients’ accuracy in recording clinical performance check-list items", Med Educ, 26, pp.99~104 (1992)
- Park H, "The agreement of checklist recordings between faculties and standardized patients in an objective structured clinical examination(OSCE)", Korean J Med Educ, 15, pp.143~152 (2003)
- 김주자, "일개 의과대학에서 실시한 표준화환자 (SP)를 이용한 임상수행능력평가시험 (CPX) 결과 분석", Korean Journal of Medical Education, 16(1), pp.51~62 (2004), 한국의학교육학회
- 권복규, "표준화 환자를 이용한 임상수행능력평가시험 (CPX)에서 교수와 표준화 환자의 평가 결과 비교", Korean Journal of Medical Education, 17(2), pp.173~184 (2005), 한국의학교육학회
- 박재현, "표준화 환자를 이용한 진료수행시험에서 교수와 표준화 환자의 채점 정확도", Korean Journal of Medical Education, 21(3), pp.287~297 (2009), 한국의학교육학회
- Miller GE, "The assessment of clinical skills", competence, Acad Med, pp.1990, 65(9 Suppl)[DOI]
- De Champlain AF, "Standardized patients’ accuracy in recording examinees’ behaviors using checklists", Acad Med, 72(10 Suppl 1), pp.S85~S87 (1997)
- Hodges B, "Analytic global OSCE ratings are sensitive to level of training", Med Educ, 37, pp.1012~1016 (2003)
- Scheffer S, "Assessing students’ communication skills : validation of a global rating", Adv Health Sci Educ Theory Pract, 13, pp.583~592 (2008)
- Heine N, "An analysis of standardised patient checklist errors and their effect on student scores", Med Educ, 37, pp.99~104 (2003)
- McLaughlin K, "Can standardized patients replace physicians as OSCE examiners", BMC Med Educ, 6, pp.12~ (2006)
- Tamblyn RM, "Sources of unreliability and bias in standardized-patient rating", Teach Learn Med, 3, pp.74~85 (1991)
- Domingues RC, "Global overall rating for assessing clinical competence : what does it really show", Med Educ, 43, pp.883~886 (2009)
- Cunnington JP, "The risks of thoroughness : reliability and validity of global ratings and checklists in an OSCE", Adv Health Sci Educ Theory Pract, 1, pp.227~233 (1996)
- Nielsen DG, "Objective structured assessment of technical competence in transthoracic echocardiography : a validity study in a standardized setting", BMC Med Educ, 13, pp.47~ (2013)
- Regehr G, "Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination", Acad Med, 73, pp.993~997 (1998)
- Cohen R, "Validity and generalizability of global ratings in an objective structured clinical examination", Acad Med, 66, pp.545~548 (1991)
- LeBlanc VR, "Psychometric properties of an integrated assessment of technical and communication skills", Am J Surg, 197, pp.96~101 (2009)
- Hodges B, "OSCE checklists do not capture increasing levels of expertise", Acad Med, 74, pp.1129~1134 (1999)
- Turner K, "Correlation between global rating scale and specific checklist scores for professional behaviour of physical therapy students in practical examinations", Educ Res Int, 2014, pp.219512~ (2014)
- Wilkinson TJ, "Patients’ global ratings of student competence : unreliable contamination or gold standard", Med Educ, 36, pp.1117~1121 (2002)
- Ilgen JS, "A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment", Med Educ, 49, pp.161~173 (2015)
- Newble D, "Techniques for measuring clinical competence : objective structured clinical examinations", Med Educ, 38, pp.199~203 (2004)
- Gerard JM, "Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure", Simul Healthc, 8, pp.148~154 (2013)
- van Luijk SJ, "Teaching and assessing clinical competence", pp.497~502 (1990), Boekwerk Publications
- Klein SP, "Analytic versus holistic scoring of science performance tasks", Appl Meas Educ, 11, pp.121~137 (1998)
- Regehr G, "Assessing the generalizability of OSCE measures across content domains", Acad Med, 74, pp.1320~1322 (1999)
- 최자윤, "호흡기계표준화환자를이용한간호대학생의임상수행능력평가타당도와신뢰도검증", Journal of Korean Academy of Nursing, 38(1), pp.83~91 (2008), 한국간호과학회