Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
2
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Purpose This study aims to apply the yes/no Angoff and Hofstee methods to actual Korean Medical Licensing Examination (KMLE) 2022 written examination data to estimate cut scores for the written KMLE. Methods Fourteen panelists gathered to derive the cut score of the 86th KMLE written examination data using the yes/no Angoff method. The panel reviewed the items individually before the meeting and shared their respective understanding of the minimum-competency physician. The standard setting process was conducted in 5 rounds over a total of 800 minutes. In addition, 2 rounds of the Hofstee method were conducted before starting the standard setting process and after the second round of yes/no Angoff. Results For yes/no Angoff, as each round progressed, the panel’s opinion gradually converged to a cut score of 198 points, and the final passing rate was 95.1%. The Hofstee cut score was 208 points out of a maximum 320 with a passing rate of 92.1% at the first round. It scored 204 points with a passing rate of 93.3% in the second round. Conclusion The difference between the cut scores obtained through yes/no Angoff and Hofstee methods did not exceed 2% points, and they were within the range of cut scores from previous studies. In both methods, the difference between the panelists decreased as rounds were repeated. Overall, our findings suggest the acceptability of cut scores and the possibility of independent use of both methods.
Abstract
Purpose This study aims to apply the yes/no Angoff and Hofstee methods to actual Korean Medical Licensing Examination (KMLE) 2022 written examination data to estimate cut scores for the written KMLE. Methods Fourteen panelists gathered to derive the cut score of the 86th KMLE written examination data using the yes/no Angoff method. The panel reviewed the items individually before the meeting and shared their respective understanding of the minimum-competency physician. The standard setting process was conducted in 5 rounds over a total of 800 minutes. In addition, 2 rounds of the Hofstee method were conducted before starting the standard setting process and after the second round of yes/no Angoff. Results For yes/no Angoff, as each round progressed, the panel’s opinion gradually converged to a cut score of 198 points, and the final passing rate was 95.1%. The Hofstee cut score was 208 points out of a maximum 320 with a passing rate of 92.1% at the first round. It scored 204 points with a passing rate of 93.3% in the second round. Conclusion The difference between the cut scores obtained through yes/no Angoff and Hofstee methods did not exceed 2% points, and they were within the range of cut scores from previous studies. In both methods, the difference between the panelists decreased as rounds were repeated. Overall, our findings suggest the acceptability of cut scores and the possibility of independent use of both methods.
참고문헌 (12)
- Ahn DS, "Reconsidering the cut score of Korean National Medical Licensing Examination", J Educ Eval Health Prof, 4 (2007)[DOI]
- Mikyoung Yim, "Comparison of results between modified-Angoff and bookmark methods for estimating cut score of the Korean medical licensing examination", Korean Journal of Medical Education, 30(4), pp.347~357 (2018), 한국의학교육학회[DOI]
- 양은배, "의사면허국가시험 출제문제 공개에 관한 인식", Korean Journal of Medical Education, 27(3), pp.167~175 (2015), 한국의학교육학회[DOI]
- Park J, "Comparison of standard-setting methods for the Korea Radiological technologist Licensing Examination : Angoff, Ebel, Bookmark, and Hofstee", J Educ Eval Health Prof, 15, pp.32~ (2018)[DOI]
- 임미경, "Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination", Journal of Educational Evaluation for Health Professions, 17, pp.1~9 (2020), 한국보건의료인국가시험원[DOI]
- Bourque J, "Performance of the Ebel standard-setting method in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions", J Educ Eval Health Prof, 17, pp.12~ (2020)[DOI]
- 박장희, "Possibility of using the yes", no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study, 2022, pp.19, 1~9[DOI]
- 허선, "Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers", Journal of Educational Evaluation for Health Professions, 19, pp.1~6 (2022), 한국보건의료인국가시험원[DOI]
- McKinley DW, "How to set standards on performance-based examinations: AMEE guide no. 85", Med Teach, 36, pp.97~110 (2014)[DOI]
- Yudkowsky R, "Assessment in health professions education", pp.119~148 (2009), Routledge
- 박장희, "Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination", Journal of Educational Evaluation for Health Professions, 17, pp.1~10 (2020), 한국보건의료인국가시험원[DOI]
- Tannenbaum RJ, "Consistency of Angoff-based standard-setting judgments : are item judgments and passing scores replicable across different panels of experts", Educ Assess, 20, pp.66~78 (2015)[DOI]