본문 바로가기
KERD
논문KCI 등재

Assessment methods and the validity and reliability of measurement tools in online objective structured clinical examinations: a systematic scoping review

2

조회

0

다운로드

1

인용

0

참고문헌

키워드

초록

The coronavirus disease 2019 (COVID-19) pandemic has required educators to adapt the in-person objective structured clinical examination (OSCE) to online settings in order for it to remain a critical component of the multifaceted assessment of a student’s competency. This systematic scoping review aimed to summarize the assessment methods and validity and reliability of the measurement tools used in current online OSCE (hereafter, referred to as teleOSCE) approaches. A comprehensive literature review was undertaken following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Articles were eligible if they reported any form of performance assessment, in any field of healthcare, delivered in an online format. Two reviewers independently screened the results and analyzed relevant studies. Eleven articles were included in the analysis. Pre-recorded videos were used in 3 studies, while observations by remote examiners through an online platform were used in 7 studies. Acceptability as perceived by students was reported in 2 studies. This systematic scoping review identified several insights garnered from implementing teleOSCEs, the components transferable from telemedicine, and the need for systemic research to establish the ideal teleOSCE framework. TeleOSCEs may be able to improve the accessibility and reproducibility of clinical assessments and equip students with the requisite skills to effectively practice telemedicine in the future.

Abstract

The coronavirus disease 2019 (COVID-19) pandemic has required educators to adapt the in-person objective structured clinical examination (OSCE) to online settings in order for it to remain a critical component of the multifaceted assessment of a student’s competency. This systematic scoping review aimed to summarize the assessment methods and validity and reliability of the measurement tools used in current online OSCE (hereafter, referred to as teleOSCE) approaches. A comprehensive literature review was undertaken following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Articles were eligible if they reported any form of performance assessment, in any field of healthcare, delivered in an online format. Two reviewers independently screened the results and analyzed relevant studies. Eleven articles were included in the analysis. Pre-recorded videos were used in 3 studies, while observations by remote examiners through an online platform were used in 7 studies. Acceptability as perceived by students was reported in 2 studies. This systematic scoping review identified several insights garnered from implementing teleOSCEs, the components transferable from telemedicine, and the need for systemic research to establish the ideal teleOSCE framework. TeleOSCEs may be able to improve the accessibility and reproducibility of clinical assessments and equip students with the requisite skills to effectively practice telemedicine in the future.

참고문헌 (24)

  1. Lockyer J, "Core principles of assessment in competency-based medical education", Med Teach, 39, pp.609~616 (2017)[DOI]
  2. Romanick-Schmiedl S, "Telemedicine : maintaining quality during times of transition", Nat Rev Dis Primers, 6, pp.45~ (2020)[DOI]
  3. Quinlin L, "Development and implementation of an e-visit objective structured clinical examination to evaluate student ability to provide care by telehealth", J Am Assoc Nurse Pract, 33, pp.359~365 (2020)[DOI]
  4. Tricco AC, "PRISMA Extension for Scoping Reviews(PRISMA-ScR) : checklist and explanation", Ann Intern Med, 169, pp.467~473 (2018)[DOI]
  5. Webb EA, "Improving postgraduate clinical assessment tools : the introduction of video recordings to assess decision making", Med Teach, 34, pp.404~410 (2012)[DOI]
  6. Chan J, "The objective structured clinical examination : can physician-examiners participate from a distance", Med Educ, 48, pp.441~450 (2014)[DOI]
  7. Watson P, "Neurology assessment by objective structured video examination", Clin Teach, 13, pp.348~351 (2016)[DOI]
  8. Isaak R, "Comparing real-time versus delayed video assessments for evaluating ACGME sub-competency milestones in simulated patient care environments", Cureus, 10, pp.e2267~ (2018)[DOI]
  9. Lara S, "Remote assessment of clinical skills during COVID-19 : a virtual, high-stakes, summative pediatric objective structured clinical examination", Acad Pediatr, 20, pp.760~761 (2020)[DOI]
  10. Shehata MH, "A toolbox for conducting an online OSCE", Clin Teach, 18, pp.236~242 (2021)[DOI]
  11. Baribeau DA, "Using an objective structured video exam to identify differential understanding of aspects of communication skills", Med Teach, 34, pp.e242~e250 (2012)[DOI]
  12. Nickel F, "Direct observation versus endoscopic video recording-based rating with the objective structured assessment of technical skills for training of laparoscopic cholecystectomy", Eur Surg Res, 57, pp.1~9 (2016)[DOI]
  13. Bautista JM, "Using video recording in evaluating students’ clinical skills", Med Sci Educ, 27, pp.645~650 (2017)[DOI]
  14. Chen TC, "Remote and onsite scoring of OSCEs using generalisability theory : a three-year cohort study", Med Teach, 41, pp.578~583 (2019)[DOI]
  15. House JB, "Prospective comparison of live evaluation and video review in the evaluation of operator performance in a pediatric emergency airway simulation", J Grad Med Educ, 4, pp.312~316 (2012)[DOI]
  16. St-Onge C, "Validity : one word with a plurality of meanings", Adv Health Sci Educ Theory Pract, 22, pp.853~867 (2017)[DOI]
  17. Hamilton AFC, "Audience effects : what can they tell us about social neuroscience, theory of mind and autism", Cult Brain, 4, pp.159~177 (2016)[DOI]
  18. Khamisa K, "The implementation and evaluation of an e-Learning training module for objective structured clinical examination raters in Canada", J Educ Eval Health Prof, 15, pp.18~ (2018)[DOI]
  19. Sanchez-Ferrer F, "Utility of eye-tracking technology for preparing medical students in Spain for the summative objective structured clinical examination", J Educ Eval Health Prof, 14, pp.27~ (2017)[DOI]
  20. Chong L, "The sights and insights of examiners in objective structured clinical examinations", J Educ Eval Health Prof, 14, pp.34~ (2017)[DOI]
  21. Zaki J, "Cue integration : a common framework for social cognition and physical perception", Perspect Psychol Sci, 8, pp.296~312 (2013)[DOI]
  22. Chong L, "Examiner seniority and experience are associated with bias when scoring communication, but not examination, skills in objective structured clinical examinations in Australia", J Educ Eval Health Prof, 15, pp.17~ (2018)[DOI]
  23. Wilhelm O, "Test battery for measuring the perception and recognition of facial expressions of emotion", Front Psychol, 5, pp.404~ (2014)[DOI]
  24. Moher D, "Preferred reporting items for systematic reviews and meta-analyses : the PRISMA statement", PLoS Med, 6, pp.e1000097~ (2009)[DOI]

관련 논문AI추천

Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in AustraliaDigby Wigram Allen한국보건의료인국가시험원 · Journal of Educational Evaluation for Health Professions2021 · 제18권 Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive studyAniket Bharadwaj, Nimalesh Yogarajah, Vashist Motkur한국보건의료인국가시험원 · Journal of Educational Evaluation for Health Professions2022 · 제19권 Feasibility of clinical performance assessment of medical students on a virtual sub-internship in the United StatesAmit Pahwa, Ariella Apfel, Danelle Cayea, Janet Record, John Woller, Sean Tackett, Shannon Walker한국보건의료인국가시험원 · Journal of Educational Evaluation for Health Professions2021 · 제18권 The efficacy of peer assessment in objective structured clinical examinations for formative feedback: a preliminary study김경지, 김기운한국의학교육학회 · Korean Journal of Medical Education2020 · 제32권 제1호Can disclosure of scoring rubric for basic clinical skills improve objective structured clinical examination?채수진, Ki Hong Chang, Miran Kim한국의학교육학회 · Korean Journal of Medical Education2016 · 제28권 제2호Experience of introducing an electronic health records station in an objective structured clinical examination to evaluate medical students’ communication skills in Canada: a descriptive studyDavid Topps, Debra Pugh, Douglas Archibald, Ilona Bartman, Isabelle Desjardins, Kuan-chin Jean Chen, Melissa Forgie한국보건의료인국가시험원 · Journal of Educational Evaluation for Health Professions2023 · 제20권

이 간행물의 다른 논문 (32건)