Insights into undergraduate medical student selection tools: a systematic review and meta-analysis
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Purpose Evaluating medical school selection tools is vital for evidence-based student selection. With previous reviews revealing knowledge gaps, this meta-analysis offers insights into the effectiveness of these selection tools. Methods A systematic review and meta-analysis were conducted applying the following criteria: peer-reviewed articles available in English, published from 2010 and which include empirical data linking performance in selection tools with assessment and dropout outcomes of undergraduate entry medical programs. Systematic reviews, meta-analyses, general opinion pieces, or commentaries were excluded. Effect sizes (ESs) of the predictability of academic and clinical performance within and by the end of the medicine program were extracted, and the pooled ESs were presented. Results Sixty-seven out of 2,212 articles were included, which yielded 237 ESs. Previous academic achievement predicted medical program academic performance (Cohen’s d=0.697 in early program; 0.619 in end of program) and clinical exams (0.545 in end of program). Overall aptitude tests predicted academic achievement in both the early and last years (0.550 and 0.371, respectively), as well as the end of program clinical exams (0.448). Within aptitude tests, verbal reasoning and quantitative reasoning best predicted academic achievement in the early program (0.704 and 0.643, respectively). Panel interviews showed no significant effect. However, multiple mini-interviews’ effects on early clinical exams and academic performance, as well as situational judgement tests (SJT)’s effect on early academic performance, were statistically significant. Conclusion Current evidence suggests that learning outcomes are predicted by previous academic achievement and aptitude tests. The predictive value of SJT and topics such as selection algorithms, features of interview (e.g., content of the questions) and the way the interviewers’ reports are used, warrant further research.
Abstract
Purpose Evaluating medical school selection tools is vital for evidence-based student selection. With previous reviews revealing knowledge gaps, this meta-analysis offers insights into the effectiveness of these selection tools. Methods A systematic review and meta-analysis were conducted applying the following criteria: peer-reviewed articles available in English, published from 2010 and which include empirical data linking performance in selection tools with assessment and dropout outcomes of undergraduate entry medical programs. Systematic reviews, meta-analyses, general opinion pieces, or commentaries were excluded. Effect sizes (ESs) of the predictability of academic and clinical performance within and by the end of the medicine program were extracted, and the pooled ESs were presented. Results Sixty-seven out of 2,212 articles were included, which yielded 237 ESs. Previous academic achievement predicted medical program academic performance (Cohen’s d=0.697 in early program; 0.619 in end of program) and clinical exams (0.545 in end of program). Overall aptitude tests predicted academic achievement in both the early and last years (0.550 and 0.371, respectively), as well as the end of program clinical exams (0.448). Within aptitude tests, verbal reasoning and quantitative reasoning best predicted academic achievement in the early program (0.704 and 0.643, respectively). Panel interviews showed no significant effect. However, multiple mini-interviews’ effects on early clinical exams and academic performance, as well as situational judgement tests (SJT)’s effect on early academic performance, were statistically significant. Conclusion Current evidence suggests that learning outcomes are predicted by previous academic achievement and aptitude tests. The predictive value of SJT and topics such as selection algorithms, features of interview (e.g., content of the questions) and the way the interviewers’ reports are used, warrant further research.
참고문헌 (31)
- Razack S, "‘Meritocracy’ and ‘fairness’ in medical student selection : comparing UK and Australia", Med Educ, 55, pp.772~774 (2021)[DOI]
- Kreiter C, "A meta-analytic perspective on the valid use of subjective human judgement to make medical school admission decisions", Med Educ Online, 23, pp.1522225~ (2018)[DOI]
- Patterson F, "How effective are selection methods in medical education? : a systematic review", Med Educ, 50, pp.36~60 (2016)[DOI]
- Lin JC, "Best practices for interviewing applicants for medical school admissions : a systematic review", Perspect Med Educ, 11, pp.239~246 (2022)[DOI]
- Webster ES, "Situational judgement test validity for selection : a systematic review and meta-analysis", Med Educ, 54, pp.888~902 (2020)[DOI]
- Goho J, "The effectiveness of academic admission interviews : an exploratory meta-analysis", Med Teach, 28, pp.335~340 (2006)[DOI]
- Callwood A, "Personal domains assessed in multiple mini interviews(MMIs)for healthcare student selection : a narrative synthesis systematic review", Nurse Educ Today, 64, pp.56~64 (2018)[DOI]
- Greatrix R, "Does the UKCAT predict performance in medical and dental school? : a systematic review", BMJ Open, 11, pp.e040128~ (2021)[DOI]
- Bala L, "Assessing the predictive validity of the UCAT : a systematic review and narrative synthesis", Med Teach, 44, pp.401~409 (2022)[DOI]
- Ali S, "Multiple mini-interviews : current perspectives on utility and limitations", Adv Med Educ Pract, 10, pp.1031~1038 (2019)[DOI]
- Patterson F, "Situational judgement tests in medical education and training: research, theory and practice: AMEE guide no. 100", Med Teach, 38, pp.3~17 (2016)[DOI]
- Simpson PL, "Predictive validity of a new integrated selection process for medical school admission", BMC Med Educ, 14, pp.86~ (2014)[DOI]
- Page MJ, "The PRISMA 2020 statement : an updated guideline for reporting systematic reviews", BMJ, 372, pp.n71~ (2021)[DOI]
- Zhu J, "A tale of two databases : the use of Web of Science and Scopus in academic papers", Scientometrics, 123, pp.321~335 (2020)[DOI]
- Tufanaru C, "Fixed or random effects meta-analysis? : common methodological issues in systematic reviews of effectiveness", Int J Evid Based Healthc, 13, pp.196~207 (2015)[DOI]
- Agarwal A, "Methods commentary: risk of bias in cross-sectional surveys of attitudes and practices", McMaster University, ~ (2019)
- Luder A, "Active education for future doctors", pp.67~78 (2020), Springer
- Lovakov A, "Empirically derived guidelines for effect size interpretation in social psychology", Eur J Soc Psychol, 51, pp.485~504 (2021)[DOI]
- Shulruf B, "The efficacy of medical student selection tools in Australia and New Zealand", Med J Aust, 208, pp.214~218 (2018)[DOI]
- Shulruf B, "Selecting top candidates for medical school selection interviews : a non-compensatory approach", BMC Med Educ, 20, pp.113~ (2020)[DOI]
- Sladek RM, "Predicting success in medical school : a longitudinal study of common Australian student selection tools", BMC Med Educ, 16, pp.187~ (2016)[DOI]
- Hurley KF, "Probing the effect of OSCE checklist length on inter-observer reliability and observer accuracy", Med Educ Online, 20, pp.29242~ (2015)[DOI]
- Sim JH, "A closer look at checklist scoring and global rating for four OSCE stations: do the scores correlate well?", Educ Med J, 7, pp.e39~e44 (2015)[DOI]
- O’Sullivan DM, "Medical school selection criteria as predictors of medical student empathy : a cross-sectional study of medical students, Ireland", BMJ Open, 7, pp.e016076~ (2017)[DOI]
- Kumar N, "Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom : a cohort study", Adv Med Educ Pract, 9, pp.239~245 (2018)[DOI]
- Reiter H, "Vive la Différence : the freedom and inherent responsibilities when designing and implementing multiple mini-interviews", Acad Med, 93, pp.969~971 (2018)[DOI]
- Towaij C, "Trust me, I know them : assessing interpersonal bias in surgery residency interviews", J Grad Med Educ, 14, pp.289~294 (2022)[DOI]
- Griffin B, "Multiple mini interviews : revealing similarities across institutions", BMC Med Educ, 18, pp.190~ (2018)[DOI]
- Shulruf B, "Medical student selection process enhanced by improving selection algorithms and changing the focus of interviews in Australia : a descriptive study", J Educ Eval Health Prof, 19, pp.31~ (2022)[DOI]
- Dolan BM, "Establishing trust when assessing learners : barriers and opportunities", Acad Med, 94, pp.1851~1853 (2019)[DOI]
- Moltner A, "The reliability of the pass", fail decision for assessments comprised of multiple components, 2015, pp.32, DOC42~