A new performance evaluation indicator for the LEE Jong-wook Fellowship Program of Korea Foundation for International Healthcare to better assess its long-term educational impacts: a Delphi study
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Purpose The Dr. LEE Jong-wook Fellowship Program, established by the Korea Foundation for International Healthcare (KOFIH), aims to strengthen healthcare capacity in partner countries. The aim of the study was to develop new performance evaluation indicators for the program to better assess long-term educational impact across various courses and professional roles. Methods A 3-stage process was employed. First, a literature review of established evaluation models (Kirkpatrick’s 4 levels, context/input/process/product evaluation model, Organization for Economic Cooperation and Development Assistance Committee criteria) was conducted to devise evaluation criteria. Second, these criteria were validated via a 2-round Delphi survey with 18 experts in training projects from May 2021 to June 2021. Third, the relative importance of the evaluation criteria was determined using the analytic hierarchy process (AHP), calculating weights and ensuring consistency through the consistency index and consistency ratio (CR), with CR values below 0.1 indicating acceptable consistency. Results The literature review led to a combined evaluation model, resulting in 4 evaluation areas, 20 items, and 92 indicators. The Delphi surveys confirmed the validity of these indicators, with content validity ratio values exceeding 0.444. The AHP analysis assigned weights to each indicator, and CR values below 0.1 indicated consistency. The final set of evaluation indicators was confirmed through a workshop with KOFIH and adopted as the new evaluation tool. Conclusion The developed evaluation framework provides a comprehensive tool for assessing the long-term outcomes of the Dr. LEE Jong-wook Fellowship Program. It enhances evaluation capabilities and supports improvements in the training program’s effectiveness and international healthcare collaboration.
Abstract
Purpose The Dr. LEE Jong-wook Fellowship Program, established by the Korea Foundation for International Healthcare (KOFIH), aims to strengthen healthcare capacity in partner countries. The aim of the study was to develop new performance evaluation indicators for the program to better assess long-term educational impact across various courses and professional roles. Methods A 3-stage process was employed. First, a literature review of established evaluation models (Kirkpatrick’s 4 levels, context/input/process/product evaluation model, Organization for Economic Cooperation and Development Assistance Committee criteria) was conducted to devise evaluation criteria. Second, these criteria were validated via a 2-round Delphi survey with 18 experts in training projects from May 2021 to June 2021. Third, the relative importance of the evaluation criteria was determined using the analytic hierarchy process (AHP), calculating weights and ensuring consistency through the consistency index and consistency ratio (CR), with CR values below 0.1 indicating acceptable consistency. Results The literature review led to a combined evaluation model, resulting in 4 evaluation areas, 20 items, and 92 indicators. The Delphi surveys confirmed the validity of these indicators, with content validity ratio values exceeding 0.444. The AHP analysis assigned weights to each indicator, and CR values below 0.1 indicated consistency. The final set of evaluation indicators was confirmed through a workshop with KOFIH and adopted as the new evaluation tool. Conclusion The developed evaluation framework provides a comprehensive tool for assessing the long-term outcomes of the Dr. LEE Jong-wook Fellowship Program. It enhances evaluation capabilities and supports improvements in the training program’s effectiveness and international healthcare collaboration.
참고문헌 (14)
- Saaty TL, "A scaling method for priorities in hierarchical structures", J Math Psychol, 15, pp.234~281 (1977)[DOI]
- Saaty TL, "Mathematical models for decision support", pp.109~121 (1988), Springer
- Saaty TL, "Fundamentals of decision making and priority theory with the analytic hierarchy process" (1994), RWS Publications
- Ayre C, "Critical values for Lawshe’s content validity ratio : revisiting the original methods of calculation", Meas Eval Counsel Dev, 47, pp.79~86 (2014)[DOI]
- Lawshe CH, "A quantitative approach to content validity", Pers Psychol, 28, pp.563~575 (1975)[DOI]
- Kirkpatrick D, "Evaluating training programs : the four levels" (2006), Berrett-Koehler Publishers
- Kellaghan T, "International handbook of educational evaluation: part one: perspectives" (2003), Springer
- Bickman L, "The functions of program theory", New Dir Eval, 1987, pp.5~18 (1987)[DOI]
- Frye AW, "Program evaluation models and related theories: AMEE guide no. 67", Med Teach, 34, pp.e288~e299 (2012)[DOI]
- Chianca T, "The OECD", DAC criteria for international development evaluations : an assessment and ideas for improvement, 2008, pp.5, 41~51
- Ju H, "Adapting an integrated program evaluation for promoting competency-based medical education", Korean Med Educ Rev, 23, pp.56~67 (2021)[DOI]
- Lee C, "Performance analysis of National Human Resource Development Consortium Project using logic model", J Vocat Educ Res, 31, pp.195~216 (2012)
- Lee Y, "The diagnostic analysis of consortium training programs for small and middle-sized corporations, using CIPP evaluation model", Korean J Hum Resour Dev Q, 11, pp.91~113 (2009)[DOI]
- Bae J, "Evaluation of effectiveness on KOICAs training program: focus on KOICAs Country Training Program for Tanzania", Int Dev Coop Rev, 3, pp.228~255 (2011)