Validity and reliability assessment of a peer evaluation method in team-based learning classes
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Purpose: Team-based learning (TBL) is increasingly employed in medical education because of its potential to promote active group learning. In TBL, learners are usually asked to assess the contributions of peers within their group to ensure accountability. The purpose of this study is to assess the validity and reliability of a peer evaluation instrument that was used in TBL classes in a single medical school. Methods: A total of 141 students were divided into 18 groups in 11 TBL classes. The students were asked to evaluate their peers in the group based on evaluation criteria that were provided to them. We analyzed the comments that were written for the highest and lowest achievers to assess the validity of the peer evaluation instrument. The reliability of the instrument was assessed by examining the agreement among peer ratings within each group of students via intraclass correlation coefficient (ICC) analysis. Results: Most of the students provided reasonable and understandable comments for the high and low achievers within their group, and most of those comments were compatible with the evaluation criteria. The average ICC of each group ranged from 0.390 to 0.863, and the overall average was 0.659. There was no significant difference in inter-rater reliability according to the number of members in the group or the timing of the evaluation within the course. Conclusion: The peer evaluation instrument that was used in the TBL classes was valid and reliable. Providing evaluation criteria and rules seemed to improve the validity and reliability of the instrument.
Abstract
Purpose: Team-based learning (TBL) is increasingly employed in medical education because of its potential to promote active group learning. In TBL, learners are usually asked to assess the contributions of peers within their group to ensure accountability. The purpose of this study is to assess the validity and reliability of a peer evaluation instrument that was used in TBL classes in a single medical school. Methods: A total of 141 students were divided into 18 groups in 11 TBL classes. The students were asked to evaluate their peers in the group based on evaluation criteria that were provided to them. We analyzed the comments that were written for the highest and lowest achievers to assess the validity of the peer evaluation instrument. The reliability of the instrument was assessed by examining the agreement among peer ratings within each group of students via intraclass correlation coefficient (ICC) analysis. Results: Most of the students provided reasonable and understandable comments for the high and low achievers within their group, and most of those comments were compatible with the evaluation criteria. The average ICC of each group ranged from 0.390 to 0.863, and the overall average was 0.659. There was no significant difference in inter-rater reliability according to the number of members in the group or the timing of the evaluation within the course. Conclusion: The peer evaluation instrument that was used in the TBL classes was valid and reliable. Providing evaluation criteria and rules seemed to improve the validity and reliability of the instrument.
참고문헌 (16)
- Thompson BM, "Teambased learning at ten medical schools: two years later", Med Educ, 41(3), pp.250~257 (2007)[DOI]
- Koles PG, "The impact of team-based learning on medical students' academic performance", Acad Med, 85(11), pp.1739~1745 (2010)[DOI]
- Sun Huh, "How to Administer the Peer Evaluation in Team-Based Learning", Korean Journal of Medical Education, 24(4), pp.359~361 (2012), Korean Society of Medical Education[DOI]
- Wahawisan J, "Reliability assessment of a peer evaluation instrument in a team-based learning course", Pharm Pract (Granada), 14(1), pp.676~ (2016)[DOI]
- Ferguson KJ, "Assessing the relationship between peer and facilitator evaluations in case-based learning", Med Educ, 41(9), pp.906~908 (2007)[DOI]
- Johnston L, "Assessing contributions to group assignments", Assess Eval High Educ, 29(6), pp.751~768 (2004)[DOI]
- Steensels C, "Peer assessment: a valuable tool to differentiate between student contributions to group work?", Pharm Educ, 6(2), pp.111~118 (2006)[DOI]
- Machado JL, "Self- and peer assessment may not be an accurate measure of PBL tutorial process", BMC Med Educ, 8, pp.55~ (2008)[DOI]
- 유선미, "의과대학생들의 소그룹 발표에 대한 동료평가의 의의", Korean Journal of Medical Education, 26(1), pp.31~40 (2014), 한국의학교육학회[DOI]
- Strumpel CA, "Peer assessment in the team-based learning classroom", pp.2011
- Arnold L, "Use of peer evaluation in the assessment of medical students", J Med Educ, 56(1), pp.35~42 (1981)
- Davis JD, "Comparison of faculty, peer, self, and nurse assessment of obstetrics and gynecology residents", Obstet Gynecol, 99(4), pp.647~651 (2002)
- Dijcks R, "Validity of objective tests towards peer-rated competence by students", Med Teach, 25(3), pp.273~276 (2003)[DOI]
- Falchikov N, "Student peer assessment in higher education: a meta-analysis comparing peer and teacher marks", Rev Educ Res, 70(3), pp.287~322 (2000)[DOI]
- Pocock TM, "The impact of teamwork in peer assessment: a qualitative analysis of a group exercise at a UK medical school", Biosci Educ, 15(1), pp.1~12 (2010)
- Levine RE, "Team-Based Learning for Health Professions Education: A Guide to Using Small Groups for Improving Learning", pp.103~116 (2007), Stylus Publishing LLC