본문 바로가기
KERD
논문KCI 등재

Impacts of Badges and Leaderboards on Academic Performance: A Meta-Analysis

이수영|김아름
한국교육공학회 (Korean Society Of Educational Technology)|2022.10|Educational Technology International|제23권 제2호|pp. 207-237

0

조회

0

다운로드

0

인용

0

참고문헌

키워드

GamificationMeta-analysisOnline learningBadgeLeaderboard
주제분류교육학

초록

As technological changes continue to accelerate every day, meeting the needs of a shifting educational landscape requires leaving an exclusively “in-person” education behind. Gamified learning environments should be carefully designed in light of conflicting studies to suit students’ needs. The purpose of this meta-analysis is to draw conclusive results regarding the application of the most commonly used game elements in education, i.e., badges and leaderboards, through a comprehensive analysis of their impact on academic performance in online learning. Review Manager (RevMan 5.4) was used to analyze eligible studies selected from Emerald, SAGE, ERIC, EBSCO, and ProQuest between January 2011 and January 2022. Analyzing 37 studies found that using leaderboards and badges in online education enhanced academic performance when compared to traditional learning without gamification (SMD = 0.39). The badge-only intervention showed a larger effect size (SMD = 0.33) than the leaderboard-only intervention (SMD = 0.27). Badges and leaderboards together exhibited a larger effect size (SMD = 0.48) than individual game elements (SMD = 0.40). The impact of the game elements on academic performance was greater in the humanities (SMD = 0.51) than in STEM fields (SMD = 0.32) and was greater for K-12 students (SMD = 0.63) than for college students (SMD = 0.31). This study contributes to a timely discussion of the use of badges and leaderboards in COVID-19 online learning trends and provides relevant data for designing integrations of online education and gamification models.

Abstract

As technological changes continue to accelerate every day, meeting the needs of a shifting educational landscape requires leaving an exclusively “in-person” education behind. Gamified learning environments should be carefully designed in light of conflicting studies to suit students’ needs. The purpose of this meta-analysis is to draw conclusive results regarding the application of the most commonly used game elements in education, i.e., badges and leaderboards, through a comprehensive analysis of their impact on academic performance in online learning. Review Manager (RevMan 5.4) was used to analyze eligible studies selected from Emerald, SAGE, ERIC, EBSCO, and ProQuest between January 2011 and January 2022. Analyzing 37 studies found that using leaderboards and badges in online education enhanced academic performance when compared to traditional learning without gamification (SMD = 0.39). The badge-only intervention showed a larger effect size (SMD = 0.33) than the leaderboard-only intervention (SMD = 0.27). Badges and leaderboards together exhibited a larger effect size (SMD = 0.48) than individual game elements (SMD = 0.40). The impact of the game elements on academic performance was greater in the humanities (SMD = 0.51) than in STEM fields (SMD = 0.32) and was greater for K-12 students (SMD = 0.63) than for college students (SMD = 0.31). This study contributes to a timely discussion of the use of badges and leaderboards in COVID-19 online learning trends and provides relevant data for designing integrations of online education and gamification models.

참고문헌 (88)

  1. Ahmad, A., "The impact of gamification on learning outcomes of computer science majors", ACM Transactions on Computing Education, 20(2), pp.1~25 (2020)
  2. Andrade, F. R., "The bright and dark sides of gamification", Intelligent Tutoring Systems, 9864, pp.176~186 (2016)
  3. Auvinen, T., "Increasing students’ awareness of their behavior in online learning environments with visualizations and achievement badges", IEEE Transactions on Learning Technologies, 8(3), pp.261~273 (2015)
  4. Balci, S., "Comparing the effectiveness of badges and leaderboards on academic performance and motivation of students in fully versus partially gamified online physics classes", Education and Information Technologies, 27(6), pp.8669~8704 (2022)
  5. Barata, G., "Identifying student types in a gamified learning experience", International Journal of Game-Based Learning, 4(4), pp.19~36 (2014)
  6. Baydas, O., "The examination of the gamification process in undergraduate education: A scale development study", Technology, Pedagogy and Education, 28(3), pp.269~285 (2019)
  7. Bernik, A., "A pilot study of the influence of gamification on the effectiveness of an e-Learning course", 26th Central European Conference on Information and Intelligent Systems (CECIIS 2015), pp.73~79 (2015)
  8. Çakıroğlu, Ü, "Enhancing statistical literacy skills through real life activities enriched with gamification elements : An experimental study", ELearning and Digital Media, 18(5), pp.441~459 (2021)
  9. Chans, G. M., "Gamification as a strategy to increase motivation and engagement in higher education chemistry students", Computers, 10(10), pp.132~ (2021)
  10. Chen, C. -C., "Gamify online courses with tools built into your learning management system(LMS)to enhance selfdetermined and active learning", Online Learning, 22(3), pp.41~54 (2018)
  11. Christy, K. R., "Leaderboards in a virtual classroom : A test of stereotype threat and social comparison explanations for women’s math performance", Computers & Education, 78, pp.66~77 (2014)
  12. Cohen, J., "Statistical power analysis for the behavioral sciences" (2013), Taylor and Francis
  13. Crabtree, J., "Self-evaluation and social comparison amongst adolescents with learning difficulties", Journal of Community & Applied Social Psychology, 11(5), pp.347~359 (2001)
  14. Denny, P., "Empirical support for a causal relationship between gamification and learning outcomes", Proceedings of the 2018 CHI conference on human factors in computing systems, pp.1~13 (2018)
  15. Dichev, C., "Gamifying education : What is known, what is believed and what remains uncertain : A critical review", International Journal of Educational Technology in Higher Education, 14(1), pp.9~ (2017)
  16. Ding, L., "Applying gamifications to asynchronous online discussions: A mixed methods study", Computers in Human Behavior, 91, pp.1~11 (2019)
  17. Domínguez, A., "Gamifying learning experiences : Practical implications and outcomes", Computers & Education, 63, pp.380~392 (2013)
  18. Fanfarelli, J. R., "Designing effective digital badges: Applications for learning" (2019), Routledge
  19. Festinger, L., "A theory of social comparison processes", Human Relations, 7(2), pp.117~140 (1954)
  20. Furdu, I., "Pros and cons gamification and gaming in classroom", Broad Research in Artificial Intelligence and Neuroscience, 8(2), pp.56~62 (2017)
  21. Ghasia, M., "Micro-credentials in higher education institutions : An exploratory study of its place in Tanzania", The International Journal of Education and Development Using Information and Communication Technology, 15(1), pp.219~230 (2019)
  22. Goosby, C. L, "Using Digital Badges to Impact First Grade Math Achievement: A Quasi-experimental Study of Digital Badges and the North Carolina Early Numeracy Skills Indicators (Publication No. 28318393)", doctor, pp.2020
  23. Goudeau, S., "Why lockdown and distance learning during the COVID-19 pandemic are likely to increase the social class achievement gap", Nature Human Behaviour, 5(10), pp.1273~1281 (2021)
  24. Greitzer, F. L., "Cognitive science implications for enhancing training effectiveness in a serious gaming context", Journal on Educational Resources in Computing, 7(3), pp.2~ (2007)
  25. Gündüz, A. Y., "Effectiveness of gamification in flipped learning", SAGE Open, 10(4), pp.215824402097983~ (2020)
  26. Hakulinen, L., "The effect of gamification on students with different achievement goal orientations", Proceedings of the 2014 international conference on teaching and learning in computing and engineering, pp.9~16 (2014)
  27. Hakulinen, L., "The effect of achievement badges on students’ behavior : An empirical study in a university-level computer science course", International Journal of Emerging Technologies in Learning, 10(1), pp.18~30 (2015)
  28. Hamari, J., "Do badges increase user activity? A field experiment on the effects of gamification", Computers in Human Behavior, 71, pp.469~478 (2017)
  29. Hanus, M. D., "Assessing the effects of gamification in the classroom : A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance", Computers and Education, 80, pp.152~161 (2015)
  30. Heinert, S. W., "Engaging youth through digital badges to promote health in underserved communities", Health promotion practice, 22(5), pp.631~637 (2021)
  31. Foon, H. K., "Shaping the future of education, communication and technology. Educational communications and technology yearbook", pp.71~90 (2019), Springer
  32. Higgins, J. P. T., "Measuring inconsistency in meta-analyses", British Medical Journal, 327(7414), pp.557~560 (2003)
  33. Homer, R., "Comparing digital badges-and-points with classroom token systems : Effects on elementary school ESL students’classroom behavior and English learning", Journal of Educational Technology &Society, 21(1), pp.137~151 (2018)
  34. Huang, B., "Do points, badges and leaderboard increase learning and activity: A quasi-experiment on the effects of gamification", Proceedings of the 23rd International Conference on Computers in Education, pp.275~280 (2015)
  35. Huang, R., "The impact of gamification in educational settings on student learning outcomes : A meta-analysis", Educational Technology Research and Development, 68(4), pp.1875~1901 (2020)
  36. Jagušt, T., "Examining competitive, collaborative and adaptive gamification in young learners’ math learning", Computers & Education, 125, pp.444~457 (2018)
  37. Jensen, J. C., "Virtualizing cyber-physical systems:Bringing CPS to online education", Proceedings of the First Workshop on CPS Education (CPS-Ed), pp.1~5 (2013)
  38. Kapsalis, G. D., "Kahoot! as a formative assessment tool in foreign language learning : A case study in greek as an L2", Theory and Practice in Language Studies, 10(11), pp.1343~1350 (2020)
  39. Kivunja, C., "Theoretical perspectives of how digital natives learn", International Journal of Higher Education, 3(1), pp.94~109 (2014)
  40. Kim, J., "Effects of gamification on behavioral change in education : A meta-analysis", International Journal of Environmental Research and Public Health, 18(7), pp.3550~ (2021)
  41. Kim, J. T., "Dynamical model for gamification of learning(DMGL)", Multimedia Tools and Applications, 74(19), pp.8483~8493 (2015)
  42. 김종우, "A Comparative Study on the Differences in Cultural Attributes of Gamification between Korea and Austria", 한국산업정보학회논문지, 23(1), pp.109~122 (2018), 한국산업정보학회
  43. Kusuma, G. P., "Analysis of gamification models in education using MDA framework", Procedia Computer Science, 135, pp.385~392 (2018)
  44. Kyewski, E., "To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course", Computers & Education, 118(1), pp.25~37 (2018)
  45. Lam, Y. W., "Improving argumentative writing : Effects of a blended learning approach and gamification", Language learning &technology, 22(1), pp.97~118 (2018)
  46. Landers, R. N., "Gamification of task performance with leaderboards : A goal setting experiment", Computers in Human Behavior, 71, pp.508~515 (2017)
  47. Legaki, N. Z., "The effect of challenge-based gamification on learning : An experiment in the context of statistics education", International Journal of Human-Computer Studies, 144, pp.102496~ (2020)
  48. Looyestyn, J., "Does gamification increase engagement with online programs? A systematic review", PloS One, 12(3), pp.e0173403~ (2017)
  49. Mah, D. K., "Learning analytics and digital badges: Potential impact on student retention in higher education", Technology, Knowledge and Learning, 21(3), pp.285~305 (2016)
  50. Manzano-León, A., "Between level up and game over : A systematic literature review of gamification in education", Sustainability, 13(4), pp.2247~ (2021)
  51. Margolis, H., "Improving self-efficacy and motivation : What to do, what to say", Intervention in School and Clinic, 41(4), pp.218~227 (2006)
  52. Maslow, A. H., "Motivation and Personality" (1954), Harper & Row Publishers
  53. Mazarakis, A., "Gamification is working, but which one exactly? Results from an experiment with four game design elements", International Journal of Human–Computer Interaction, pp.1~16 (2022)
  54. McGonigal, J., "Reality is broken: Why games make us better and how they can change the world" (2011), Penguin Press
  55. Mekler, E. D., "Towards understanding the effects of individual gamification elements on intrinsic motivation and performance", Computers in Human Behavior, 71, pp.525~534 (2017)
  56. Morris, B. J., "Comparing badges and learning goals in low-and high-stakes learning contexts", Journal of Computing in Higher Education, 31(3), pp.573~603 (2019)
  57. Newby, T. J., "Instructional digital badges : Effective learning tools", Educational Technology Research and Development, 68(3), pp.1053~1067 (2020)
  58. Nieto-Escamez, F. A., "Gamification as online teaching strategy during COVID-19 : A mini-review", Frontiers in Psychology, 12, pp.648552~ (2021)
  59. Ongoro, C. A., "Effects of digital games on enhancing language learning in Tanzanian preschools", Knowledge Management & E-Learning, 11(3), pp.325~344 (2019)
  60. Özdener, N, "Gamification for enhancing Web 2.0 based educational activities: The case of pre-service grade school teachers using educational Wiki pages", Telematics and Informatics, 35(3), pp.564~578 (2018)
  61. Poondej, C., "Gamification in e-learning : A Moodle implementation and its effect on student engagement and performance", Interactive Technology and Smart Education, 17(1), pp.56~66 (2019)
  62. Pursell, D. P., "Adapting to student learning styles : Engaging students with cell phone technology in organic chemistry instruction", Journal of Chemical Education, 86(10), pp.1219~ (2009)
  63. Ritzhaupt, A. D., "A meta-analysis on the influence of gamification in formal educational settings on affective and behavioral outcomes", Educational Technology Research and Development, 69(5), pp.2493~2522 (2021)
  64. Sailer, M., "How gamification motivates : An experimental study of the effects of specific game design elements on psychological need satisfaction", Computers in Human Behavior, 69, pp.371~380 (2017)
  65. Sailer, M., "The gamification of learning : A meta-analysis", Educational Psychology Review, 32(1), pp.77~112 (2020)
  66. Sailer, M., "Gamification of inclass activities in flipped classroom lectures", British Journal of Educational Technology, 52(1), pp.75~90 (2021)
  67. Saleem, A. N., "Gamification applications in Elearning:A literature review", Technology, Knowledge and Learning, 27, pp.1~21 (2021)
  68. Sarkar, N., "Engaging digital natives through social learning", Systemics, Cybernetics and Informatics, 15(2), pp.1~4 (2017)
  69. Schardt, C., "Utilization of the PICO framework to improve searching PubMed for clinical questions", BMC medical informatics and decision making, 7(1), pp.1~6 (2007)
  70. Seaborn, K., "Gamification in theory and action : A survey", International Journal of Human-Computer Studies, 74, pp.14~31 (2015)
  71. Silpasuwanchai, C., "Developing a comprehensive engagement framework of gamification for reflective learning", Proceedings of the 2016 ACM Conference on Designing Interactive Systems, pp.459~472 (2016)
  72. Stetson-Tiligadas, S. M, "The impact of digital achievement badges on undergraduate learner motivation (Publication No. 10037460)", Doctor, pp.2016
  73. Strmečki, D., "Gamification in E-learning : Introducing gamified design elements into E-learning Systems", Journal of Computer Science, 11(12), pp.1108~1117 (2015)
  74. Tahir, F., "Investigating the causal relationships between badges and learning outcomes in SQL-Tutor", Research and Practice in Technology Enhanced Learning, 17(1), pp.7~ (2022)
  75. Teasley, S. D., "Student facing dashboards: One size fits all?", Technology, Knowledge and Learning, 22(3), pp.377~384 (2017)
  76. Tsay, C. H. -H., "Enhancing student learning experience with technology-mediated gamification : An empirical study", Computers & Education, 121, pp.1~17 (2018)
  77. Turan, Z., "Gamification and education : Achievements, cognitive loads, and views of students", International journal of emerging technologies in learning, 11(7) (2016)
  78. Uanhoro, J., "Investigation of the effect of badges in the online homework system for undergraduate general physics course", Education Sciences, 12(3), pp.217~ (2022)
  79. Ugur-Erdogmus, F., "Effect of gamified mobile applications and the role of player types on the achievement of students", Journal of Educational Computing Research, 60(4), pp.1063~1080 (2022)
  80. Uz Bilgin, C., "Investigating the effectiveness of gamification on group cohesion, attitude, and academic achievement in collaborative learning environments", TechTrends, 64(1), pp.124~136 (2019)
  81. Von Stumm, S., "Socioeconomic status amplifies the achievement gap throughout compulsory education independent of intelligence", Intelligence, 60, pp.57~62 (2017)
  82. Watson-Huggins, J, "An experimental study on the effects of a gamified software intervention in mathematics achievement among sixth grade students (Publication No. 13820081)", Doctor, pp.2018
  83. Werbach, K., "For the win: How game thinking can revolutionize your business" (2012), Wharton School Press
  84. Xu, J., "Psychological interventions of virtual gamification within academic intrinsic motivation : A systematic review", Journal of Affective Disorders, 293, pp.444~465 (2021)
  85. Zainuddin, Z., "Students’ learning performance and perceived motivation in gamified flipped-class instruction", Computers & Education, 126, pp.75~88 (2018)
  86. Zhou, T., "Genome-wide analysis of human hotspot intersected genes highlights the roles of meiotic recombination in evolution and disease", BMC Genomics, 14(1) (2013)
  87. Zimmerling, E., "Exploring the influence of common game elements on ideation output and motivation", Journal of Business Research, 94, pp.302~312 (2019)
  88. Zichermann, G., "Gamification by design: Implementing game mechanics in web and mobile apps" (2011), O'Reilly Media, Inc

관련 논문AI추천

Proposing a Badge System Based on Gamification Theories for a Digital Learning Environment김영식, 정복문교육연구소 · 교육연구논총2017 · 제38권 제4호Perceptions of Students for Gamified e-Quizzes: A Comparative Case Study오경미한국멀티미디어언어교육학회 · 멀티미디어 언어교육2021 · 제24권 제3호A Study of Gamified e-Quizzes for Online Student Engagement during COVID-19오경미한국멀티미디어언어교육학회 · 멀티미디어 언어교육2020 · 제23권 제3호Gamified Formative Assessment in a Flipped Class오경미한국멀티미디어언어교육학회 · 멀티미디어 언어교육2022 · 제25권 제1호Gamification as a Helping Hand for Students’ Learning Adaptation due to Covid-19 PandemicBambang Yudi Cahyono, Rida Afrilyasanti아시아테플 · Journal of Asia TEFL2022 · 제19권 제4호Interaction of Learning Motivation with Dashboard Intervention and Its Effect on Learning Achievement김정현, 박연정, 조일현, 허다미한국교육공학회 · Educational Technology International2017 · 제18권 제2호

이 간행물의 다른 논문 (3건)

Interactive Video Player for Supporting Learner Engagement in Video-Based Online LearningHua ZHENG, 윤미현, 조일현pp. 129-155Application of AIG Implemented within CLASS Software for Generating Cognitive Test Item Models류현석, 사승연, 류지훈pp. 157-181Development of Interactive Mobile Learning Media on Teaching Terms of Mental Status Examination (MSE) for Nursing StudentsDjoko PRIYONO, Irma YANTI, M. Ali MAULANA, Triyana Harlia PUTRI, Thoriq Tri PRABOWOpp. 183-205
원문 보기