본문 바로가기
KERD
논문KCI 등재

Systematic Review of Predictive Learning Analytics Using Online Learning Engagement Data

한예진|조영환|Florence Martin
한국교육정보미디어학회 (Korea Association for Educational Information and Media)|2022.12|교육정보미디어연구 (Journal of Korean Association for Educational Information and Media)|제28권 제4호|pp. 903-927

0

조회

0

다운로드

3

인용

0

참고문헌

키워드

기계학습Machine learning고등교육온라인 학습Higher educationOnline learningPredictive learning analytics학습예측분석
주제분류교육학

초록

Learning analytics has been effectively used to predict learning performance in online learning environments. On the basis of prediction results, instructors and administrators have made efforts to improve the quality of higher education. However, there is a concern that learning analytics does not guarantee the improvement of teaching and learning activities without an in-depth understanding of educational theory and practice. This study aims to review previous studies on predictive learning analytics (PLA) using online learning engagement data in higher education so as to explore the future direction of PLA. A total of 94 papers, published from 2011 to 2020, were reviewed in regard to (a) research trends, (b) types of online learning engagement, and (c) educational implications. The research on predictive learning analytics has increased rapidly, using the data collected from online learning environments. Nevertheless, there was lack of research analyzing online learning activities across different domains, which might limit a prediction model’s generalizability. In addition, PLA frequently used behavioral, cognitive, and social engagement data to predict learning performance, but not emotional engagement data. There were also limitations in giving prescriptive implications to educational stakeholders based on PLA results, although many studies focused on the accuracy of prediction. These findings imply that interdisciplinary research is necessary not only to predict learning performance accurately but also to interpret and use PLA results for the improvement of higher education.

Abstract

Learning analytics has been effectively used to predict learning performance in online learning environments. On the basis of prediction results, instructors and administrators have made efforts to improve the quality of higher education. However, there is a concern that learning analytics does not guarantee the improvement of teaching and learning activities without an in-depth understanding of educational theory and practice. This study aims to review previous studies on predictive learning analytics (PLA) using online learning engagement data in higher education so as to explore the future direction of PLA. A total of 94 papers, published from 2011 to 2020, were reviewed in regard to (a) research trends, (b) types of online learning engagement, and (c) educational implications. The research on predictive learning analytics has increased rapidly, using the data collected from online learning environments. Nevertheless, there was lack of research analyzing online learning activities across different domains, which might limit a prediction model’s generalizability. In addition, PLA frequently used behavioral, cognitive, and social engagement data to predict learning performance, but not emotional engagement data. There were also limitations in giving prescriptive implications to educational stakeholders based on PLA results, although many studies focused on the accuracy of prediction. These findings imply that interdisciplinary research is necessary not only to predict learning performance accurately but also to interpret and use PLA results for the improvement of higher education.

참고문헌 (54)

  1. Ahmed Al-Azawei, "Predicting Learners' Performance in Virtual Learning Environment (VLE) based on Demographic, Behavioral and Engagement Antecedents", International Journal of Emerging Technologies in Learning (iJET), 15(09), pp.60~ (2020), International Association of Online Engineering (IAOE)[DOI]
  2. Baker, R. S., "Analyzing early at-risk factors in higher education e-learning courses", Proceedings of the 8th International Conference on Educational Data Mining, pp.1~6 (2015)
  3. Jaclyn Broadbent, "Academic success is about self-efficacy rather than frequency of use of the learning management system", Australasian Journal of Educational Technology (2016), Australasian Society for Computers in Learning in Tertiary Education[DOI]
  4. Rebeca Cerezo, "Procrastinating Behavior in Computer-Based Learning Environments to Predict Performance: A Case Study in Moodle", Frontiers in Psychology, 8 (2017), Frontiers Media SA[DOI]
  5. Michelene T. H. Chi, "Translating the ICAP Theory of Cognitive Engagement Into Practice", Cognitive Science, 42(6), pp.1777~1832 (2018), Wiley[DOI]
  6. Connell, J., "Self Process in Development: Minnesota Symposium on Child Psychology", pp.167~216 (1991), Lawrence Erlbaum
  7. Owen Corrigan, "Using Educational Analytics to Improve Test Performance", Proceedings of the 10th European Conference on Technology Enhanced Learning, pp.42~55 (2015)
  8. Ben Daniel, "Big Data and analytics in higher education: Opportunities and challenges", British Journal of Educational Technology, 46(5), pp.904~920 (2014), Wiley[DOI]
  9. Di Mitri, D., "Learning pulse : A machine learning approach for predicting performance in self-regulated learning using multimodal data", Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp.188~197 (2017)
  10. Xu Du, "A systematic meta-Review and analysis of learning analytics research", Behaviour & Information Technology, 40(1), pp.49~62 (2019), Informa UK Limited[DOI]
  11. Dunleavy, J., "What did you do in school today? Exploring the concept of fostering learning", Learning Environment Research, 3, pp.135~158 (2009)
  12. Jennifer A Fredricks, "School Engagement: Potential of the Concept, State of the Evidence", Review of Educational Research, 74(1), pp.59~109 (2004), American Educational Research Association (AERA)[DOI]
  13. Andreas F. Gkontzis, "A predictive analytics framework as a countermeasure for attrition of students", Interactive Learning Environments, 30(6), pp.1028~1043 (2019), Informa UK Limited[DOI]
  14. Lesley Gourlay, "Student Engagement, ‘Learnification’ and the Sociomaterial: Critical Perspectives on Higher Education Policy", Higher Education Policy, 30(1), pp.23~34 (2017), Springer Science and Business Media LLC[DOI]
  15. Gray, G., "An application of classification models to predict learner progression in tertiary education", Proceedings of 2014 IEEE International Advance Computing Conference (IACC), pp.549~554 (2014)
  16. Greller, W., "Translating learning into numbers : A generic framework for learning analytics", Journal of Educational Technology & Society, 15(3), pp.42~57 (2012)
  17. Saeed‐Ul Hassan, "Virtual learning environment to predict withdrawal by leveraging deep learning", International Journal of Intelligent Systems, 34(8), pp.1935~1952 (2019), Wiley[DOI]
  18. Hellas, A., "Predicting academic performance : A systematic literature review", Proceedings companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education, pp.175~199 (2018)
  19. 조일현, "Analysis of Online Behavior and Prediction of Learning Performance in Blended Learning Environments", Educational Technology International, 15(2), pp.71~88 (2014), 한국교육공학회
  20. Ella R. Kahu, "Framing student engagement in higher education", Studies in Higher Education, 38(5), pp.758~773 (2013), Informa UK Limited[DOI]
  21. Khasanah, A. U., "A review of student’s performance prediction using educational data mining techniques", Journal of Engineering and Applied Sciences, 13, pp.5302~5307 (2018)
  22. Dongho Kim, "Toward evidence-based learning analytics: Using proxy variables to improve asynchronous online discussion environments", The Internet and Higher Education, 30, pp.30~43 (2016), Elsevier BV[DOI]
  23. Dongho Kim, "Learning analytics to support self-regulated learning in asynchronous online courses: A case study at a women's university in South Korea", Computers & Education, 127, pp.233~251 (2018), Elsevier BV[DOI]
  24. George D. Kuh, "What We're Learning About Student Engagement From NSSE: Benchmarks for Effective Educational Practices", Change: The Magazine of Higher Learning, 35(2), pp.24~32 (2003), Informa UK Limited[DOI]
  25. Martín Liz-Domínguez, "Systematic Literature Review of Predictive Analysis Tools in Higher Education", Applied Sciences, 9(24), pp.5569~ (2019), MDPI AG[DOI]
  26. Luckin, R., "Designing educational technologies in the age of AI : A learning sciences driven approach", British Journal of Educational Technology, 50(6), pp.2824~2838 (2019)
  27. Katrina A. Meyer, "Student Engagement in Online Learning: What Works and Why", ASHE Higher Education Report, 40(6), pp.1~114 (2014), Wiley[DOI]
  28. S. M. Muthukrishnan, "Systematic mapping review on student’s performance analysis using big data predictive model", Journal of Fundamental and Applied Sciences, 9(4S), pp.730~ (2018), African Journals Online (AJOL)[DOI]
  29. Imani Mwalumbwe, "Using Learning Analytics to Predict Students’ Performance in Moodle Learning Management System: A Case of Mbeya University of Science and Technology", The Electronic Journal of Information Systems in Developing Countries, 79(1), pp.1~13 (2017), Wiley[DOI]
  30. Na, K. S., "Identifying at-risk students in online learning by analysing learning behaviour : A systematic review", Proceedings of 2017 IEEE Conference on Big Data and Analytics(ICBDA), pp.118~123 (2017)
  31. Nespereira, C. G., "Am I failing this course? Risk prediction using e-learning data", Proceedings of the 3rd International Conference on Technological Ecosystems for Enhancing Multiculturality, pp.271~276 (2015)
  32. Palmer, S., "Modelling engineering student academic performance using academic analytics", International Journal of Engineering Education, 29(1), pp.132~138 (2013)
  33. Abelardo Pardo, "Combining University Student Self-Regulated Learning Indicators and Engagement with Online Learning Events to Predict Academic Performance", IEEE Transactions on Learning Technologies, 10(1), pp.82~92 (2017), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
  34. Parsons, J., "Improving student engagement", Current Issues in Education, 14(1) (2011)
  35. Benazir Quadir, "Analyzing the educational goals, problems and techniques used in educational big data research from 2010 to 2018", Interactive Learning Environments, 30(8), pp.1539~1555 (2020), Informa UK Limited[DOI]
  36. Petrea Redmond, "An Online Engagement Framework for Higher Education", Online Learning, 22(1) (2018), The Online Learning Consortium[DOI]
  37. C. Romero, "Educational data mining: A survey from 1995 to 2005", Expert Systems with Applications, 33(1), pp.135~146 (2007), Elsevier BV[DOI]
  38. Cristobal Romero, "Educational data mining and learning analytics: An updated survey", WIREs Data Mining and Knowledge Discovery, 10(3) (2020), Wiley[DOI]
  39. Cristobal Romero, "Web usage mining for predicting final marks of students that use Moodle courses", Computer Applications in Engineering Education, 21(1), pp.135~146 (2010), Wiley[DOI]
  40. Carolyn P. Rosé, "Explanatory learner models: Why machine learning (alone) is not the answer", British Journal of Educational Technology, 50(6), pp.2943~2958 (2019), Wiley[DOI]
  41. Joshua M. Rosenberg, "Context and Technological Pedagogical Content Knowledge (TPACK): A Systematic Review", Journal of Research on Technology in Education, 47(3), pp.186~210 (2015), Informa UK Limited[DOI]
  42. Augusto Sandoval, "Centralized student performance prediction in large courses based on low-cost variables in an institutional context", The Internet and Higher Education, 37, pp.76~89 (2018), Elsevier BV[DOI]
  43. Mohammed Saqr, "How the study of online collaborative learning can guide teachers and predict students’ performance in a medical course", BMC Medical Education, 18(1) (2018), Springer Science and Business Media LLC[DOI]
  44. Amirah Mohamed Shahiri, "A Review on Predicting Student's Performance Using Data Mining Techniques", Procedia Computer Science, 72, pp.414~422 (2015), Elsevier BV[DOI]
  45. Kshitij Sharma, "Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach", British Journal of Educational Technology, 50(6), pp.3004~3031 (2019), Wiley[DOI]
  46. Siemens, G., "Guest editorial-learning and knowledge analytics", Journal of Educational Technology & Society, 15(3), pp.1~2 (2012)
  47. Tal Soffer, "Students' engagement characteristics predict success and completion of online courses", Journal of Computer Assisted Learning, 35(3), pp.378~389 (2019), Wiley[DOI]
  48. Carlos J. Villagrá-Arnedo, "Improving the expressiveness of black-box models for predicting student performance", Computers in Human Behavior, 72, pp.621~631 (2017), Elsevier BV[DOI]
  49. Hajra Waheed, "Predicting academic performance of students from VLE big data using deep learning models", Computers in Human Behavior, 104, pp.106189~ (2020), Elsevier BV[DOI]
  50. "Standards handbook (Version 4.0)" (2017)[URL]
  51. Widyahastuti, F., "Prediction model students’ performance in online discussion forum", Proceedings of the 5th International Conference on Information and Education Technology, pp.6~10 (2017)
  52. Anna Wilson, "Learning analytics: challenges and limitations", Teaching in Higher Education, 22(8), pp.991~1007 (2017), Informa UK Limited[DOI]
  53. Young, M., "Why educators must differentiate knowledge from experience", Journal of the Pacific Circle Consortium for Education, 22(1), pp.9~20 (2010)
  54. Yaqun Zhang, "Using Learning Analytics to Predict Students Performance in Moodle LMS", International Journal of Emerging Technologies in Learning (iJET), 15(20), pp.102~ (2020), International Association of Online Engineering (IAOE)[DOI]

관련 논문AI추천

Analysis of Online Behavior and Prediction of Learning Performance in Blended Learning Environments김정현, 박연정, 송종우, 조일현한국교육공학회 · Educational Technology International2014 · 제15권 제2호Learning Activities and Learning Behaviors for Learning Analytics in e-Learning Environments김윤영, 성은모, 진성희한국교육공학회 · Educational Technology International2016 · 제17권 제2호국내 디지털 교수학습 환경에서의 학습분석 연구 및 데이터 활용에 대한 체계적 문헌 고찰김혜빈, 신안나, 조일현, 표은서한국교육공학회 · 교육공학연구2023 · 제39권 제2호머신러닝 기반의 학업성취 예측 모형 탐색: 대학의 오프라인 수업을 중심으로이종문, 이현우, 차윤미한국교육방법학회 · 교육방법연구2021 · 제33권 제1호The Clustered Patterns of Engagement in MOOCs and Their Effects on Teaching Presence and Learning Persistence이정민, 정연지, Hannah Kim한국콘텐츠학회 · International Journal of Contents2020 · 제16권 제4호교육 데이터와 분석 기법: 사례 연구를 중심으로권영옥사)한국빅데이터학회 · 한국빅데이터학회 학회지2019 · 제4권 제1호

이 간행물의 다른 논문 (8건)

원격교육 유형과 학습자 배경변인에 따른 초등학생의 실재감 지각 차이에 관한 연구박인우, 신민철pp. 873-902온라인교육 확대에 따른 대학생의 학습과 일상의 변화김동심pp. 929-957실시간 온라인 수업에서 대학생의 비디오 사용에 영향을 미치는 변인 연구박서진, 신나민pp. 959-982목표 지향 디자인 방법론에 근거한 메타버스 수업설계 전략 개발 연구임상훈, 이은택pp. 983-1010사이버대학생의 학업성취도 관련 국내 연구동향 분석장은정, 윤정미pp. 1011-1040
원문 보기