본문 바로가기
KERD
논문KCI 등재

다중양식 학습분석의 연구동향과 발전방안 탐색을 위한 체계적 문헌 고찰

Systematic Literature Review to Explore Research Trends and Future Directions of Multimodal Learning Analytics

조영환|이한솔|김혜은|함윤희
한국교육정보미디어학회 (Korea Association for Educational Information and Media)|2021.06|교육정보미디어연구 (Journal of Korean Association for Educational Information and Media)|제27권 제2호|pp. 501-529

0

조회

0

다운로드

11

인용

0

참고문헌

키워드

체계적 문헌 고찰머신러닝Machine LearningSystematic Literature ReviewEducational Data Mining맞춤형 학습교육 데이터 마이닝Personalized Learning다중양식 학습분석Multimodal Learning Analytics
주제분류교육학

초록

다양한 자원으로부터 수집된 학습 데이터를 분석하여 학습과정을 이해하고 수업을 개선하고 지능형 학습 도구를 개발하기 위해 다중양식 학습분석이 활발하게 이루어지고 있다. 본 연구는 2011년에서 2020년까지 출판된 문헌을 체계적으로 고찰하여 다중양식 학습분석의 최근 동향과 제한점을 분석하고 발전방안을 탐색하고자 한다. 이를 위해 총 36편의 국내외 논문을 선정하고 다중양식 학습분석의 연구 목적, 연구 맥락, 학습 요소, 데이터 수집과 분석, 제한점에 대해서 선행연구와 전문가 검토로부터 도출된 분석틀을 활용하여 체계적으로 문헌을 검토하였다. 다중양식 학습분석은 주로 학습결과에 영향을 미치는 요인과 학습 패턴을 탐색하려는 목적으로 이루어졌으며, 성인 학습자를 대상으로 실제적인 수업 환경보다는 통제된 실험실 맥락에서 많이 이루어졌다. 학습의 인지적 요소와 상호작용적 요소에 대한 연구가 다수 있고, 자기조절학습에 초점을 둔 연구는 매우 적은 편이다. 다중양식 학습분석을 위해 웹 데이터, 음성, 설문 자료, 시선, 생리심리반응 등을 수집하여 전통적인 통계분석과 머신러닝을 이용한 분류분석을 주로 실시하였다. 다중양식 학습분석의 제한점으로는 적은 샘플 수, 데이터 노이즈와 같은 데이터 수집 및 분석과 관련된 이슈가 주로 언급되었으며, 데이터 활용과 윤리적 문제에 대한 논의는 매우 부족하였다. 본 연구는 다중양식 학습분석이 여러 제한점에도 불구하고 전통적인 연구 방법으로 쉽게 관찰하기 어려운 학습의 다양한 측면을 드러낸다는 점에서 학습과 학습자에 대한 깊이 있는 이해를 촉진할 수 있다는 것을 보여준다. 향후 데이터 수집과 분석 방법을 개선하기 위한 노력과 함께 맞춤형 학습환경에서 교수 및 학습 활동을 개선하기 위해 다중양식 학습분석을 효과적이고 윤리적으로 활용하기 위한 연구와 실천이 필요하다.

Abstract

Multimodal learning analytics (MLA) is actively conducted to understand learning process, improve instructional practice, and develop intelligent learning tools through analyzing learning data collected from multiple sources. The purpose of this study is to explore the research trends and future directions of MLA through systematically reviewing previous studies published from 2011 to 2020. This study reviewed 36 articles of MLA with a coding scheme about research purposes, research contexts, learning elements, data collection and analysis, and limitations. This study found that MLA was mainly conducted to explore learning patterns and factors to influence learning outcomes. Adult learners were likely to participate in the research of MLA in well-controlled experiment contexts rather than authentic instructional ones. Many studies focused on cognitive and interactive elements of learning, whereas there was a lack of research on MLA for self-regulated learning. MLA studies collected web, audio, survey, eye tracking, physiological data, etc., and analyzed them mainly with statistical and classification analyses. The limitations of MLA were pointed out in regard to data collection and analysis (e.g., small sample size, data noise) rather than data usage and ethical problems. The findings imply that MLA can enhance an in-depth understanding of learning and learners because it reveals diverse aspects of learning that are not easily identified through traditional research methods. Future research is necessary not only to improve data collection and analysis methods but also to use MLA effectively and ethically for improving teaching and learning activities in personalized learning environments.

참고문헌 (60)

  1. 성한올, "온라인 학습 상황에서 행동 로그, 생리심리반응 및 시험불안을 통한 멀티모달(Multimodal) 학업성취 예측모형 개발", 교육공학연구, 34(2), pp.287~308 (2018), 한국교육공학회[DOI]
  2. 안미리, "해외 학습분석학(Learning Analytics) 연구에 대한 동향 분석: 실증 연구 중심으로", 교육정보미디어연구, 21(4), pp.601~643 (2015), 한국교육정보미디어학회[DOI]
  3. 이현경, "학습분석의 윤리적 이슈와 실천방안에 대한 초등학교 교사의 인식", 교육정보미디어연구, 26(1), pp.157~181 (2020), 한국교육정보미디어학회[DOI]
  4. Abrahamson, D., "The future of embodied design for mathematics teaching and learning", Frontiers in Education, 5(147), pp.1~29 (2020)
  5. Abrahamson, D., "Eye-tracking Piaget: Capturing the emergence of attentional anchors in the coordination of proportional motor action", Human Development, 58(4-5), pp.218~244 (2015)
  6. Ahn, B. T., "Facial expressions when learning with a queer history app : Application of the control value theory of achievement emotions", British Journal of Educational Technology, 51(5), pp.1563~1576 (2020)
  7. Andrade, A., "A methodological framework for the exploratory analysis of multimodal features in learning activities", Joint proceedings of the sixth multimodal learning analytics (MMLA)workshop and the second cross-LAK workshop, pp.99~103 (2017), CEUR
  8. Andrade, A., "Using multimodal learning analytics to model student behaviour : A systematic analysis of behavioural framing", Journal of Learning Analytics, 3(2), pp.282~306 (2016)
  9. Avella, J. T., "Learning analytics methods, benefits, and challenges in higher education : A systematic literature review", Online Learning, 20(2), pp.13~29 (2016)
  10. Blikstein, P., "Multimodal learning analytics and education data mining : Using computational technologies to measure complex learning tasks", Journal of Learning Analytics, 3(2), pp.220~238 (2016)
  11. Camacho, V. L., "Data capture and multimodal learning analytics focused on engagement with a new wearable IoT approach", IEEE Transactions on Learning Technologies, 13(4), pp.704~717 (2020)
  12. Celdrán, A. H., "A scalable architecture for the dynamic deployment of multimodal learning analytics applications in smart classrooms", Sensors, 20(10), pp.2923~ (2020)
  13. Cook, D. A., "Conducting systematic reviews in medical education : A stepwise approach", Medical Education, 46, pp.943~952 (2012)
  14. Cornide-Reyes, H., "Introducing low-cost sensors into the classroom settings : Improving the assessment in agile practices with multimodal learning analytics", Sensors, 19(15), pp.3291~ (2019)
  15. Cornide-Reyes, H., "A multimodal real-time feedback platform based on spoken interactions for remote active learning support", Sensors, 20(21), pp.6337~ (2020)
  16. Crescenzi-Lanna, L., "Multimodal learning analytics research with young children : A systematic review", British Journal of Educational Technology, 51(5), pp.1485~1504 (2020)
  17. Cukurova, M., "The NISPI framework : Analysing collaborative problem-solving from students' physical interactions", Computers & Education, 116, pp.93~109 (2018)
  18. Di Mitri, D., "From signals to knowledge : A conceptual model for multimodal learning analytics", Journal of Computer Assisted Learning, 34(4), pp.338~349 (2018)
  19. Di Mitri, D., "Detecting mistakes in CPR training with multimodal data and neural networks", Sensors, 19(14), pp.3099~ (2019)
  20. Dindar, M., "What does physiological synchrony reveal about metacognitive experiences and group performance", British Journal of Educational Technology, 51(5), pp.1577~1596 (2020)
  21. Elias, T., "Learning analytics", pp.1~22 (2011), Learning
  22. Emerson, A., "Multimodal learning analytics for game-based learning", British Journal of Educational Technology, 51(5), pp.1505~1526 (2020)
  23. Giannakos, M. N., "Multimodal data as a means to understand the learning experience", International Journal of Information Management, 48, pp.108~119 (2019)
  24. Hellas, A., "Predicting academic performance: A systematic literature review", Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE 2018 Companion), pp.175~199 (2018), ACM
  25. Larmuseau, C., "Multimodal learning analytics to investigate cognitive load during online problem solving", British Journal of Educational Technology, 51(5), pp.1548~1562 (2020)
  26. Leedy, P. D., "Practical research : Planning and design" (2005), Merrill
  27. Liu, R., "Learning linkages : Integrating data streams of multiple modalities and timescales", Journal of Computer Assisted Learning, 35(1), pp.99~109 (2019)
  28. Luckin, R., "Designing educational technologies in the age of AI : A learning sciences-driven approach", British Journal of Educational Technology, 50(6), pp.2824~2838 (2019)
  29. Mangaroska, K., "Learning analytics for learning design : A systematic literature review of analytics-driven design to enhance learning", IEEE Transactions on Learning Technologies, 12(4), pp.516~534 (2018)
  30. Mangaroska, K., "Multimodal learning analytics to inform learning design : Lessons learned from computing education", Journal of Learning Analytics, 7(3), pp.79~97 (2020)
  31. Martinez-Maldonado, R., "Collocated collaboration analytics : Principles and dilemmas for mining multimodal interaction data", Human Computer Interaction, 34(1), pp.1~50 (2019)
  32. Mu, S., "Multimodal data fusion in learning analytics : A systematic review", Sensors, 20(23), pp.6856~ (2020)
  33. Munoz, R., "Development of a software that supports multimodal learning analytics : A case study on oral presentations", Journal of Universal Computer Science, 24(2), pp.149~170 (2018)
  34. Noel, R., "Exploring collaborative writing of user stories with multimodal learning analytics : A case study on a software engineering course", IEEE Access, 6, pp.67783~67798 (2018)
  35. Okada, M., "Multimodal analytics to understand self-regulation process of cognitive and behavioral strategies in real-world learning", IEICE TRANSACTIONS on Information and Systems, 103(5), pp.1039~1054 (2020)
  36. Olsen, J. K., "Temporal analysis of multimodal data to predict collaborative learning outcomes", British Journal of Educational Technology, 51(5), pp.1527~1547 (2020)
  37. Oviatt, S., "Dynamic handwriting signal features predict domain expertise", ACM Transactions on Interactive Intelligent Systems(TiiS), 8(3), pp.1~21 (2018)
  38. Papamitsiou, Z., "Utilizing multimodal data through fsQCA to explain engagement in adaptive learning", IEEE Transactions on Learning Technologies, 13(4), pp.689~703 (2020)
  39. Peng, S., "Recognizing student’s learning-centered affective states in conversation with intelligent multimodal analytics", Creative Education, 11(09), pp.1697~1719 (2020)
  40. Perveen, A., "Facilitating multiple intelligences through multimodal learning analytics", Turkish Online Journal of Distance Education, 19(1), pp.18~30 (2018)
  41. Pijeira-Díaz, H. J., "Profiling sympathetic arousal in a physics course : How active are students?", Journal of Computer Assisted Learning, 34(4), pp.397~408 (2018)
  42. Prieto, L. P., "Multimodal teaching analytics : Automated extraction of orchestration graphs from wearable sensor data", Journal of computer assisted learning, 34(2), pp.193~203 (2018)
  43. Roque, F., "Using depth cameras to detect patterns in oral presentations : A case study comparing two generations of computer engineering students", Sensors, 19(16), pp.3493~ (2019)
  44. Samuelsen, J., "Integrating multiple data sources for learning analytics—review of literature", Research and Practice in Technology Enhanced Learning, 14(1), pp.1~20 (2019)
  45. Schneider, B., "Unraveling students' interaction around a tangible interface using multimodal learning analytics", Journal of Educational Data Mining, 7(3), pp.89~116 (2015)
  46. Schneider, J., "Beyond reality—Extending a presentation trainer with an immersive VR module", Sensors, 19(16), pp.3457~ (2019)
  47. Schuurmans, A. A., "Validity of the empatica e4 wristband to measure heart rate variability(HRV)parameters : A comparison to electrocardiography(ECG)", Journal of medical systems, 44(11), pp.1~11 (2020)
  48. Schwendimann, B. A., "Perceiving learning at a glance : A systematic literature review of learning dashboard research", IEEE Transactions on Learning Technologies, 10(1), pp.30~41 (2016)
  49. Shankar, S. K., "A review of multimodal learning analytics architectures", 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT) (2018)
  50. Sharma, K., "Multimodal data capabilities for learning : What can multimodal data tell us about learning", British Journal of Educational Technology, 51(5), pp.1450~1484 (2020)
  51. Sharma, K., "Eye-tracking and artificial intelligence to enhance motivation and learning", Smart Learning Environments, 7, pp.1~19 (2020)
  52. Sharma, K., "Utilizing interactive surfaces to enhance learning, collaboration and engagement : Insights from learners’ gaze and speech", Sensors, 20(7), pp.1964~ (2020)
  53. Sharma, K., "Building pipelines for educational data using AI and multimodal analytics : A"grey-box"approach", British Journal of Educational Technology, 50(6), pp.3004~3031 (2019)
  54. Siemens, G., "Penetrating the fog : Analytics in learning and education", EDUCAUSE review, 46(5), pp.31~40 (2011)
  55. Spikol, D., "Supervised machine learning in multimodal learning analytics for estimating success in project-based learning", Journal of Computer Assisted Learning, 34(4), pp.366~377 (2018)
  56. Taub, M., "The agency effect : The impact of student agency on learning, emotions, and problem-solving behaviors in a game-based learning environment", Computers & Education, 147, pp.1~19 (2020)
  57. Tempelaar, D., "A multi-modal study into students' timing and learning regulation : Time is ticking", Interactive Technology and Smart Education, 15(4), pp.298~313 (2018)
  58. Vujovic, M., "Round or rectangular tables for collaborative problem solving? A multimodal learning analytics study", British Journal of Educational Technology, 51(5), pp.1597~1614 (2020)
  59. Worsley, M., "A multimodal analysis of making", International Journal of Artificial Intelligence in Education, 28(3), pp.385~419 (2018)
  60. Zhu, G., "Exploring emotional and cognitive dynamics of knowledge building in grades 1 and 2", User Modeling and User-Adapted Interaction, 29(4), pp.789~820 (2019)

관련 논문AI추천

국내 디지털 교수학습 환경에서의 학습분석 연구 및 데이터 활용에 대한 체계적 문헌 고찰김혜빈, 신안나, 조일현, 표은서한국교육공학회 · 교육공학연구2023 · 제39권 제2호학습자 역량 측정 연구 동향 분석: 멀티모달 학습분석을 중심으로박소영, 이성혜, 장나영한국교육공학회 · 교육공학연구2025 · 제41권 제3호다중양식 학습분석 연구동향 분석: 온라인 학습을 중심으로김동심, 류다현한국정보교육학회 · 정보교육학회논문지2023 · 제27권 제1호학습분석학 국내 문헌 고찰: 로그 데이터를 이용한 실증연구를 중심으로고윤미, 김민하, 배윤희, 안미리, 최윤영한국교육공학회 · 교육공학연구2016 · 제32권 제2호협력학습에서의 자기조절학습 측정을 위한 멀티모달 학습분석(MMLA) 활용: 체계적 문헌고찰김준원, 김창조, 신윤희, 임예린한국교육공학회 · 교육공학연구2025 · 제41권 제3호멀티모달 학습분석을 활용한 학습자 참여 연구 동향 분석박소영, 이성혜, 장나영한국실천공학교육학회 · 실천공학교육논문지2025 · 제17권 제6호

이 간행물의 다른 논문 (13건)

코로나 19 상황에서 대학 교수자의 온라인 수업 경험 탐색양혜리, 임지은pp. 371-400인공지능 애플리케이션을 활용한 고등학교 수학 수업설계의 효과: 학업성취도와 정의적 영역을 중심으로김현진, 오석환pp. 401-422디지털 콘텐츠 저작 도구 Namo Author를 활용한 교육 프로그램 설계 과정과 효과에 관한 연구이웅기, 이종찬, 염지윤, 정혜원, 최서연, 임철일pp. 423-446대학 교수자의 온라인 수업효능감 분석: A대학을 중심으로신종호, 박수영pp. 447-467유튜브 아동보호 가이드라인의 구성요소에 관한 연구: 가이드라인 및 소셜 빅데이터 이슈를 중심으로고범석, 최홍규pp. 469-500
원문 보기