본문 바로가기
KERD
논문KCI 등재

이러닝 학습환경에서 학습분석기반 대시보드 연구동향 분석

An Analytic Review of the studies on Learning Analytics based Dashboard in e-Learning Environments

2

조회

0

다운로드

34

인용

0

참고문헌

키워드

주제분류교육학

초록

이러닝 학습환경에서의 학습이 증가하면서 온라인에서의 학습자들의 학습행동 데이터를 분석하여이를 처방적으로 활용하는 것으로 목적으로 하는 학습분석에 대한 관심이 고조되고 있다. 학습분석은활용주체, 접근 관점, 데이터의 종류에 따라 활용 방법과 적용 맥락이 다양해 질 수 있으며, 이를 시각화하여 대시보드에 표상하는 방법 또한 매우 다양하다. 본 연구에서는 학습분석에 기반한 대시보드관련 선행연구를 분석함으로써 연구동향을 정리하고 향후 학습분석 관련 연구에 대한 시사점을 제안하고자 하였다. 연구방법은 문헌고찰연구방법론에 따라 선행연구 선정 기준 설정, 선행문헌 선정, 분석 체계 수립, 선행문헌 선정 및 분석 체계에 대한 전문가 타당화, 선행문헌 분석 및 교차 검토의 과정으로 이루어졌다. 27개의 대시보드 관련 선행연구들을 선정하였고, 분석 체계를 수립하여 타당성을확보하기 위해 연구자를 포함한 3명의 교육공학전문가들을 대상으로 타당화 검사를 실시하였다(r=0.92). 선정된 27개의 대시보드 관련 선행연구들을 분석체계를 기반으로 분석한 결과 대시보드 개발 목적은 학습지원을 위한 것이 18개(67%), 교수지원을 위한 연구가 16개(59%), 교육기관에서 교육정책을 수립하기 위한 의사결정을 위한 것이 2개(7%)로 나타났다. 대시보드 개발을 위해 사용한 데이터소스는 학생산출물, 학습시간, 상호작용, 시험·과제 결과, 학습자료, 기타로 구분될 수 있으며, 이를 시각화하는 기법으로는 비교, 관계, 시간, 분포, 텍스트, 공간, 메타포가 있었다. 대시보드 효과분석은 선행문헌 중 16개(59%)가 실시하지 않았으며, 효과성을 분석한 연구는 7개(26%), 효율성을 분석한 연구는 2개였다. 연구 결과에 기반하여 학습분석 관련 추후 연구 방향으로 처방적 대시보드 설계 및 개발필요, 대시보드 시각화 설계 원리 연구, 대시보드 효과분석을 통한 일반화를 제안하였다

Abstract

Learning Analytics (LA) has been recently paid attention due to the increase of e-learning. LA aims to use prescriptively the results which learner’s behaviors are analyzed and can be utilized in various ways according to users, objectives, or data source. The purpose of this study is to synthetically review the studies on LA based dashboard in e-Learning environments and to suggest directions of future research on LA based dashboard. To achieve this purpose, the review study methodology was adopted: setting up inclusion criteria, literature search and selection, setting up coding scheme, verification of literature selection and coding scheme, and analysis. 27 studies were selected and three experts with Ph.D in Educational Technology participated to the expert review survey. The research selection and coding scheme was verified(r=0.92). The analysis results were suggested according to coding scheme. Related to the purpose of dashboard development, 18 studies (67%) were for supporting learning, 16 studies (59%) were for supporting teaching, and 2 studies (7%) were for supporting decision-making related education policy. Data source which was used for developing dashboard was categorized into artifacts produced by learners, social interaction, exam/assignment results, resource use, and etc. The visualization techniques used in the previous studies were comparison, relationship, changes over time, distribution, text analysis, space, metaphor. 16 studies (59%) did not evaluate the effects of the developed dashboard, only 7 studies(26%) evaluated the effects of dashboard, and 2 studies evaluated the efficiency of dashboard. Future research directions are suggested based on the review results.

참고문헌 (88)

  1. Kang Jeeae Stephanie, "학습분석학적 관점에서 개발한 시각화 대시보드가 학업성취도에 미치는 영향에 대한 탐색적 연구", 석사, 교육공학과, pp.2014
  2. 김재휘, "설득 심리 이론" (2013), 커뮤니케이션북스
  3. 박연정, "학습관리시스템의 대시보드 설계를 위한 학습자 중심 요구분석: 분석과 설계 도구로서 활동이론의 적용", 교육공학연구, 30(2), pp.221~258 (2014), 한국교육공학회
  4. 박연정, "학습 분석학 기반 대시보드의 설계와 적용", 교육정보미디어연구, 20(2), pp.191~216 (2014), 한국교육정보미디어학회
  5. 하건희, "학습자 성찰 도구로써 대시보드 개발을 위한 사용성 검사 연구", 석사, pp.2015
  6. "빅데이터 시각화 기술" (2014)[URL]
  7. Ali, L., "A qualitative evaluation of evolution of a learning analytics tool", Computers & Education, 58(1), pp.470~489 (2012)
  8. Arnold, K. E., "Course signals at Purdue: using learning analytics to increase student success", Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, pp.267~270 (2012), ACM
  9. "A Quantitative Evaluation of LOCO-Analyst: A Tool for Raising Educators' Awareness in Online Learning Environments" (2011)[URL]
  10. Bajzek, D., "Inventing the Digital Dashboard for Learning", World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp.1084~1092
  11. Bakharia, A., "SNAPP: a bird's-eye view of temporal participant interaction", Proceedings of the 1st international conference on learning analytics and knowledge, pp.168~173 (2011), ACM
  12. Brown, W. E., "Improving the feedback cycle to improve learning in introductory biology using the digital dashboard", Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2006, pp.1030~1035 (2006), AACE
  13. Campbell, J. P., "Academic analytics", EDUCAUSE, pp.1~20 (2007)
  14. Chatti, M. A., "A reference model for learning analytics", International Journal of Technology Enhanced Learning, 4(5), pp.318~331 (2012)
  15. Clow, D., "An overview of learning analytics", Teaching in Higher Education, 18(6), pp.683~695 (2013)
  16. Dawson, S., "SNAPP : Realising the affordances of real-time SNA within networked learning environments", Proceedings of the Seventh International Conference on Networked Learning 2010 (2010)
  17. Dix, A., "Human-computer interaction" (1998), Prentice Hall
  18. Donath, J., "The chat circles series: explorations in designing abstract graphical communication interfaces", Proceedings of the 4th conference on Designing interactive systems: processes, practices, methods, and techniques, pp.359~369 (2002), ACM
  19. Dourish, P., "Awareness and coordination in shared workspaces", Proceedings of the 1992 ACM conference on Computer-supported cooperative work, pp.107~114 (1992), ACM
  20. Duval, E., "Attention please!: learning analytics for visualization and recommendation", Proceedings of the 1st International Conference on Learning Analytics and Knowledge, pp.9~17 (2011), ACM
  21. Dyckhoff, A. L., "Design and implementation of a learning analytics toolkit for teachers", Educational Technology & Society, 15(3), pp.58~76 (2012)
  22. Dyckhoff, A. L., "eLAT: An Exploratory Learning Analytics Tool for Reflection and Iterative Improvement of Technology Enhanced Learning", Proceedings of the 4th International Conference on Educational Data Mining, pp.355~356 (2011)
  23. Elias, T., "Learning analytics Definitions, processes and potential", Learning, 23, pp.134~148 (2011)
  24. Elkina, M., "The learning analytics application LeMo–rationals and first results", International Journal of Computing, 12(3), pp.226~234 (2013)
  25. Essa, A., "Improving Student Success Using Predictive Models and Data Visualisations", Research in Learning Technology, 20, pp.58~70 (2012)
  26. Essa, A., "Student success system: Risk analytics and data visualization using ensembles of predictive models", Proceedings of the Second International Conference on Learning Analytics and Knowledge, pp.158~161 (2012), ACM
  27. Ferguson, R., "Social learning analytics: five approaches", Proceedings of the Second international conference on learning analytics and knowledge, pp.23~33 (2012), ACM
  28. Festinger, L., "A theory of social comparison processes", Human relations, 7(2), pp.117~140 (1954)
  29. Few, S., "Information dashboard design" (2006), O'Reilly Media, Inc
  30. Fortenbacher, A., "LeMo: A learning analytics application focussing on user path analysis and interactive visualization", Proceedings of the IEEE Seventh International Conference on Intelligent Data Acquisition and Advanced Computing Systems, pp.12~14 (2013)
  31. Fulantelli, G., "A semantic approach to mobile learning analytics", Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality, pp.287~292 (2013), ACM
  32. Garrard, J., "Health sciences literature review made easy: The matrix method" (2007), Jones &Bartlett
  33. "A Guide to Creating Dashboards People Love to Use" (2009)[URL]
  34. Govaerts, S., "The student activity meter for awareness and self-reflection", CHI'12 Extended Abstracts on Human Factors in Computing Systems, pp.869~884 (2012), ACM
  35. Gross, T., "User-centered awareness in computer-supported cooperative work-systems : Structured embedding of findings from social sciences", International Journal of Human-Computer Interaction, 18(3), pp.323~360 (2005)
  36. Gutwin, C., "Workspace Awareness in Real-Time Distributed Groupware", doctor, pp.1997
  37. Gutwin, C., "Support for group awareness in real-time desktop conferences", Proceedings of the Second New Zealand Computer Science Research Students' Conference, pp.18~21 (1995)
  38. Gutwin, C., "A descriptive framework of workspace awareness for real-time groupware", Computer Supported Cooperative Work, 11(3-4), pp.411~446 (2002)
  39. Holzinger, A., "Usability engineering methods for software developers", Communications of the ACM, 48(1), pp.71~74 (2005)
  40. Howlin, C., "Learning and Academic Analytics in the Realizeit System", Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2014, pp.862~872 (2014), AACE
  41. Iandoli, L., "A debate dashboard to enhance online knowledge sharing", VINE, 42(1), pp.67~93 (2012)
  42. Ifenthaler, D., "Encyclopedia of educational technology", pp.447~451 (2015), SAGE Publications, Inc
  43. Ji, M., "Open Learning and Teaching in Educational Communities", pp.440~445 (2014), Springer International Publishing
  44. Ji, M., "DDART: an Awareness System to Favor Reflection during Project-Based Learning", Proceedings of the 4th Workshop on Awareness and Reflection in Technology Enhanced Learning, pp.81~83 (2014)
  45. "The 2011 Horizon report" (2011)[URL]
  46. Jovanović, J., "Using semantic web technologies to analyze learning content", Internet Computing, IEEE, 11(5), pp.45~53 (2007)
  47. Jovanović, J., "Creating New Learning Experiences on a Global Scale", pp.112~126 (2007), Springer Berlin Heidelberg
  48. Jovanović, J., "Ontologies for effective use of context in e-learning settings", Educational Technology & Society, 10(3), pp.47~59 (2007)
  49. Karoulis, A., "Usability evaluation of online learning programs", pp.88~109 (2003)
  50. Kimmerle, J., "Group awareness and self-presentation in computer-supported information exchange", International Journal of Computer-Supported Collaborative Learning, 3(1), pp.85~97 (2008)
  51. Kulyk, O., "Situational awareness support to enhance teamwork in collaborative environments", Proceedings of the 15th European conference on Cognitive ergonomics: the ergonomics of cool interaction, pp.5~ (2008), ACM
  52. Kurosu, M., "Apparent usability vs. inherent usability: experimental analysis on the determinants of the apparent usability", Proceedings of the Conference companion on Human factors in computing systems, pp.292~293 (1995), ACM
  53. Leony, D., "GLASS: a learning analytics visualization tool", Proceedings of the 2nd international conference on learning analytics and knowledge, pp.162~163 (2012), ACM
  54. Llora, X., "Analyzing active interactive genetic algorithms using visual analytics", Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp.1417~1418 (2006), ACM
  55. Long, P., "Penetrating the fog : Analytics in learning and education", EDUCAUSE Review, 46(5), pp.30~32 (2011)
  56. Mazza, R., "Using information visualisation to facilitate instructors in web-based distance learning", doctor, pp.2004
  57. Mazza, R., "Monitoring an online course with the GISMO tool : A case study", Journal of Interactive Learning Research, 18(2), pp.251~265 (2007)
  58. Mazza, R., "Visualising student tracking data to support instructors in web-based distance education", Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, pp.154~161 (2004), ACM
  59. Mazza, R., "CourseVis : A graphical student monitoring tool for supporting instructors in web-based distance courses", International Journal of Human-Computer Studies, 65(2), pp.125~139 (2007)
  60. Mazza, R., "Gismo: a graphical interactive student monitoring tool for course management systems", TEL’04 Technology Enhanced Learning’04 International Conference, pp.1~8 (2004)
  61. Mazza, R., "Exploring usage analysis in learning systems: Gaining insights from visualisations", AIED’05 workshop on Usage analysis in learning systems, pp.65~72 (2005)
  62. Merceron, A., "Visual exploration of interactions and performance with lemo", Proceedings of the Sixth International Conference on Educational Data Mining, pp.396~397 (2013)
  63. Michel, C., "21st Century Learning for 21st Century Skills", pp.250~263 (2012), Springer Berlin Heidelberg
  64. Murray, T., "Brief Overview of Social Deliberative Skills", Proceedings of AIED 2013 Self-Regulated Learning Workshop, pp.1~6 (2013)
  65. Murray, T., "Supporting Deeper Deliberative Dialogue Through Awareness Tools", proceedings of Build Peace 2014 Conference (2014)
  66. Murray, T., "A Dashboard for Visualizing Deliberative Dialogue in Online Learning", Proceedings of Artificial Intelligence in Education 2013 (2013)
  67. Murray, T., "A prototype facilitators dashboard: assessing and visualizing dialogue quality in online deliberations for education and work", Proceedings of 2013 International Conference on e-Learning, e-Business, Enterprise Information Systems, and e-Government (2013)
  68. Pallotta, V., "Automatic argumentative analysis for interaction mining", Argument &Computation, 2(2-3), pp.77~106 (2011)
  69. Park, Y., "Development of the Learning Analytics Dashboard to Support Students'Learning Performance", Journal of Universal Computer Science, 21(1), pp.110~133 (2015)
  70. Pearrow, M., "Web Site Usability Handbook with Cdrom" (2000), Charles River Media, Inc
  71. Podgorelec, V., "Taking advantage of education data : Advanced data analysis and reporting in virtual learning environments", Elektronika ir Elektrotechnika, 114(8), pp.111~116 (2011)
  72. Rada, R., "Efficiency and effectiveness in computer-supported peer-peer learning", Computers and Education, 30(3), pp.137~146 (1998)
  73. Santos, J. L., "Scaling up Learning for Sustained Impact", pp.314~327 (2013), Springer Berlin Heidelberg
  74. Santos, J. L., "Goal-oriented visualizations of activity tracking: a case study with engineering students", Proceedings of the Second International Conference on Learning Analytics and Knowledge, pp.143~152 (2012), ACM
  75. Santos, J. L., "Addressing learner issues with StepUp!: an Evaluation", Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp.14~22 (2013), ACM
  76. Simsek, D., "Learning Analytics Beyond Learning Management Systems: Proposing a Learner Dashboard which collects, analyzes and reports learner-generated data from social networking sites", master, pp.2012
  77. Softic, S., "Human Factors in Computing and Informatics", pp.74~90 (2013), Springer Berlin Heidelberg
  78. Tobarra, L., "Integrated Analytic dashboard for virtual evaluation laboratories and collaborative forums", Tecnologias Aplicadas a la Ensenanza de la Electronica (Technologies Applied to Electronics Teaching)(TAEE), pp.1~6 (2014), IEEE
  79. Torniai, C., "E-learning meets the social semantic web", Advanced Learning Technologies, 2008. ICALT'08. Eighth IEEE International Conference, pp.389~393 (2008), IEEE
  80. Tractinsky, N., "Aesthetics and apparent usability: empirically assessing cultural and methodological issues", Proceedings of the ACM SIGCHI Conference on Human factors in computing systems, pp.115~122 (1997), ACM
  81. Verbert, K., "Learning analytics dashboard applications", American Behavioral Scientist (2013)
  82. Verbert, K., "Learning analytics dashboard applications", American Behavioral Scientist, 57(10), pp.1500~1509 (2013)
  83. Verbert, K., "Learning dashboards : an overview and future research opportunities", Personal and Ubiquitous Computing, 18(6), pp.1499~1514 (2014)
  84. Viégas, F. B., "Chat circles", Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp.9~16 (1999), ACM
  85. Willis, P., "Cultural production is different from cultural reproduction is different from social reproduction is different from reproduction", Interchange, 12(2-3), pp.48~67 (1981)
  86. Wise, A. F., "Learning analytics for online discussions: a pedagogical model for intervention with embedded and extracted analytics", Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp.48~56 (2013), ACM
  87. Wolff, A., "Improving retention by identifying and supporting “at-risk” students", EDUCAUSE Review (2012)
  88. Yang, S. J. H., "A Social Network-based System for Supporting Interactive Collaboration in Knowledge Sharing over Peer-to-Peer Network", International Journal of Human-Computer Studies, 66(1), pp.36~50 (2008)

관련 논문AI추천

이 간행물의 다른 논문 (6건)