시선추적-뇌파 기반의 비디오 요약 생성 방안 연구
Video Summarization Using Eye Tracking and Electroencephalogram (EEG) Data
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구는 시선 및 뇌파 정보를 이용하여 오디오-비주얼(audio-visual, AV) 시맨틱스 기반의 동영상 요약 방법들을 개발하고 평가해 보았다. 이를 위해서 27명의 대학생들을 대상으로 시선추적과 뇌파 실험을 수행하였다. 평가 결과, 뇌파와 동공크기 데이터를 함께 사용한 방법의 평균 재현율(0.73)이 뇌파 또는 동공크기 데이터만을 사용한 방법의 평균 재현율(뇌파: 0.50, 동공크기: 0.68)보다 높게 나타났다. 또한 AV 시맨틱스 기반의 개인화된 동영상 요약의 평균 재현율(0.57)이 AV 시맨틱스 기반의 일반적인 동영상 요약의 평균 재현율(0.69)보다 낮게 나타난 원인들을 분석하였다. 끝으로, AV 시맨틱스 기반 동영상 요약 방법과 텍스트 시맨틱스 기반 동영상 요약 방법 간의 차이 및 특성도 비교분석해 보았다.
Abstract
This study developed and evaluated audio-visual (AV) semantics-based video summarization methods using eye tracking and electroencephalography (EEG) data. For this study, twenty-seven university students participated in eye tracking and EEG experiments. The evaluation results showed that the average recall rate (0.73) of using both EEG and pupil diameter data for the construction of a video summary was higher than that (0.50) of using EEG data or that (0.68) of using pupil diameter data. In addition, this study reported that the reasons why the average recall (0.57) of the AV semantics-based personalized video summaries was lower than that (0.69) of the AV semantics-based generic video summaries. The differences and characteristics between the AV semantics-based video summarization methods and the text semantics-based video summarization methods were compared and analyzed.
참고문헌 (28)
- 김현희, "뇌파측정기술(EEG)에 기초한 멀티미디어 자료의 주제 적합성에 관한 연구", 한국문헌정보학회지, 50(3), pp.361~381 (2016), 한국문헌정보학회[DOI]
- 김현희, "뇌파측정기술을 활용한 언어 기반 사운드 요약의 생성 방안 연구", 정보관리학회지, 36(3), pp.131~148 (2019), 한국정보관리학회[DOI]
- 안형모, "텍스트 기반 학습에서 학습매체의 크기와 종류가 학습 집중도에 미치는 영향", 석사, 초등컴퓨터교육전공, pp.2013
- 윤정원, "페이스북 건강정보 게시물 형식이 시각적 주의와 인지결과에 미치는 영향", 한국문헌정보학회지, 55(3), pp.219~237 (2021), 한국문헌정보학회[DOI]
- 정영미, "정보검색연구" (2012), 연세대학교 출판문화원
- Ajanki, A., "Can eyes reveal interest? Implicit queries from gaze patterns", User Modeling and User-Adapted Interaction, 19(4), pp.307~339 (2009)
- Bezugam, S. S., "Efficient video summarization framework using EEG and eye-tracking signals", arXiv preprint arXiv: 2101.11249, ~ (2021)
- Bhattacharya, N., "Relating eye-tracking measures with changes in knowledge on search tasks", arXiv preprint arXiv: 1805.02399, ~ (2018)
- Bhattacharya, N., "Relevance prediction from eye-movements using semi-interpretable convolutional neural networks", Proceedings of the 2020 Conference on Human Information Interaction and Retrieval, pp.223~233 (2020)
- Evans, W. J., "Olfactory event-related potentials in normal human subjects : Effects of age and gender", Electroencephalography and Clinical Neurophysiology, 95(4), pp.293~301 (1995)
- Foley, J. J., "Encyclopedia of Information Science and Technology", pp.5897~5905 (2015), IGI Global
- Golenia, J. E., "Implicit relevance feedback from electroencephalography and eye tracking in image search", Journal of Neural Engineering, 15(2), pp.026002~ (2018)
- Gwizdka, J., "Temporal dynamics of eye-tracking and EEG during reading and relevance decisions", Journal of the Association for Information Science and Technology, 68(10), pp.2299~2312 (2017)
- Gwizdka, J., "Differences in eye-tracking measures between visits and revisits to relevant and irrelevant web pages", Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.811~814 (2015), ACM
- Hansen, D., "Analyzing social media networks with NodeXL : Insights from a connected world" (2010), Morgan Kaufmann
- Hughes, J. R., "Conventional and quantitative electroencephalography in psychiatry", The Journal of neuropsychiatry and clinical neurosciences, 11(2), pp.190~208 (1999)
- Katti, H., "Affective video summarization and story board generation using pupillary dilation and eye gaze", 2011 IEEE International Symposium on Multimedia, pp.319~326 (2011), IEEE
- Kaur, T., "Text extraction from natural scene using PCA", International Journal of Computer Science Engineering & Technology, 5(7), pp.272~277 (2015)
- Kim, H. H., "Video summarization using event-related potential responses to shot boundaries in real-time video watching", Journal of the Association for Information Science and Technology, 70(2), pp.164~175 (2019)
- Kim, H. H., "ERP", MMR algorithm for classifying topic-relevant and topic-irrelevant visual shots of documentary videos, 2019, pp.70(9), 931~941
- Moon, J., "Detecting user attention to video segments using interval EEG features", Expert Systems with Applications, 115, pp.578~592 (2019)[DOI]
- Oliveira, F. T., "Discriminating the relevance of web search results with measures of pupil size", Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.2209~2212 (2009), ACM
- Schumacher, P. B., "Positional influences on information packaging : Insights from topological fields in German", Journal of Memory and Language, 67(2), pp.295~310 (2012)
- Shi, Z. F., "Attention evaluation with eye tracking glasses for EEG-based emotion recognition", 2017 8th International IEEE, 2017, 86~89[DOI]
- Syn, S. Y., "Investigation on reading behaviors and cognitive outcomes of Facebook health information", Online Information Review, 45(6), pp.1097~1115 (2021)
- van Berkum, J. J., "Semantic integration in sentences and discourse : Evidence from the N400", Journal of Cognitive Neuroscience, 11(6), pp.657~671 (1999)
- Wang, L., "New is not always costly : evidence from online processing of topic and contrast in Japanese", Frontiers in Psychology, 4, pp.363~ (2013)
- Zhao, L. M., "Active feedback framework with scan-path clustering for deep affective models", International Conference on Neural Information Processing, pp.330~340 (2018), Springer