뇌파측정기술을 활용한 언어 기반 사운드 요약의 생성 방안 연구
Towards the Generation of Language-based Sound Summaries Using Electroencephalogram Measurements
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
본 연구는 시청자가 사운드 자료의 주제를 파악하는 과정과 사운드의 특성을 이해하기 위한 인지적 정보처리 모형을 구성하였다. 이후 사건관련유발전위(event related potentials, ERP)의 두뇌의 전후측에 걸쳐서 발현하는 N400, P600 구성요소들을 인지적 정보처리 모형의 언어적 표상에 접목시켜 사운드 요약을 생성하는 방안을 제안하기 위해서 연구 가설들을 수립하였다. 뇌파 실험을 통해서 연구 가설들을 검증한 결과, P600이 사운드 요약의 핵심 구성 요소로 나타났다. 본 연구 결과는 분류 알고리즘 설계에 적용되어 내용 기반 메타데이터 즉, 일반적인 또는 개인화된 미디어 요약(사운드 요약, 비디오 스킴)을 생성하는 데에 활용될 수 있을 것이다.
Abstract
This study constructed a cognitive model of information processing to understand the topic of a sound material and its characteristics. It then proposed methods to generate sound summaries, by incorporating anterior-posterior N400/P600 components of event-related potential (ERP) response, into the language representation of the cognitive model of information processing. For this end, research hypotheses were established and verified them through ERP experiments, finding that P600 is crucial in screening topic-relevant shots from topic-irrelevant shots. The results of this study can be applied to the design of classification algorithm, which can then be used to generate the content-based metadata, such as generic or personalized sound summaries and video skims.
참고문헌 (25)
- Huda Omar Alwehaibi, "The Impact Of Using YouTube In EFL Classroom On Enhancing EFL Students' Content Learning", Journal of College Teaching & Learning (TLC), 12(2), pp.121~126 (2015), Clute Institute[DOI]
- Baddeley, A., "Working memory, thought, and action" (2007), Oxford University Press
- Juliane Burmester, "Context updating during sentence comprehension: The effect of aboutness topic", Brain and Language, 137, pp.62~76 (2014), Elsevier BV[DOI]
- Buzzetto-More, N., "Student attitudes towards the integration of youtube in online, hybrid, and web-assisted courses: An examination of the impact of course modality on perception", MERLOT Journal of Online Learning and Teaching, 11, pp.55~73 (2015)
- Katherine A. DeLong, "Predictability, plausibility, and two late ERP positivities during written sentence comprehension", Neuropsychologia, 61, pp.150~162 (2014), Elsevier BV[DOI]
- W. James Evans, "Olfactory event-related potentials in normal human subjects: effects of age and gender", Electroencephalography and Clinical Neurophysiology, 95(4), pp.293~301 (1995), Elsevier BV[DOI]
- Geyer, A., "Plausibility and sentence comprehension. An ERP study", Cognitive Neuroscience Supplement, 1(1) (2006)
- Hakoda, Y., "인지심리학" (2014), 한국뇌기반교육연구소
- Weiming Hu, "A Survey on Visual Content-Based Video Indexing and Retrieval", IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 41(6), pp.797~819 (2011), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Albert Kim, "The independence of combinatory semantic processing: Evidence from event-related potentials", Journal of Memory and Language, 52(2), pp.205~225 (2005), Elsevier BV[DOI]
- Hyun Hee Kim, "Generic speech summarization of transcribed lecture videos: Using tags and their semantic relations", Journal of the Association for Information Science and Technology, 67(2), pp.366~379 (2016), Wiley[DOI]
- Hyun Hee Kim, "Video summarization using event-related potential responses to shot boundaries in real-time video watching", Journal of the Association for Information Science and Technology, 70(2), pp.164~175 (2019), Wiley[DOI]
- Hyun Hee Kim, "ERP", MMR algorithm for classifying topic-relevant and topic-irrelevant visual shots of documentary videos, 2019, pp.70(9), 931~941[DOI]
- Marta Kutas, "Thirty Years and Counting: Finding Meaning in the N400 Component of the Event-Related Brain Potential (ERP)", Annual Review of Psychology, 62(1), pp.621~647 (2011), Annual Reviews[DOI]
- Luck, S. J., "An introduction to the event-related potential technique" (2014), MIT Press
- Maskey, S., "Summarizing speech without text using hidden markov models", Proceedings of the Human Language Technology Conference of the NAACL, pp.89~92 (2006), Association for Computational Linguistics
- Martin, D., "YouTube: The ultimate 2018 guide to grow your youtube channel, make money fast with proven techniques and foolproof step by step strategies" (2018), CreateSpace Independent Publishing Platform
- Mayer, R. E., "The Cambridge handbook of multimedia learning", pp.134~146 (2005), Cambridge University Press
- Jinyoung Moon, "Detecting user attention to video segments using interval EEG features", Expert Systems with Applications, 115, pp.578~592 (2019), Elsevier BV[DOI]
- Hiroko Nakano, "Electrophysiological response to omitted stimulus in sentence processing", NeuroReport, 25(14), pp.1169~1174 (2014), Ovid Technologies (Wolters Kluwer Health)[DOI]
- Mante S. Nieuwland, "If the real world were irrelevant, so to speak: The role of propositional truth-value in counterfactual sentence comprehension", Cognition, 122(1), pp.102~109 (2012), Elsevier BV[DOI]
- Jos J. A. van Berkum, "Semantic Integration in Sentences and Discourse: Evidence from the N400", Journal of Cognitive Neuroscience, 11(6), pp.657~671 (1999), MIT Press - Journals[DOI]
- Luming Wang, "New is not always costly: evidence from online processing of topic and contrast in Japanese", Frontiers in Psychology, 4 (2013), Frontiers Media SA[DOI]
- Stephen M. Wilson, "Convergence of spoken and written language processing in the superior temporal sulcus", NeuroImage, 171, pp.62~74 (2018), Elsevier BV[DOI]
- Justin Jian Zhang, "Active learning with semi-automatic annotation for extractive speech summarization", ACM Transactions on Speech and Language Processing, 8(4), pp.1~25 (2012), Association for Computing Machinery (ACM)[DOI]