정서 재인 방법 고찰을 통한 통합적 모델 모색에 관한 연구
Toward an integrated model of emotion recognition methods based on reviews of previous work
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
정서 컴퓨팅 분야는 인간과 컴퓨터 간 상호작용을 효과적이게 하기 위하여 사용자의 정서를 재인하는 컴퓨터 시스템을 개발했다. 본 연구의 목적은 심리학적 이론에 기반한 정서 재인 연구들을 고찰하고 보다 진보된 정서 재인 방법을 제안하고자 하였다. 본 연구의 본론에서는 심리학적 이론에 근거한 대표적인 정서 재인 방법을 고찰하였다. 첫째, Darwin 이론에 근거한 얼굴 표정 기반 정서 재인 방법을 고찰하였다. 둘째, James 이론에 근거한 생리신호기반 정서 재인 방법을 고찰하였다. 셋째, 앞의 두 이론에 통합적으로 근거한 다중 모달리티 기반 정서 재인 방법을 고찰하였다. 세 가지 방법들은 이론적 배경과 현재까지 이루어진 연구 결과들을 중심으로 고찰되었다. 결론에서는 선행 연구의 한계점을 종합하여 보다 진보된 정서 재인 방법을 제안하였다. 본 연구는 첫째, 현재 사용되고 있는 제한된 생리 신호에 대한 대안으로 다양한 생리 반응(예., 뇌 활동, 얼굴 온도 등)을 정서 재인에 포함하도록 제안하였다. 둘째, 애매한 정서를 구분할 수 있도록 정서의 차원 개념에 기반한 정서 재인 방법을 제안하였다. 셋째, 정서 유발에 영향을 미치는 인지적 요소를 정서 재인에 포함하도록 제안하였다. 본 연구에서 제안한 정서 재인 방법은 다양한 생리 신호를 포함하고, 정서의 차원적 개념에 기반하며, 인지적 요소를 고려한 통합적인 모델이다.
Abstract
Current researches on emotion detection classify emotions by using the information from facial, vocal, and bodily expressions, or physiological responses. This study was to review three representative emotion recognition methods, which were based on psychological theory of emotion. Firstly, literature review on the emotion recognition methods based on facial expressions was done. These studies were supported by Darwin’s theory. Secondly, review on the emotion recognition methods based on changes in physiology was conducted. These researches were relied on James’ theory. Lastly, a review on the emotion recognition was conducted on the basis of multimodality(i.e., combination of signals from face, dialogue, posture, or peripheral nervous system). These studies were supported by both Darwin’s and James’ theories. In each part, research findings was examined as well as theoretical backgrounds which each method was relied on. This review proposed a need for an integrated model of emotion recognition methods to evolve the way of emotion recognition. The integrated model suggests that emotion recognition methods are needed to include other physiological signals such as brain responses or face temperature. Also, the integrated model proposed that emotion recognition methods are needed to be based on multidimensional model and take consideration of cognitive appraisal factors during emotional experience.
참고문헌 (101)
- Afzal, S, "Natural Affect Data: Collection & Annotation in a Learning Context", Proc. Third Int’l Conf. Affective Computing and Intelligent Interaction and Workshops, pp.1~7 (2009)
- AlZoubi, O, "Classification of EEG for Emotion Recognition: An Adaptive Approach", Proc. 22nd Australasian Joint Conf. Artificial Intelligence, pp.52~61 (2009)
- Andreassi, J. L, "Human Behaviour and Physiological Response" (2007), Taylor & Francis
- Arroyo, I, "Emotion Sensors Go to School", Proc. 14th Conf. Artificial Intelligence in Education, pp.17~24 (2009)
- Ashraf, A. B, "The Painful Face: Pain Expression Recognition Using Active Appearance Models", Proc. Ninth ACM Int’l Conf. Multimodal Interfaces (ICMI ’07), pp.9~14 (2007)
- Bailenson, J. N, "Real-Time Classification of Evoked Emotions Using Facial Feature Tracking and Physiological Responses", Int’l Journal of Human- Computer Studies, 66(66), pp.303~317 (2008)
- Balomenos, T, "Emotion Analysis in Man-Machine Interaction Systems", LNCS, (3361), pp.318~328 (2005)
- Barrett, L, "Are Emotions Natural Kinds", Perspectives on Psychological Science, (1), pp.28~58 (2006)
- Bartlett, M.S, "A Prototype for Automatic Recognition of Spontaneous Facial Actions", Advances in Neural Information Processing Systems, (15), pp.1271~1278 (2003)
- Bartlett, M.S, "Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior", Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR ’05), pp.568~573 (2005)
- Bartlett, M. S, "Fully Automatic Facial Action Recognition in Spontaneous Behavior", Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition (AFGR ’06), pp.223~230 (2006)
- Calvo, R. A, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications", IEEE Transactions on Affective Computing, 1(1), pp.18~37 (2010)
- Calvo, R. A, "Effect of Experimental Factors on the Recognition of Affective Mental States through Physiological Measures", Proc. 22nd Australasian Joint Conf. Artificial Intelligence (2009)
- tellano, G, "Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech, Affect and Emotion in Human-Computer Interaction", Springer, pp.92~103 (2008)
- ang, Y, "Manifold Based Analysis of Facial Expression", Journal of Image and Vision Computing, 24(6), pp.605~614 (2006)
- Chang, Y, "Probabilistic Expression Analysis on Manifolds", Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR ’04), (2), pp.520~527 (2004)
- Chang, Y, "Automatic 3D Facial Expression Analysis in Videos", Proc. IEEE Int’l Workshop Analysis and Modeling of Faces and Gestures (AMFG ’05), (3723), pp.293~307 (2005)
- Ciceri, M. R, "Multidimensional Emotional Appraisal Semantic Space(MEAS): Evaluating HM Affective Interactions", Knowledge-Based Intelligent Information and Engineering Systems, (4693), pp.395~402 (2007)
- Cohen, L, "Facial Expression Recognition from Video Sequences: Temporal and Static Modeling", Computer Vision and Image Understanding, 91(1), pp.160~187 (2003)
- Cohn, J. F, "Foundations of Human Computing: Facial Expression and Emotion", Proc. Eighth ACM Int’l Conf. Multimodal Interfaces (ICMI ’06), pp.233~238 (2006)
- Cohn, J. F, "Automatic Analysis and Recognition of Brow Actions and Head Motion in Spontaneous Facial Behavior", Proc. IEEE Int’l Conf. Systems, Man, and Cybernetics (SMC ’04), (1), pp.610~616 (2004)
- Cohn, J. F, "The Timing of Facial Motion in Posed and Spontaneous Smiles", Int’l Journal of Wavelets, Multiresolution and Information Processing, (2), pp.1~12 (2004)
- Damasio A. R, "Subcortical and cortical brain activity during the feeling of self-generated emotions", Nature Neuroscience, 3(10), pp.1049~1056 (2000)
- Darwin,C, "Expression of the Emotions in Man and Animals", pp.2002, New York
- Davidson, R. J, "Approach-Withdrawal and Cerebral Asymmetry: Emotional Expression and Brain Physiology I", Journal of Personality and Social Psychology, (58), pp.330~341 (1990)
- Ekman,P, "Universals and Cultural Differences in Facial Expressions of Emotion" (1971), Nebraska: Univ. of Nebraska Press
- Ekman,P, "Expression and the Nature of Emotion, Approaches to Emotion", pp.319~344 (1984), Erlbaum
- Ekman,P, "An Argument for Basic Emotions", pp.169~200 (1992), Cognition and Emotion
- Ekman, P, "Facial Action Coding System: A Technique for the Measurement of Facial Movement: Investigator’s Guide 2 Parts" (1978), Consulting Psychologists Press
- Ekman, P, "Unmasking the Face" (2003), Malor Books
- Ekman, P, "Facial Action Coding System", A Human Face, ~ (2002)
- Ekman, P, "Autonomic Nervous System Activity Distinguishes among Emotions", Science, (221), pp.1208~1210 (1983)
- Ekman, P, "The Duchenne Smile—Emotional Expression and Brain Physiology .2", Journal of Personality and Social Psychology, (58), pp.342~353 (1990)
- Kaliouby, R, "Real-Time Inference of Complex Mental States from Facial Expression and Head Gestures", Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition (CVPR ’04), 3, 154, (3), pp.154~ (2004)
- Fasel, B, "Latent Semantic Analysis of Facial Action Codes for Automatic Facial Expression Recognition", Proc. Sixth ACM Int’l Workshop Multimedia Information Retrieval (MIR ’04), pp.181~188 (2004)
- Fridlund, A. J, "Facial Expressions of Emotion, Nonverbal Behavior and Communication", pp.143~223 (1987), Erlbaum
- Frijda,N, "Emotion, Cognitive Structure, and Action Tendency", Cognition and Emotion, (1), pp.115~143 (1987)
- Gu, H, "An Automated Face Reader for Fatigue Detection", Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition (AFGR ’04), pp.111~116 (2004)
- Gunes, H, "Affect Recognition from Face and Body: Early Fusion versus Late Fusion", Proc. IEEE Int’l Conf. Systems, Man, and Cybernetics (SMC ’05), pp.3437~3443 (2005)
- Gunes, H, "Fusing Face and Body Display for Bi-Modal Emotion Recognition: Single Frame Analysis and Multi-Frame Post-Integration", Proc. First Int’l Conf. Affective Computing and Intelligent Interaction (ACII ’05), pp.102~111 (2005)
- Guo, G, "Learning from Examples in the Small Sample Case: Face Expression Recognition", IEEE Transactions on Systems, Man, and Cybernetics Part B, 35(3), pp.477~488 (2005)
- Haag, A, "Emotion Recognition Using Bio-Sensors: First Steps towards an Automatic System", Affective Dialogue Systems, pp.36~48 (2004)
- Heraz, A, "Predicting the Three Major Dimensions of the Learner’s Emotions from Brainwaves", World Academy of Science, Engineering and Technology, (25), pp.323~329 (2007)
- Hudlicka, E, "Increasing SIA architecture realism by modeling and adapting to affect and personality. In Multiagent Systems, Artificial Societies, and Simulated Organizations" (2002), Kluwer Academic Publishers, Dordrecht, The Netherlands
- Hudlicka,E, "To feel or not to feel:The role of affect in human–computer interaction", International Journal of Human-Computer Studies, 59(59), pp.1~32 (2003)
- Ioannou, S, "Emotion Recognition through Facial Expression Analysis Based on a Neurofuzzy Method", Neural Networks, (18), pp.423~435 (2005)
- Izard, C, "The Face of Emotion", Appleton-Century- Crofts, ~ (1971)
- Izard,C, "Innate and Universal Facial Expressions: Evidence from Developmental and Cross-Cultural Research", Psychological Bulletin, (115), pp.288~299 (1994)
- James, W, "What Is an Emotion", Mind, (9), pp.188~205 (1884)
- Ji, Q, "A Probabilistic Framework for Modeling and Real-Time Monitoring Human Fatigue", IEEE Systems, Man, and Cybernetics Part A, 36(5), pp.862~875 (2006)
- Kanade, T, "Comprehensive Database for Facial Expression Analysis", Proc. IEEE Int’l Conf. Face and Gesture Recognition (AFGR ’00), pp.46~53 (2000)
- Kapoor, A, "Multimodal Affect Recognition in Learning Environments", Proc. 13th Ann. ACM Int’l Conf. Multimedia, pp.677~682 (2005)
- Kapoor, A, "Automatic Prediction of Frustration", Int’l Journal of Human-Computer Studies, (65), pp.724~736 (2007)
- Khan, M. M, "Automated Facial Expression Classification and Affect Interpretation Using Infrared Measurement of Facial Skin Temperature Variations", ACM Transactions on Autonomous and Adaptive Systems, 1(1), pp.91~113 (2006)
- Kim, K., "Emotion Recognition System Using Short-Term Monitoring of Physiological Signals", Medical and Biological Engineering and Computing, (42), pp.419~427 (2004)
- Kleef, G. A. V, "An Interpersonal Approach to Emotion in Social Decision Making: The Emotions as Social Information Model", Advances in Experimental Social Psychology, (42), pp.45~96 (2010)
- Lee, C, "Facial Expression Analysis Using Nonlinear Decomposable Generative Models", Proc. Second IEEE Int’l Workshop Analysis and Modeling of Faces and Gestures (AMFG) (2005)
- tlewort, G. C., "Faces of Pain: Automated Measurement of Spontaneous Facial Expressions of Genuine and Posed Pain", Proc. Ninth ACM Int’l Conf. Multimodal Interfaces (ICMI ’07), pp.15~21 (2007)
- Liu, C, "Physiology-Based Affect Recognition for Computer- Assisted Intervention of Children with Autism Spectrum Disorder", Int’l Journal of Human-Computer Studies, (66), pp.662~677 (2008)
- Lucey, S, "Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face, Face Recognition", pp.275~286 (2007), I-Tech Education and Publishing
- Murphy, F. C, "Functional neuroanatomy of emotions: A meta-analysis, Cognitive, Affective, & Behavioral Neuroscience", 3(3), pp.207~233 (2003)
- Nakanishi, R, "Facial Skin Temperature Decreases in Infants with Joyful Expression", Infant Behavior and Development, (31), pp.137~144 (2008)
- Nasoz, F, "Emotion Recognition from Physiological Signals Using Wireless Sensors for Presence Technologies", Cognition, Technology and Work, (6), pp.4~14 (2004)
- Pantic, M, "Machine Analysis of Facial Expressions, Face Recognition", pp.377~416 (2007), I-Tech Education and Publishing
- Pantic, M, "Dynamics of Facial Expression: Recognition of Facial Actions and Their Temporal Segments Form Face Profile Image Sequences, IEEE Transactions on Systems", Man, and Cybernetics Part B, 36(2), pp.433~449 (2006)
- Pantic, M, "Toward an Affect-Sensitive Multimodal Human-Computer Interaction", Proc. IEEE, 91(9), pp.1370~1390 (2003)
- Pantic, M, "Facial Action Recognition for Facial Expression Analysis from Static Face Images", IEEE Transactions on Systems, Man, and Cybernetics Part B, 34(3), pp.1449~1461 (2004)
- Pantic, M, "Case-Based Reasoning for User-Profiled Recognition of Emotions from Face Images", Proc. 12th ACM Int’l Conf. Multimedia (Multimedia ’04), pp.391~394 (2004)
- Petrantonakis, P. C, "Emotion Recognition from EEG Using Higher Order Crossings", IEEE Transactions on Information Technology in Biomedicine, 14(2), pp.186~194 (2010)
- Phan, K. L, "Functional Neuroanatomy of Emotion: A Meta-Analysis of Emotion Activation Studies in PET and fMRI", NeuroImage, (16), pp.331~348 (2002)
- Picard, R. W, "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State", IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), pp.1175~1191 (2001)
- Ruch,W, "Will the Real Relationship between Facial Expression and Affective Experience Please Stand Up—The Case of Exhilaration", Cognition and Emotion, 9, pp.33~58 (1995)
- Russell, J. A, "Pancultureal aspects of the human conceptual organization of emotions", Journal of Personaltiy and Social Pscyhology, (45), pp.1281~1288 (1983)
- Russell, J. A, "Core Affect and the Psychological Construction of Emotion", Psychological Review, 110, pp.145~172 (2003)
- Scherer, K, "Multimodal Expression of Emotion: Affect Programs or Componential Appraisal Patterns", Emotion, (7), pp.158~171 (2007)
- Sebe, N., "Authentic Facial Expression Analysis", Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition (AFGR) (2004)
- Sharma, R, "Toward Multimodal Human-Computer Interface", Proc. IEEE, 86(5), pp.853~869 (1998)
- Sitaram, R, "Real-time support vector classification and feedback of multiple emotional brain states, NeuroImage", neuroimage (2010)[DOI]
- Sugimoto1, Y, "A Method for Detecting Transitions of Emotional States using a Thermal Facial Image based on a Synthesis of Facial Expressions", Robotics and Autonomous Systems, (31), pp.147~160 (2000)
- Teeters, A, "Self-Cam: Feedback from What Would Be Your Social Partner", Proc. ACM SIGGRAPH’06, pp.138~ (2006)
- Tian, Y. L, "Facial Expression Analysis, Handbook of Face Recognition", pp.247~276 (2005), Springer
- Tomkins, S. S, "Affect Imagery Consciousness: Volume I" (1962), Springer
- Tong, Y, "Facial Action Unit Recognition by Exploiting Their Dynamics and Semantic Relationships", IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(10), pp.1683~1699 (2007)
- Valstar, M.F, "How to Distinguish Posed from Spontaneous Smiles Using Geometric Features", Proc. Ninth ACM Int’l Conf. Multimodal Interfaces (ICMI ’07), pp.38~45 (2007)
- Valstar, M, "Spontaneous versus Posed Facial Behavior: Automatic Analysis of Brow Actions", Proc. Eight Int’l Conf. Multimodal Interface (ICMI ’06), pp.162~170 (2006)
- Valstar, M, "Motion History for Facial Action Detection from Face Video", Proc. IEEE Int’l Conf. Systems, Man, and Cybernetics (SMC ’04), (1), pp.635~640 (2004)
- Villon, O, "A User-Modeling Approach to Build User’s Psycho-Physiological Maps of Emotions Using Bio-Sensors", Proc. IEEE RO-MAN, 15th IEEE Int’l Symp. Robot and Human Interactive Comm., Session Emotional Cues in Human-Robot Interaction, pp.269~276 (2006)
- Vyzas, E, "Affective Pattern Classification", Proc. AAAI Fall Symp. Series: Emotional and Intelligent: The Tangled Knot of Cognition, pp.176~182 (1998)
- Wagner, N. J, "From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification", Proc. IEEE Int’l Conf. Multimedia and Expo, pp.940~943 (2005)
- Wang, H, "Facial Expression Decomposition", Proc. Ninth IEEE Int’l Conf. Computer Vision (ICCV ’03), pp.958~ (2003)
- Wang, J, "3D Facial Expression Recognition Based on Primitive Surface Feature Distribution", Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition(CVPR ’06), (2), pp.1399~1406 (2006)
- Wen, Z, "Capturing Subtle Facial Motions in 3D Face Tracking", Proc. Ninth IEEE Int’l Conf. Computer Vision (ICCV’03), pp.1343~1350 (2003)
- Whitehill, J, "Haar Features for FACS AU Recognition", Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition (AFGR ’06), pp.217~222 (2006)
- Wundt, W, "Fundamentals of Physiological Psychology(5th Ed.)" (1905), Engelmann, Leipzig
- Yeasin, M, "Recognition of Facial Expressions and Measurement of Levels of Interest from Video", IEEE Transactions on Multimedia, 8(3), pp.500~507 (2006)
- Yin, L, "A 3D Facial Expression Database for Facial Behavior Research", Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition (AFGR ’06), pp.211~216 (2006)
- Yoshitomi, Y, "Facial expression recognition using thermal image processing and neural network", The 1997 6th IEEE International Workshop on Robot and Human Communication, pp.1~6 (1997)
- Zeng, Z, "Spontaneous Emotional Facial Expression Detection", J. Multimedia, 1(5), pp.1~8 (2006)
- Zeng, Z, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions", IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), pp.39~58 (2009)
- Zhang, Y, "Active and Dynamic Information Fusion for Facial Expression Understanding from Image Sequences", IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), pp.699~714 (2005)
- Zhu, Z, "Robust Real-Time Face Pose and Facial Expression Recovery", Proc. IEEE Int’l Conf. Computer Vision and Pattern Recognition(CVPR ’06), (1), pp.681~688 (2006)
