본문 바로가기
KERD
논문KCI 등재

감정인식 기술의 현황과 전망

And Perspectives on Emotion Recognition Technologies

김남수
에스케이텔레콤 (주) (SKTelecom)|2009.10|Telecommunications Review|제19권 제5호|pp. 773-785

3

조회

0

다운로드

4

인용

0

참고문헌

키워드

음성인식얼굴표정인식감정인식상황인지Emotion recognitionContext awarenessSpeech recognitionFacial expression recognition
주제분류전자/정보통신공학

초록

감정인식은 인간 중심의 human-to-machine interface의 가장 중요한 요소 중 하나이다. 현재의 감정인식 기술 은 주로 얼굴 영상, 음성, 생체신호를 이용하여 특징을 추출하고 이를 미리 학습시킨 인식 모델을 통하여 각 감정의 범 주로 분류한다. 본 논문에서는 영상과 오디오를 이용하는 다양한 감정인식 기술에 대하여 설명하고 현재의 연구, 개발추세 를 간략히 정리한다. 각 기법의 장단점을 비교하며 향후 실제적 응용을 위해서 필요한 기술적 요건을 제시한다. 또한 통신 이론을 바탕으로 감정표출과 감정인식 과정을 심도 있게 분석하고 이를 바탕으로 기술적인 발전 방향을 제시한다. 감정인 식은 감정표현과 결합하여 진정한 인간 중심의 human-to-machine interface를 제공할 것으로 전망한다.

Abstract

Emotion recognition is one of the most important factors in human-centric human-to-machine interface. In most of the state of the art technologies on emotion recognition, features are extracted from the image, speech and biosignals and they are classified into specific emotional categories based on pre-trained recognition models. In this paper, we introduce various techniques of audio-visual-based emotion recognition and summarize the current research directions. Comparison among different techniques in conjunction with the practical requirements for commercial application is provided. In addition, an in-depth analysis on the emotion generation and recognition is given based on the communications theory, and several perspectives on the technological advancement are provided. Emotion recognition combined with emotion expression capability is expected to be placed at the heart of human-centric human-to-machine interface.

참고문헌 (66)

  1. R. Cowie, "Emotion recognition in humancomputer interaction", IEEE Signal Processing Magazine, pp.32~80 (2001)
  2. Z. Zeng, "A survey of affect recognition methods: audio, visual, and spontaneous expressions", IEEE Trans. Pattern Analysis and machine Intelligence, 31(1) (2009)
  3. C. M. Lee, "Toward detecting emotions in spoken dialogs", IEEE Trans. Speech and Audio Processing, 130(2) (2005)
  4. Q. Ji, "A probabilistic framework for modeling and real-time monitoring human fatigue", IEEE Trans. Systems, Man and Cybernetics Part A, 36(5), pp.862~875 (2006)
  5. H. Gunes, "Affect recognition from face and body: early fusion versus late fusion", IEEE Int’l Conf. Systems, Man and Cybernetics (SMC ’05), pp.3437~3443 (2005)
  6. M. F. Valstar, "How to distinguish posed from spontaneous smiles using geometric features", Proc. Ninth ACM Int’l Conf. Multimedia Interfaces (ICMI ’07), pp.38~45 (2007)
  7. A. Kapoor, "Automatic prediction of frustration", Int’l J. Human-Computer Studies, 64(8), pp.724~736 (2007)
  8. C. D. Katsis, "Toward emotion recognition in carracing drivers: A biosignal processing approach", IEEE Trans. Systems, Man and Cybernetics Part A, 38(3), pp.502~512 (2008)
  9. L. C. DeSilva, "Facial emotion recognition using multi-modal information", Proc. IEEE Int’l Conf. Inf., Commun., Signal Process, pp.397~401 (1997)
  10. J. T. Cacioppo, "Inferring physiological significance from physiological signals", Amer. Psychol, 45(1), pp.16~28 (1990)
  11. P. Ekman, "Emotion in Human Face. Second" (1982), Cambridge Univ. Press
  12. M. Greenwald, "Affective judgment and psychophysiological response: dimensional covariance in the evaluation of pictorial stimuli", Psychophysiology, 3, pp.51~64 (1989)
  13. J. Russell, "Evidence for a threefactor theory of emotions", J. Research in Personality, 11, pp.273~294 (1977)
  14. K. R. Scherer, "Appraisal theory. in: Handbook of Cognition and Emotion", pp.637~663 (1999), Wiley
  15. C. L. Lisetti, "MAUI: A multimodal affective user interface", Proc. 10 th Int’l Conf. Multimedia (Multimedia ’02), pp.161~170 (2002)
  16. Z. Duric, "Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction", IEEE, 90(7), pp.1272~1289 (2002), Proc
  17. L. Maat, "Gaze-X: Adaptive affective multimodal interface for single-user office scenarios", Proc. Eighth ACM Int’l Conf. Multimodal Interfaces (ICMI ’06), pp.171~178 (2006)
  18. E. Navas, "An objective and subjective study of the role of semantics and prosodic features in building corpora for emotional TTS", IEEE Tr. Audio, Speech and Language Processing, 14(4), pp.1117~1127 (2006)
  19. C. M. Bishop, "Pattern Recognition and Machine Learning" (2006), Springer
  20. P. Ekman, "What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System" (2005), Oxford Univ. Press
  21. A. C. de, "Facial expression of pain: An evolutionary account", Behavioral and Brain Sciences, 25(4), pp.439~488 (2002)
  22. Y. Chang, "Manifold based analysis of facial expression", J. Image and Vision Computing, 24(6), pp.605~614 (2006)
  23. M. Pantic, "Dynamics of facial expression: Recognition of facial actions and their temporal segments form face profile image sequences", IEEE Tran. Systems, Man and Cybernetics Part B, 36(2), pp.433~449 (2006)
  24. M. Valstar, "Spontaneous versus posed facial behavior: Automatic analysis of brow actions", Proc. Eighth Int’l Conf. Multimodal Interfaces (ICM‘I 06), pp.162~170 (2006)
  25. M. S. Bartlett, "A prototype for automatic recognition of spontaneous facial actions", Advances in Neural Information Processing Systems, 15, pp.1271~1278 (2003)
  26. I. Whitehill, "Harr features for FACS AU recognition", Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition (AFGR ’06), pp.217~222 (2006)
  27. K. Anderson, "A real-time automated system for recognition of human faxial expressions", IEEE Tran. Systems, Man and Cybernetics Part B, 36(1), pp.96~105 (2006)
  28. M. Valstar, "Motion history for facial action detection from face video", Proc. IEEE Int’l Conf. Systems, Man and Cybernetics (SMC‘ 04), pp.635~640 (2004)
  29. L. Cohen, "Facial expression recognition from video sequences: Temporal and static modeling", Computer Vision and Image Understanding, 91(1-2), pp.160~187 (2003)
  30. S. Steidl, "Off all things the measure is man: Automatic classification of emotions and inter-labeler consistency", Proc. IEEE Int’l Conf. Acoustics, Speech and Signal Processing (ICASSP‘ 05), pp.317~320 (2005)
  31. Z. Wen, "Capturing subtle facial motions in 3D face tracking", Proc. Ninth Int’l Conf. Computer Vision (ICCV ’03), pp.1343~1350 (2003)
  32. J. F. Cohn, "Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior", Proc. IEEE Int’l Conf. Systems, Man and Cybernetics (SMC‘ 04), pp.610~616 (2004)
  33. Emotion analysis in man-machine interaction systems, "T. Balomenos", 318~328, pp.LNCS 3361 (2005)
  34. M. S. Bartlett, "Recognizing facial expression: Machine learning and application to spontaneous behavior", Proc. Int’l Conf. Computer Vision and Pattern recognition (CVPR ’05), pp.568~573 (2005)
  35. B. Fasel, "Latent semantic analysis of facial action codes for automatic facial expression recognition", Proc. Sixth ACM Int’l Workshop Multimedia Information Retrieval (MIR ’04), pp.181~188 (2004)
  36. G. C. Littlewort, "Faces of pain: Automated measurement of spontaneous facial expressions of genuine and posed pain", Ninth ACM Int’l Conf. Multimodal Interfaces (ICMI ’07), pp.15~21 (2007)
  37. M. Pantic, "Case-based reasoning for user-profiled recognition of emotions from face images", Proc. 12th ACM Int’l Conf. Multimedia (Multimedia ’04), pp.391~394 (2004)
  38. N. Sebe, "facial expression analysis", Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition (AFGR) (2004)
  39. M. Yeasin, "Recognition of facial expressions and measurement of levels of interest from video", IEEE Trans. Multimedia, 8(3), pp.500~507 (2006)
  40. Z. Zeng, "Spontaneous emotional facial expression detection", J. Multimedia, 1(5), pp.1~8 (2006)
  41. J. Hirschberg, "Distinguishing deceptive from non-deceptive speech", INTERSPEECH ’05, pp.1833~1836 (2005)
  42. J. Liscombe, "Detecting certainness in spoken tutorial dialogues", INTERSPEECH ’05 (2005)
  43. O. W. Kwon, "Emotion recognition by speech signals", INTERSPEECH ’03 (2003)
  44. T. Zhang, "Children’s emotion recognition in an intelligent tutoring scenario", INTERSPEECH ’04 (2004)
  45. A. Batliner, "How to find trouble in communication", Speech Communication, 40, pp.117~143 (2003)
  46. J. Ang, "Prosody-based automatic detection of annoyance and frustration in human-computer dialog", ICSLP’02 (2002)
  47. K. P. Truong, "Automatic discrimination between laughter and speech", Speech Communication, 49, pp.144~158 (2007)
  48. S. Matos, "Detection of cough signals in continuous audio recordings using HMM", IEEE Trans. Biomedical Eng, 53(6), pp.1078~1083 (2006)
  49. P. Pal, "Emotion detection from infant facial expressions and cries", Proc. IEEE Int’l Conf. Acoustics, Speech and Signal Processing (ICASSP ’06), pp.721~724 (2006)
  50. L. Devillers, "Real-life emotions detection with lexical and paralinguistic cues on humanhuman call center dialogs", Proc. Ninth Int’l Conf. Spoken Language Processing (INTERSPEECH‘ 06) (2006)
  51. M. Graciarena, "Combining prosodic, lexical and cepstral systems for deceptive speech detection", Proc. IEEE Int’l Conf. Acoustics, Speech and Signal Processing (ICASSP‘ 06), pp.1033~1036 (2006)
  52. D. J. Litman, "Predicting student emotions in computer-human tutoring dialogues", Proc. 42nd Ann. Meeting of the Assoc. Computational Linguistics (ACL‘ 04) (2004)
  53. D. Neiberg, "Emotion recognition in spontaneous speech using GMM", Proc. Int’l Conf. Spoken Language Processing (INTERSPEECH‘ 06), pp.809~812 (2006)
  54. C. Busso, "Analysis of emotion recognition using facial expressions, speech and multimodal information", Proc. Sixth ACM Int’l Conf. Multimodal Interfaces (ICMI ’04), pp.205~211 (2004)
  55. H. J. Go, "Emotion recognition from facial image and speech signal", Proc. Int’l Conf. Soc. Of Instrument and Control Engineers, pp.2890~2895 (2003)
  56. H. Hoch, "Bimodal fusion of emotional data in automotive environment", Proc. IEEE Int’l Conf. Acoustics, Speech and Signal Processing (ICASSP ’05), pp.1085~1088 (2005)
  57. Z. Zeng, "Audio-visual spontaneous emotion recognition. in: Artificial Intelligence for Human Computing", pp.72~90 (2007), Springer
  58. Z. Zeng, "Audio-visual affect recognition", IEEE Trans. Multimedia, 9(2), pp.424~428 (2007)
  59. G. Caridakis, "Modeling naturalistic affective states via facial and vocal expression recognition", Proc. Eighth ACM Int’l Conf. Multimodal Interfaces (ICMI ’06), pp.146~154 (2006)
  60. F. Fragopanagos, "Emotion recognition in human-computer interaction", Neural Networks, 18, pp.389~405 (2005)
  61. N. Sebe, "Emotion recognition based on joint visual and audio cues", Proc. 18th Int’l Conf. Pattern Recognition (ICPR‘ 06), pp.1136~1139 (2006)
  62. M. Song, "Audio-visual-based emotion recognition: A new approach", Proc. Int’l Conf. Computer Vision and Pattern Recognition (CVPR‘ 04), pp.1020~1025 (2004)
  63. K. Karpouzis, "Modeling naturalistic affective states via facial, vocal, and bodily expression recognition", LNAI 4451, pp.91~112 (2007)
  64. S. Petridis, "Audio-visual discrimination between laughter and speech", Proc. IEEE Int’l Conf. Acoustics, Speech and Signal Processing (ICASSP ’08), pp.5117~5120 (2008)
  65. Y. Wang, "Recognizing human emotion from audiovisual information", Proc. IEEE Int’l Conf. Acoustics, Speech and Signal Processing (ICASSP ’05),, pp.1125~1128 (2005)
  66. D. Skillicorn, "Understanding Complex Datasets: Data Mining with Matrix Decompositions" (2007), Chapman&Hall

관련 논문AI추천

인간과 기계의 지능형 인터페이스를 위한 감정 인식 기술박철헌, 이종석에스케이텔레콤 (주) · Telecommunications Review2008 · 제18권 제3호사용자의 성향 기반의 얼굴 표정을 통한 감정 인식률 향상을 위한 연구신동희, 이종식한국감성과학회 · 감성과학2014 · 제17권 제1호정서 재인 방법 고찰을 통한 통합적 모델 모색에 관한 연구박미숙, 박지은, 손진훈한국감성과학회 · 감성과학2011 · 제14권 제1호얼굴영상과 음성을 이용한 멀티모달 감정인식김동주, 이현구(사)디지털산업정보학회 · (사)디지털산업정보학회 논문지2012 · 제8권 제1호사용자의 감성인식을 통한 감성통신 시스템조면균대한임베디드공학회 · 대한임베디드공학회논문지2011 · 제6권 제4호Emotion Recognition based on Multiple Modalities김동주, 이현구, 홍광석한국융합신호처리학회 · 융합신호처리학회 논문지2011 · 제12권 제4호

이 간행물의 다른 논문 (11건)

주요 국가의 IT 융합 기술 정책 현황 및 전망정명애pp. 698-707융합 기술의 동향: 인지과학과 정보 기술(IT)의 융합을 중심으로박지은, 손진훈pp. 708-716융합컴퓨터의 유형과 주요 기술 동향성낙선, 정명애pp. 717-729IT 융합 기술 활용 유헬스케어 구현을 위한 생체정보 검출용 센서기술이국녕, 성우경pp. 730-740지능형 자동차용 전동 시스템 및 구동모터 개발허진pp. 741-758
원문 보기