Emotion Recognition based on Multiple Modalities
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Emotion recognition plays an important role in the research area of human-computer interaction, and it allows a more natural and more human-like communication between humans and computer. Most of previous work on emotion recognition focused on extracting emotions from face, speech or EEG information separately. Therefore, a novel approach is presented in this paper, including face, speech and EEG, to recognize the human emotion. The individual matching scores obtained from face, speech, and EEG are combined using a weighted-summation operation, and the fused-score is utilized to classify the human emotion. In the experiment results, the proposed approach gives an improvement of more than 18.64% when compared to the most successful unimodal approach, and also provides better performance compared to approaches integrating two modalities each other. From these results, we confirmed that the proposed approach achieved a significant performance improvement and the proposed method was very effective.
Abstract
Emotion recognition plays an important role in the research area of human-computer interaction, and it allows a more natural and more human-like communication between humans and computer. Most of previous work on emotion recognition focused on extracting emotions from face, speech or EEG information separately. Therefore, a novel approach is presented in this paper, including face, speech and EEG, to recognize the human emotion. The individual matching scores obtained from face, speech, and EEG are combined using a weighted-summation operation, and the fused-score is utilized to classify the human emotion. In the experiment results, the proposed approach gives an improvement of more than 18.64% when compared to the most successful unimodal approach, and also provides better performance compared to approaches integrating two modalities each other. From these results, we confirmed that the proposed approach achieved a significant performance improvement and the proposed method was very effective.
참고문헌 (15)
- Picard,R, "Affective computing" (1997), MIT Press
- Cowie, R., "Emotion recognition in human-computer interaction", IEEE Signal Processing Magazine (2001)
- M. Song, "Audio-visual based emotion recognition-a new approach", Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp.1020~1025 (2004)
- Chang Ryu, "A portable neurofeedback system and EEG-analysis methods for evaluation", World Congress on Medical Physics and Biomedical Engineering 2006, 14(8), pp.1167~1169 (2007)
- Schaaff, K., "Towards an EEG-based emotion recognizer for humanoid robots", The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp.792~792 (2009)
- M.Turk, "Eigenfaces for recognition", J. Cognitive Neuro-sci, 3(1), pp.71~86 (1991)
- Y. Jian, "Two-dimensional PCA: A new approach to appearance-based face representation and recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), pp.131~137 (2004)
- D. O’Shaughnessy, "Speech Communication - Human and Machine" (1987), Addison-Wesley
- Reynolds, D. A., "Speaker verification using adapted Gaussian mixture models", Digital Signal Process, 10, pp.19~41 (2000)
- P. Jourlin, "Acoustic-labial speaker authentication", Pattern Recognition Letter, 18, pp.853~858 (1997)
- A. Ross, "Information fusion in biometrics", Pattern Recognition Letter, 24, pp.2115~2125 (2003)
- Michael J. Lyons, "Coding Facial Expressions with Gabor Wavelets", Proceedings, Third IEEE International Conference on Automatic Face and Gesture Recognition, pp.200~205 (1998)
- Kyung Hak Hyun, "Improvement of emotion recognition by Bayesian classifier using non-zero-pitch concept", IEEE International Workshop on Robot and Human Interactive Communication, pp.312~316 (2005)
- International affective picture system (IAPS): Affective ratings of pictures and instruction manual, "P. Lang", University of Florida, ~, pp.Technical Report A-6 (2005)
- Papanicolaou A. C, "Task-related EEG Asymmetries: a Comparison of Alpha Blocking and Beta Enhancement", International Journal of Neuroscience, 30, pp.81~85 (1998)