얼굴 열화상 기반 감정인식을 위한 CNN 학습전략
Divide and Conquer Strategy for CNN model in Facial Emotion Recognition based on Thermal Images
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
감정인식은 응용 분야의 다양성으로 많은 연구가 이루어지고 있는 기술이며, RGB 영상은 물론 열화상을이용한 감정인식의 필요성도 높아지고 있다. 열화상의 경우는 RGB 영상과 비교해 조명 문제에 거의 영향을받지 않는 장점이 있으나 낮은 해상도로 성능 높은 인식 기술을 필요로 한다. 본 논문에서는 얼굴 열화상 기반 감정인식의 성능을 높이기 위한 Divide and Conquer 기반의 CNN 학습전략을 제안하였다. 제안된 방법은먼저 분류가 어려운 유사 감정 클래스를 confusion matrix 분석을 통해 동일 클래스 군으로 분류하도록 학습시키고, 다음으로 동일 클래스 군으로 분류된 감정 군을 실제 감정으로 다시 인식하도록 문제를 나누어서 해결하는 방법을 사용하였다. 실험을 통하여, 제안된 학습전략이 제시된 모든 감정을 하나의 CNN 모델에서 인식하는 경우보다 모든 실험에서 높은 인식성능을 보이는 것을 확인하였다.
Abstract
The ability to recognize human emotions by computer vision is a very important task, with many potential applications. Therefore the demand for emotion recognition using not only RGB images but also thermal images is increasing. Compared to RGB images, thermal images has the advantage of being less affected by lighting conditions but require a more sophisticated recognition method with low-resolution sources. In this paper, we propose a Divide and Conquer-based CNN training strategy to improve the performance of facial thermal image-based emotion recognition. The proposed method first trains to classify difficult-to-classify similar emotion classes into the same class group by confusion matrix analysis and then divides and solves the problem so that the emotion group classified into the same class group is recognized again as actual emotions. In experiments, the proposed method has improved accuracy in all the tests than when recognizing all the presented emotions with a single CNN model.
참고문헌 (22)
- Chai M. Tyng, "The Influences of Emotion on Learning and Memory", Frontiers in Psychology, 8, pp.1~22 (2017), Frontiers Media SA[DOI]
- Sebastian Zepf, "Driver Emotion Recognition for Intelligent Vehicles", ACM Computing Surveys, 53(3), pp.1~30 (2021), Association for Computing Machinery (ACM)[DOI]
- Elaheh Yadegaridehkordi, "Affective computing in education: A systematic review and future research", Computers & Education, 142, pp.103649~ (2019), Elsevier BV[DOI]
- Zhentao Liu, "A facial expression emotion recognition based human-robot interaction system", IEEE, 2017, pp.4(4), 668~676[DOI]
- "Services), Technology, Application Area, End User, Vertical, Region-Global Forecast to 2026" (2021)[URL]
- Z. Yu, "Image based Static Facial Expression Recognition with Multiple Deep Network Learning", Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp.435~442 (2015)
- I.Michael Revina, "A Survey on Human Face Expression Recognition Techniques", Journal of King Saud University - Computer and Information Sciences, 33(6), pp.619~628 (2021), Elsevier BV[DOI]
- Ke Zhao, "Neural Responses to Rapid Facial Expressions of Fear and Surprise", Frontiers in Psychology, 8, pp.1~8 (2017), Frontiers Media SA[DOI]
- D. Poster, "A Large-scale, Time-synchronized Visible and Thermal Face Dataset", Proceedings of the 2021IEEE, 2021, 1559~1568
- "The Use of AI for Thermal Emotion Recognition: A Review of Problems and Limitations in Standard Design and Data" (2020)[URL]
- M. Kopaczka, "A Fully Annotated Thermal Face Database and Its Application for Thermal Facial Expression Recognition", Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (2018)
- Yomna M. Elbarawy, "Facial Expressions Recognition in Thermal Images based on Deep Learning Techniques", International Journal of Image, Graphics and Signal Processing, 11(10), pp.1~7 (2019), MECS Publisher[DOI]
- F. He, "Advances in Neural Information Processing Systems", pp.1141~1150 (2019)
- Haibing Wu, "Towards dropout training for convolutional neural networks", Neural Networks, 71, pp.1~10 (2015), Elsevier BV[DOI]
- Justin M. Johnson, "Survey on deep learning with class imbalance", Journal of Big Data, 6(1), pp.1~54 (2019), Springer Science and Business Media LLC[DOI]
- Shangfei Wang, "A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference", IEEE Transactions on Multimedia, 12(7), pp.682~691 (2010), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Karen Panetta, "A Comprehensive Database for Benchmarking Imaging Systems", IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(3), pp.509~520 (2020), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- "Retinaface: Single-stage Dense Face Localisation in the Wild" (2019)[URL]
- C. Shorten, "A Survey on Image Data Augmentation for Deep Learning", Journal of Big Data, 6(1), pp.1~48[DOI]
- K. He, "Deep Residual Learning for Image Recognition", Proceedings of the 2016IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
- S. K. M, Kamath, "TERNet : A Deep Learning Approach for Thermal Face Emotion Recognition", Mobile Multimedia, 2019, pp.10993, 1~7