Robust Facial Expression Recognition in Occluded VR Environments: Leveraging Temporal Information for Enhanced Recognition
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Driven by VR’s rapid adoption and the need for natural interaction, this study tackles FER when head-mounted displays (HMDs) occlude key facial regions. We propose a multi-region fusion architecture that processes upper-face, lower-face, and full-face streams in parallel, enhanced by a temporal attention mechanism to emphasize crucial frames. On the DFEW dataset with synthetic occlusion, we benchmarked CNN-LSTM, C3D, Transformer, and our model. Ours achieves 81.2% accuracy at 12.3 FPS under occlusion—a 19.9-point gain over the best baseline—demonstrating robust, real-time FER and maintaining competitive inference speed. This confirms that combining spatial region processing with temporal dynamics yields effective emotion detection in immersive VR, offering practical solutions to enhance emotional communication and user experience across social VR, gaming, and remote collaboration.
Abstract
Driven by VR’s rapid adoption and the need for natural interaction, this study tackles FER when head-mounted displays (HMDs) occlude key facial regions. We propose a multi-region fusion architecture that processes upper-face, lower-face, and full-face streams in parallel, enhanced by a temporal attention mechanism to emphasize crucial frames. On the DFEW dataset with synthetic occlusion, we benchmarked CNN-LSTM, C3D, Transformer, and our model. Ours achieves 81.2% accuracy at 12.3 FPS under occlusion—a 19.9-point gain over the best baseline—demonstrating robust, real-time FER and maintaining competitive inference speed. This confirms that combining spatial region processing with temporal dynamics yields effective emotion detection in immersive VR, offering practical solutions to enhance emotional communication and user experience across social VR, gaming, and remote collaboration.