얼굴 영상 기반의 심박수 추정을 위한 딥러닝 모델의 경량화 기법
Lightweight Deep Learning Model for Heart Rate Estimation from Facial Videos
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This paper proposes a deep learning method for estimating the heart rate from facial videos. Our proposed method estimates remote photoplethysmography (rPPG) signals to predict the heart rate. Although there have been proposed several methods for estimating rPPG signals, most previous methods can not be utilized in low-power single board computers due to their computational complexity. To address this problem, we construct a lightweight student model and employ a knowledge distillation technique to reduce the performance degradation of a deeper network model. The teacher model consists of 795k parameters, whereas the student model only contains 24k parameters, and therefore, the inference time was reduced with the factor of 10. By distilling the knowledge of the intermediate feature maps of the teacher model, we improved the accuracy of the student model for estimating the heart rate. Experiments were conducted on the UBFC-rPPG dataset to demonstrate the effectiveness of the proposed method. Moreover, we collected our own dataset to verify the accuracy and processing time of the proposed method on a real-world dataset. Experimental results on a NVIDIA Jetson Nano board demonstrate that our proposed method can infer the heart rate in real time with the mean absolute error of 2.5183 bpm.
Abstract
This paper proposes a deep learning method for estimating the heart rate from facial videos. Our proposed method estimates remote photoplethysmography (rPPG) signals to predict the heart rate. Although there have been proposed several methods for estimating rPPG signals, most previous methods can not be utilized in low-power single board computers due to their computational complexity. To address this problem, we construct a lightweight student model and employ a knowledge distillation technique to reduce the performance degradation of a deeper network model. The teacher model consists of 795k parameters, whereas the student model only contains 24k parameters, and therefore, the inference time was reduced with the factor of 10. By distilling the knowledge of the intermediate feature maps of the teacher model, we improved the accuracy of the student model for estimating the heart rate. Experiments were conducted on the UBFC-rPPG dataset to demonstrate the effectiveness of the proposed method. Moreover, we collected our own dataset to verify the accuracy and processing time of the proposed method on a real-world dataset. Experimental results on a NVIDIA Jetson Nano board demonstrate that our proposed method can infer the heart rate in real time with the mean absolute error of 2.5183 bpm.
참고문헌 (18)
- M. Poh, "Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam", IEEE Transactions on Biomedical Engineering, 58(1), pp.7~11 (2011)
- M. Lewandowska, "Measuring Pulse Rate with a Webcam — A Non-contact Method for Evaluating Cardiac Activity", Proceedings of 2011 Federated Conference on Computer Science and Information Systems, pp.405~410 (2011)
- G. de Haan, "Robust Pulse Rate From Chrominance-Based rPPG", IEEE Transactions on Biomedical Engineering, 60(10), pp.2878~2886 (2013)
- W. Wang, "Algorithmic Principles of Remote PPG", IEEE Transactions on Biomedical Engineering, 64(7), pp.1479~1491 (2017)
- W. Chen, "DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks", Proceedings of the European Conference on Computer Vision, pp.349~365 (2018)
- Z. Yu, "Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-temporal Networks", Proceedings of the British Machine Vision Conference (2019)
- E. M. Nowara, "The Benefit of Distraction: Denoising Camera-Based Physiological Measurements Using Inverse Attention", Proceedings of the IEEE, 2021, 4955~4964
- Z. Yu, "PhysFormer: Facial Video-Based Physiological Measurement With Temporal Difference Transformer", Proceedings of the IEEE, 2022, 4186~4196
- S. Bobbia, "Unsupervised Skin Tissue Segmentation for Remote Photoplethysmography", Pattern Recognition Letters, 124, pp.82~90 (2019)
- K. Zhang, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks", IEEE Signal Processing Letters, 23(10), pp.1499~1503 (2016)
- G. E. Hinton, "Distilling the Knowledge in a Neural Network", ~ (2015)
- S. Lin, "Knowledge Distillation via the Target-Aware Transformer", Proceedings of the IEEE, 2022, 10915~10924
- E. Lee, "Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-learner", Proceedings of the European Conference on Computer Vision, pp.392~409 (2020)
- X. Niu, "SynRhythm:Learning a Deep Heart Rate Estimator from General to Specific", Proceedings of the International Conference on Pattern Recognition, pp.3580~3585 (2018)
- F. Bousefsaf, "3d Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video", Applied Sciences, 9(20), pp.4364~ (2019)
- P. Gupta, "Accurate Heart-rate Estimation from Face Videos Using Qualitybased Fusion", Proceedings of the 2017 International Conference on Image Processing, pp.4132~4136 (2017)
- R. Song, "PulseGAN: Learning to Generate Realistic Pulse Waveforms in Remote Photoplethysmography", IEEE Journal of Biomedical and Health Informations, 25(5), pp.1373~1384 (2021)
- Z. Yu, "Remote Heart Rate Measurement from Highly Compressed Facial Videos: An End-to-end Deep Learning Solution with Video Enhancement", Proceedings of the IEEE, 2019, 151~160