Effective Hand Gesture Recognition by Key Frame Selection and 3D Neural Network
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
This paper presents an approach for dynamic hand gesture recognition by using algorithm based on 3D Convolutional Neural Network (3D_CNN), which is later extended to 3D Residual Networks (3D_ResNet), and the neural network based key frame selection. Typically, 3D deep neural network is used to classify gestures from the input of image frames, randomly sampled from a video data. In this work, to improve the classification performance, we employ key frames which represent the overall video, as the input of the classification network. The key frames are extracted by SegNet instead of conventional clustering algorithms for video summarization (VSUMM) which require heavy computation. By using a deep neural network, key frame selection can be performed in a real-time system. Experiments are conducted using 3D convolutional kernels such as 3D_CNN, Inflated 3D_CNN (I3D) and 3D_ResNet for gesture classification. Our algorithm achieved up to 97.8% of classification accuracy on the Cambridge gesture dataset. The experimental results show that the proposed approach is efficient and outperforms existing methods.
Abstract
This paper presents an approach for dynamic hand gesture recognition by using algorithm based on 3D Convolutional Neural Network (3D_CNN), which is later extended to 3D Residual Networks (3D_ResNet), and the neural network based key frame selection. Typically, 3D deep neural network is used to classify gestures from the input of image frames, randomly sampled from a video data. In this work, to improve the classification performance, we employ key frames which represent the overall video, as the input of the classification network. The key frames are extracted by SegNet instead of conventional clustering algorithms for video summarization (VSUMM) which require heavy computation. By using a deep neural network, key frame selection can be performed in a real-time system. Experiments are conducted using 3D convolutional kernels such as 3D_CNN, Inflated 3D_CNN (I3D) and 3D_ResNet for gesture classification. Our algorithm achieved up to 97.8% of classification accuracy on the Cambridge gesture dataset. The experimental results show that the proposed approach is efficient and outperforms existing methods.
참고문헌 (22)
- Q. D. Smedt, "Skeleton-Based Dynamic Hand Gesture Recognition", Computer Vision and Pattern Recognition Workshops (CVPRW) (2016)
- U. Cote-Allard, "Transfer Learning for sEMG Hand gesture recognition Using Convolutional Neural Networks", IEEE International Conference on Systems, Man, and Cybernetics (SMC) (2017)
- M. H. Rahman, "Hand Gesture Recognition using Multiclass Support Vector Machine", International Journal of Computer Applications, 74(1) (2013)
- D. Tran, "Learning spatiotemporal features with 3D convolutional networks", Proc. of IEEE Int. Conf. Comput. Vis. (ICCV), pp.4489~4497 (2015)
- G. Zhu, "Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM", IEEE Access, 5, pp.4517~4524 (2017)
- J. Donahue, "Long-term recurrent convolutional networks for visual recognition and description", Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
- V. John, "Deep Learning-Based Fast Hand Gesture Recognition Using Representative Frames", International Conference on Digital Image Computing: Techniques and Applications (DICTA) (2016)
- R. F. Rachmadi, "Video classification using compacted dataset based on selected keyframe", IEEE Region 10 Conference (TENCON) (2016)
- H. Tang, "Fast and powerful hand gesture recognition extraction and feature fusion", NeuroComputing (2019)
- H. Jiang, "Adaptive key frame extraction from RGB-D for hand gesture recognition", Tenth International Conference on Digital Image Processing (ICDIP 2018) (2018)
- J. Carreira, "Quo Vadis, Action Recognition? A New Model and the Kinetics Datase", Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
- K. Hara, "Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?", Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
- C. Szegedy, "Going Deeper with Convolutions", Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
- K. He, "Deep residual learning for image recognition", Computer Vision and Pattern Recognition (CVPR), Proc. of the IEEE Conference on, pp.770~778 (2016)
- S. E. F. d. Avila, "Vsumm: A mechanism designed to produce static video summaries and a novel evaluation method", Pattern Recognition Letters, 32(1), pp.56~68 (2011)
- N. N. Hoang, "A Real-time Multimodal Hand Gesture Recognition via 3D Convolutional Neural Network and Key Frame Extraction", Machine Learning in Medical Imaging (MLMI), pp.32~37 (2018)
- V. Badrinarayanan, "SegNet: A Deep Convolutional Encoder-Decoder Architecture Segmentation", Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
- Abhijeet Boragule, "Text Line Segmentation of Handwritten Documents by Area Mapping", 스마트미디어저널, 4(3), pp.44~49 (2015), (사)한국스마트미디어학회
- Son Tung Trieu, "Machine Printed and Handwritten Text Discrimination in Korean Document Images", 스마트미디어저널, 5(3), pp.30~34 (2016), (사)한국스마트미디어학회
- 이태석, "LSTM 기반의 sequence-to-sequence 모델을 이용한 한글자동 띄어쓰기", 스마트미디어저널, 7(4), pp.17~23 (2018), (사)한국스마트미디어학회[DOI]
- Do Nhu Tai, "Tracking by Detection of Multiple Faces using SSD and CNN Features", 스마트미디어저널, 7(4), pp.61~69 (2018), (사)한국스마트미디어학회[DOI]