A novel 3D-Convolution Neural Network for Human Interaction Recognition in videos
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Human Interaction Recognition (HIR) has already been perceived rapid progress same as human action and activity recognition. In HIR, we intend to highlight the problem of human-to-human interaction recognition in videos by exploring the long term inter-related dynamics between multiple humans. In order to understand the human-to-human interaction precisely, HIR system requires a robust feature extraction and selection method based on videos. In this paper, we propose a novel 3D convolutional neural network (3D CNN) followed by a fully connected block, to wisely trace human to human interactions in videos. We feed our proposed model with 15 sequence of video frames to our novel 3D CNN architecture which extracts deep features from all the sequences and then pass those sequences to the fully connected block to boost our efficiency. Our proposed network outperformed the existing state-of-the-art methods by accomplishing extraordinary recognition accuracy on two benchmark datasets, UT-I and TV Human Interaction dataset i.e., 84% and 74% overall and improved from the state-of-the-art techniques. Our proposed network can also be applicable to other numerous multimedia contents and security applications such as video-based learning, service combats, medical futurists, interactive gaming, and surveillance systems.
Abstract
Human Interaction Recognition (HIR) has already been perceived rapid progress same as human action and activity recognition. In HIR, we intend to highlight the problem of human-to-human interaction recognition in videos by exploring the long term inter-related dynamics between multiple humans. In order to understand the human-to-human interaction precisely, HIR system requires a robust feature extraction and selection method based on videos. In this paper, we propose a novel 3D convolutional neural network (3D CNN) followed by a fully connected block, to wisely trace human to human interactions in videos. We feed our proposed model with 15 sequence of video frames to our novel 3D CNN architecture which extracts deep features from all the sequences and then pass those sequences to the fully connected block to boost our efficiency. Our proposed network outperformed the existing state-of-the-art methods by accomplishing extraordinary recognition accuracy on two benchmark datasets, UT-I and TV Human Interaction dataset i.e., 84% and 74% overall and improved from the state-of-the-art techniques. Our proposed network can also be applicable to other numerous multimedia contents and security applications such as video-based learning, service combats, medical futurists, interactive gaming, and surveillance systems.
참고문헌 (25)
- K. Muhammad, "Human action recognition using attention based LSTM network with dilated CNN features", Future Generation Computer Systems, 125, pp.820~830 (2021)
- W. Ullah, "An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos", 21(8), pp.2811~ (2021)
- W. Ullah, "CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks", 80(11), pp.6979~16995 (2021)
- N. Dalal, "Histograms of oriented gradients for human detection", 2005 IEEE computer society conference on computer vision and pattern recognition(CVPR'05), 1, pp.886~893 (2005), Ieee
- I. Laptev, "Learning realistic human actions from movies", 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp.1~8 (2008), IEEE
- N. Dalal, "Human detection using oriented histograms of flow and appearance", European conference on computer vision, pp.428~441 (2006), Springer
- D. G. J. I. j. o. c. v. Lowe, "Distinctive image features from scale-invariant keypoints", 60(2), pp.91~110 (2004)
- C. Xu, "Multi-view intact space learning", 37(12), pp.2531~2544 (2015)
- Y. Zhao, "Multi-view manifold learning with locality alignment", 78, pp.154~166 (2018)
- M. Hoai, "Talking heads:Detecting humans and recognizing their interactions", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.875~882 (2014)
- T. Lan, "A hierarchical representation for future action prediction,", European Conference on Computer Vision, pp.689~704 (2014), Springer
- A. Vahdat, "A discriminative key pose sequence model for recognizing human interactions", 2011 IEEE International Conference on Computer Vision Workshops(ICCV Workshops), pp.1729~1736 (2011), IEEE
- X. Ji, "A new framework of human interaction recognition based on multiple stage probability fusion", 7(6), pp.567~ (2017)
- Y. Kong, "A discriminative model with multiple temporal scales for action prediction", European conference on computer vision, pp.596~611 (2014), Springer
- Y. Kong, "Max-margin action prediction machine", 38(9), pp.1844~1858 (2015)
- N. P. Trong, "A comprehensive survey on human activity prediction", International Conference on Computational Science and Its Applications, pp.411~425 (2017), Springer
- Q. Ke, "Human interaction prediction using deep temporal features", European Conference on Computer Vision, pp.403~414 (2016), Springer
- Q. Ke, "Leveraging structural context models and ranking score fusion for human interaction prediction", 20(7), pp.1712~1723 (2017)
- J. Donahue, "Long-term recurrent convolutional networks for visual recognition and description", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2625~2634 (2015)
- M. Afrasiabi, "DTW-CNN : time series-based human interaction prediction in videos using CNNextracted features", 1(13), pp.1~13 (2019)
- A. Ullah, "Event-oriented 3D convolutional features selection and hash codes generation using PCA for video retrieval", 8, pp.196529~196540 (2020)
- H. Yar, "Lung Nodule Detection and Classification using 2D and 3D Convolution Neural Networks(CNNs)", pp.365~386
- Q. Ke, "Leveraging structural context models and ranking score fusion for human interaction prediction", IEEE Transactions on Multimedia, 20(7), pp.1712~1723 (2017)
- M. Afrasiabi, "DTWCNN: time series-based human interaction prediction in videos using CNN-extracted features", The Visual Computer, pp.1~13 (2019)
- M. Afrasiabi, "Spatial-temporal dual-actor CNN for human interaction prediction in video", Multimedia Tools and Applications, 1(20), pp.1~20 (2020)