Egocentric Vision for Human Activity Recognition Using Deep Learning
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The topic of this paper is the recognition of human activities using egocentric vision, particularly captured bybody-worn cameras, which could be helpful for video surveillance, automatic search and video indexing. Thisbeing the case, it could also be helpful in assistance to elderly and frail persons for revolutionizing andimproving their lives. The process throws up the task of human activities recognition remaining problematic,because of the important variations, where it is realized through the use of an external device, similar to a robot,as a personal assistant. The inferred information is used both online to assist the person, and offline to supportthe personal assistant. With our proposed method being robust against the various factors of variability problemin action executions, the major purpose of this paper is to perform an efficient and simple recognition methodfrom egocentric camera data only using convolutional neural network and deep learning. In terms of accuracyimprovement, simulation results outperform the current state of the art by a significant margin of 61% whenusing egocentric camera data only, more than 44% when using egocentric camera and several stationarycameras data and more than 12% when using both inertial measurement unit (IMU) and egocentric camera data.
Abstract
The topic of this paper is the recognition of human activities using egocentric vision, particularly captured bybody-worn cameras, which could be helpful for video surveillance, automatic search and video indexing. Thisbeing the case, it could also be helpful in assistance to elderly and frail persons for revolutionizing andimproving their lives. The process throws up the task of human activities recognition remaining problematic,because of the important variations, where it is realized through the use of an external device, similar to a robot,as a personal assistant. The inferred information is used both online to assist the person, and offline to supportthe personal assistant. With our proposed method being robust against the various factors of variability problemin action executions, the major purpose of this paper is to perform an efficient and simple recognition methodfrom egocentric camera data only using convolutional neural network and deep learning. In terms of accuracyimprovement, simulation results outperform the current state of the art by a significant margin of 61% whenusing egocentric camera data only, more than 44% when using egocentric camera and several stationarycameras data and more than 12% when using both inertial measurement unit (IMU) and egocentric camera data.
참고문헌 (36)
- Charmi Jobanputra, "Human Activity Recognition: A Survey", Procedia Computer Science, 155, pp.698~703 (2019), Elsevier BV[DOI]
- Taha Alhersh, "Learning Human Activity From Visual Data Using Deep Learning", IEEE Access, 9, pp.106245~106253 (2021), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- "CMU-Multimodal Activity (CMU-MMAC) database" (2010)[URL]
- E. H. Spriggs, "Temporal segmentation and activity classification from firstperson sensing", Proceedings of 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp.17~24 (2009)
- A. Fathi, "Understanding egocentric activities", Proceedings of 2011 International Conference on Computer Vision, pp.407~414 (2011)
- A. Fathi, "Computer Vision – ECCV 2012", pp.314~327 (2012), Springer
- H. Pirsiavash, "Detecting activities of daily living in first-person camera views", Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp.2847~2854 (2012)
- M. S. Ryoo, "First-person activity recognition: What are they doing to me?", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2730~2737 (2013)
- S. Song, "Computer Vision – ACCV 2014 Workshops", pp.445~458 (2014), Springer
- B. Soran, "Computer Vision - ACCV 2014", pp.178~193 (2015), Springer
- M. S. Ryoo, "Pooled motion features for first-person videos", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.896~904 (2015)
- Y. Li, "Delving into egocentric actions", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.287~295 (2015)
- M. Ma, "Going deeper into first-person activity recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1894~1903 (2016)
- S. Song, "Egocentric activity recognition with multimodal fisher vector", Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2717~2721 (2016)
- S. Song, "Multimodal multi-stream deep learning for egocentric activity recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.24~31 (2016)
- S. Singh, "First person action recognition using deep learned descriptors", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2620~2628 (2016)
- L. Wang, "Computer Vision – ECCV 2016", pp.20~36 (2016), Springer
- E. A. Khalid, "A survey of activity recognition in egocentric lifelogging datasets", Proceedings of 2017 International Conference on Wireless Technologies, Embedded and Intelligent Systems (WITS), pp.1~8 (2017)
- S. Singh, "Trajectory aligned features for first person action recognition", Pattern Recognition, 62, pp.45~55 (2017)[DOI]
- Y. Liu, "Jointly recognizing object fluents and tasks in egocentric videos", Proceedings of the IEEE International Conference on Computer Vision, pp.2924~2932 (2017)
- R. Possas, "Egocentric activity recognition on a budget", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.5967~5976 (2018)
- Y. Li, "Computer Vision – ECCV 2018", pp.619~635 (2018), Springer
- "Attention is all we need: nailing down object-centric attention for egocentric activity recognition" (2018)[URL]
- S. Sudhakaran, "LSTA: long short-term attention for egocentric action recognition", Proceedings of the IEEE, 2019, 9954~9963
- E. Kazakos, "EPIC-fusion: audio-visual temporal binding for egocentric action recognition", Proceedings of the IEEE, 2019, 5492~5501
- "Autonomous human activity classification from ego-vision camera and accelerometer data" (2019)[URL]
- A. Diete, "Fusing object information and inertial data for activity recognition", Sensors, 19(19) (2019)
- Antonino Furnari, "Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video", IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11), pp.4021~4036 (2021), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- I. Rodin, "Scene understanding and interaction anticipation from first person vision", Proceedings of the 1st Workshop on Smart Personal Health Interfaces co-located with 25th International Conference on Intelligent User Interfaces, pp.78~83 (2020)
- K. Min, "Integrating human gaze into attention for egocentric activity recognition", Proceedings of the IEEE, 2021, 1069~1078
- F. Ragusa, "The MECCANO dataset : understanding humanobject interactions from egocentric videos in an industrial-like domain", Proceedings of the IEEE, 2021, 1568~1577
- A. Ghosh, "Recent Trends and Advances in Artificial Intelligence and Internet of Things", pp.519~567 (2020), Springer
- "A comprehensive guide to convolutional neural networks: the ELI5 way" (2018)[URL]
- "Free Video to JPG converter" (2022)[URL]
- "TrainingOptionsSGDM: training options for stochastic gradient descent with momentum" (2023)[URL]
- "On the hyperparameters in stochastic gradient descent with momentum" (2021)[URL]