본문 바로가기
KERD
논문KCI 등재

Deep Learning Applied Abnormal Human Behavior Detection in Video Surveillance Systems – A Survey

곽내정|이병엽
한국콘텐츠학회 (The Korea Contents Association)|2024.12|International Journal of Contents|제20권 제4호|pp. 84-95|DOI https://doi.org/10.5392/IJoC.2024.20.4.084

2

조회

0

다운로드

0

인용

0

참고문헌

키워드

Deep learningVideo SurveillanceAnomaly DetectionAbnormal Human Behavior DetectionAbnormal Human Activity Recognition
주제분류컴퓨터학

초록

As society develops, the demand for security is rapidly increasing. Accordingly, there is growing interest in research on methods to detect and prevent abnormal behavior using surveillance cameras in public places and private spaces such as shopping malls and airports for human safety. Many detection techniques based on deep learning models have been researched in the field of abnormal behavior detection. However, due to the lack of labeled abnormal behavior data, there are significant difficulties in developing an effective detection system. This paper surveys methods for deep learning methods to detect abnormal human behavior in surveillance video and presents recent techniques. First, I will introduce popular datasets that have often been used in previous research. After that, we categorized the existing methods for detecting abnormal behavior using deep learning into three types: supervised learning, unsupervised learning, and partially-supervised learning. We then explained the basic concepts and advantages of each method and summarized their shortcomings. We also briefly describe future research directions based on the advantages and disadvantages of each method. Based on this, it is expected that the technology of video surveillance systems that apply abnormal behavior detection will further develop.

Abstract

As society develops, the demand for security is rapidly increasing. Accordingly, there is growing interest in research on methods to detect and prevent abnormal behavior using surveillance cameras in public places and private spaces such as shopping malls and airports for human safety. Many detection techniques based on deep learning models have been researched in the field of abnormal behavior detection. However, due to the lack of labeled abnormal behavior data, there are significant difficulties in developing an effective detection system. This paper surveys methods for deep learning methods to detect abnormal human behavior in surveillance video and presents recent techniques. First, I will introduce popular datasets that have often been used in previous research. After that, we categorized the existing methods for detecting abnormal behavior using deep learning into three types: supervised learning, unsupervised learning, and partially-supervised learning. We then explained the basic concepts and advantages of each method and summarized their shortcomings. We also briefly describe future research directions based on the advantages and disadvantages of each method. Based on this, it is expected that the technology of video surveillance systems that apply abnormal behavior detection will further develop.

참고문헌 (74)

  1. V. Chandola, "Anomaly detection : A survey", ACM Comput. Surv, 41, pp.1~58 (2009)[DOI]
  2. R. Ito, "An Adaptive Abnormal Behavior Detection using Online Sequential Learning", Proc. IEEE International Conference on Computational Science and Engineering (CSE)and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), pp.436~440 (2019)
  3. D. Kim, "Real-Time Surveillance System for Analyzing Abnormal Behavior of Pedestrians", Appl. Sci, 11(13) (2021)[DOI]
  4. A. Patwal, "An investigation of videos for abnormal behavior detection", Procedia Comput. Sci, 218, pp.2264~2272 (2023)[DOI]
  5. P. Pandiaraja, "An Analysis of Abnormal Event Detection and Person Identification from Surveillance Cameras using Motion Vectors with Deep Learning", Proc. 2023 Second International Conference on Electronics and Renewable Systems (ICEARS), pp.1225~1232 (2023)
  6. N. Choudhry, "A Comprehensive Survey of Machine Learning Methods for Surveillance Videos Anomaly Detection", IEEE Access, 11, pp.114680~114713 (2023)[DOI]
  7. Y. Wang, "Making Reconstruction-based Method Great Again for Video Anomaly Detection", Proc. IEEE International Conference on Data Mining (ICDM), pp.1215~1220 (2022)
  8. T. Ganokratanaa, "Anomaly Event Detection Using Generative Adversarial Network for Surveillance Videos", Proc. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp.1395~1399 (2019)
  9. C. Zhang, "Weakly Supervised Anomaly Detection in Videos Considering the Openness of Events", IEEE Trans. Intell. Transp. Syst, 23, pp.21687~21699 (2022)[DOI]
  10. J. Ren, "Deep Video Anomaly Detection : Opportunities and Challenges", Proc. International Conference on Data Mining Workshops (ICDMW), pp.959~966 (2021)
  11. V. Mahadevan, "Anomaly detection in crowded scenes", 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.1975~1981 (2010)
  12. W. Luo, "A revisit of sparse coding based anomaly detection in stacked rnn framework", 2017 IEEE International Conference on Computer Vision (ICCV), pp.341~349 (2017)
  13. W. Sultani, "Real-world anomaly detection in surveillance videos", 2018 IEEE, 2018, 6479~6488
  14. R. Mehran, "Abnormal crowd behavior detection using social force model", 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.935~942 (2009)
  15. A. Adam, "Robust real-time unusual event detection using multiple fixedlocation monitors", IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), pp.555~560 (2008)[DOI]
  16. C. Lu, "Abnormal event detection at 150 fps in matlab", 2013 IEEE International Conference on Computer Vision, Sydney, pp.2720~2727 (2013)
  17. C. Hu, "A novel random forests based class incremental learning method for activity recognition", Pattern Recognit, 78, pp.277~290 (2018)[DOI]
  18. Q. Xiao, "Action recognition based on hierarchical dynamic Bayesian network", Multimed. Tools Appl, 77, pp.6955~6968 (2018)[DOI]
  19. P. Sok, "Activity recognition for incomplete spinal cord injury subjects using hidden Markov models", IEEE Sens. J, 18, pp.6369~6374 (2018)[DOI]
  20. B. M. Abidine, "The joint use of sequence features combination and modified weighted SVM for improving daily activity recognition", Pattern Anal. Appl, 21, pp.119~138 (2018)[DOI]
  21. X. Hu, "Video anomaly detection using deep incremental slow feature analysis network", IET Comput. Vis, 10, pp.258~267 (2016)[DOI]
  22. A. Krizhevsky, "Imagenet classification with deep convolutional neural networks", Communications of the ACM, 60, pp.84~90 (2012)[DOI]
  23. O. K. Oyedotun, "Deep learning in vision-based static hand gesture recognition", Neural Comput. And App, 28, pp.3941~3951 (2017)[DOI]
  24. S. Chandrakala, "V2AnomalyVec : Deep Discriminative Embeddings for Detecting Anomalous Activities in Surveillance Videos", IEEE Transactions on Computational Social Systems, 9(5), pp.1307~1316 (2022)[DOI]
  25. M. Q. Gandapur, "E2E-VSDL : End-to-end video surveillance-based deep learning model to detect and prevent criminal activities", Image Vis. Comput, 123 (2022)[DOI]
  26. J. Ahn, "SafeFac : Video-based smart safety monitoring for preventing industrial work accidents", Expert. Syst. Appl, 215 (2023)[DOI]
  27. E. M. Onyema, "Remote monitoring system using slow-fast deep convolution neural network model for identifying anti-social activities in surveillance applications", Meas. Sens, 27 (2023)[DOI]
  28. H. Ullah, "Human Activity Recognition Using Cascaded Dual Attention CNN and Bi-Directional GRU Framework", J. Imaging, 9 (2023)[DOI]
  29. H. B. Rolf, "Human behaviour recognition in data-scarce domains", Pattern Recognit, 48, pp.2377~2393 (2015)[DOI]
  30. H. Tu, "Information integration via hierarchical and hybrid bayesian networks", IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 36(1), pp.19~33 (2006)[DOI]
  31. H. T. Duong, "Deep Learning-Based Anomaly Detection in Video Surveillance : A Survey", Sensors, 11 (2023)[DOI]
  32. Y. Myagmar-Ochir, "A Survey of Video Surveillance Systems in Smart City", Electronics, 12[DOI]
  33. E. Şengönül, "An Analysis of Artificial Intelligence Techniques in Surveillance Video Anomaly Detection: A Comprehensive Survey", Appl. Sci, 13(8) (2023)[DOI]
  34. R. Nayak, "A comprehensive review on deep learning-based methods for video anomaly detection", Image Vis. Comput, 106 (2021)[DOI]
  35. D. Chen, "Anomaly detection in surveillance video based on bidirectional prediction", Image Vis. Comput, 98 (2020)[DOI]
  36. J. Yu, "Abnormal Event Detection and Localization via Adversarial Event Prediction", IEEE Transactions on Neural Networks and Learning Systems, 33(8), pp.3572~3586 (2022)[DOI]
  37. E. Sengönül, "An Analysis of Artificial Intelligence Techniques in Surveillance Video Anomaly Detection : A Comprehensive Survey", Appl. Sci, 13 (2023)[DOI]
  38. W. Pedrycz, "Fuzzy clustering with partial supervision", IEEE Trans. Syst. Man Cybern. Part B(Cybern. ), 27(5), pp.787~795 (1997)[DOI]
  39. P. Khaire, "A semi-supervised deep learning based video anomaly detection framework using RGB-D for surveillance of real-world critical environments", Forensic Sci. Int. Digit. Investig, 40 (2022)[DOI]
  40. A. Pramanik, "Video surveillance-based fall detection system using object-level feature thresholding", Knowl. Based Syst, 280 (2023)[DOI]
  41. Y. Liu, "Collaborative Normality Learning Framework for Weakly Supervised Video Anomaly Detection", IEEE Trans. Circuits Syst. II Express Briefs, 69, pp.2508~2512 (2022)[DOI]
  42. A. M. Kamoona, "Hoseinnezhad, R. Multiple instance-based video anomaly detection using deep temporal encoding–decoding", Expert. Syst. Appl, 214 (2023)[DOI]
  43. C. Dhiman, "A review of state-of-the-art techniques for abnormal human activity recognition", Eng. Appl. Artif. Intell, 77, pp.21~45 (2019)[DOI]
  44. B. Ramachandra, "A Survey of Single-Scene Video Anomaly Detection", IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5), pp.2293~2312 (2022)[DOI]
  45. D. R. Patrikar, "Anomaly detection using edge computing in video surveillance system : Review", Int. J. Multimed. Inf. Retr, 11, pp.85~110 (2022)[DOI]
  46. J. Mao, "A health monitoring system based on flexible triboelectric sensors for intelligence medical internet of things and its applications in virtual reality", Nano Energy, 118 (2023)[DOI]
  47. A. Hussain, "Low-light aware framework for human activity recognition via optimized dual stream parallel network", Alex. Eng. J, 74, pp.569~583 (2023)[DOI]
  48. W. Ullah, "An Efficient Anomaly Recognition Framework Using an Attention Residual Lstm in Surveillance Videos", Sensors, 21(8) (2021)[DOI]
  49. R. Espinosa, "A vision-based approach for fall detection using multiple cameras and convolutional neural networks : A case study using the UP-Fall detection dataset", Comput. Biol. Med, 115 (2019)[DOI]
  50. M. E. N. Gomes, "Multi-human Fall Detection and Localization in Videos", Comput. Vis. Image Underst, 220 (2022)[DOI]
  51. A. P. Kshirsagar, "YOLOv3-based human detection and heuristically modified-LSTM for abnormal human activities detection in ATM machine", J. Vis. Commun. Image Represent, 95 (2023)[DOI]
  52. I. Usman, "Abnormal Crowd Behavior Detection Using Heuristic Search and Motion Awareness", Int. J. Comput. Sci. Netw. Secur, 21, pp.131~139 (2021)[DOI]
  53. L. P. Cinelli, "Variational Methods for Machine Learning with Applications to Deep Networks" (2021), Springer International Publishing
  54. S. A. Jebur, "Review on Deep Learning Approaches for Anomaly Event Detection in Video Surveillance", Electronics, 12(1) (2022)[DOI]
  55. L. Alzubaidi, "Review of Deep Learning : Concepts, CNN Architectures, Challenges, Applications, Future Directions", J. Big Data, 8(1) (2021)[DOI]
  56. Q. Liu, "A fully connected network based on memory for video anomaly detection", 2022 IEEE 8th International Conference on Cloud Computing and Intelligent Systems (CCIS), 221(226), pp.221~226 (2022)
  57. M. Yan, "Memory clustering auto-encoder method for human action anomaly detection on surveillance camera video", IEEE Sensors Journal, 23(18), pp.20715~20728 (2023)[DOI]
  58. G. Slavic, "Multilevel anomaly detection through variational auto-encoders and bayesian models for self-aware embodied agents", IEEE Transactions on Multimedia, 24, pp.1399~1414 (2022)[DOI]
  59. L. Wang, "Unsupervised anomaly video detection via a double-flow convlstm variational auto-encoder", IEEE Access, 10, pp.44278~44289 (2022)[DOI]
  60. N. Taghinezhad, "A New Unsupervised Video Anomaly Detection Using Multi-Scale Feature Memorization and Multipath Temporal Information Prediction", IEEE Access, 11, pp.9295~9310 (2023)[DOI]
  61. H. Huang, "A Novel Unsupervised Video Anomaly Detection Framework Based on Optical Flow Reconstruction and Erased Frame Prediction", Sensors, 23 (2023)[DOI]
  62. G. Li, "Adversarial composite prediction of normal video dynamics for anomaly detection", Comput. Vis. Image Underst, 232 (2023)[DOI]
  63. M. Jovanović, "Generative Artificial Intelligence: Trends and Prospects", Computer, 55(10), pp.107~112 (2022)[DOI]
  64. I. Goodfellow, "Generative adversarial nets", Advances in Neural Information Processing Systems, 27 (2014), Curran Associates, Inc[DOI]
  65. A. Oliver, "Realistic evaluation of deep semi-supervised learning algorithms", Proc. of the 32nd International Conference on Neural Information Processing Systems, pp.3239~3250 (2018), Montreal, Curran Associates Inc.: Red Hook
  66. A. Sikdar, "An adaptive training-less framework for anomaly detection in crowd scenes", Neurocomputing, 415, pp.317~331 (2020)[DOI]
  67. R. Wu, "A. Improving video anomaly detection performance by mining useful data from unseen video frames", Neurocomputing, 462, pp.523~533 (2021)[DOI]
  68. L. Wang, "Attention-guided MIL weakly supervised visual anomaly detection", Measurement, 209 (2023)[DOI]
  69. B. Zhang, "Weakly-supervised anomaly detection with a Sub-Max strategy", Neurocomputing, 560 (2023)[DOI]
  70. J. Shin, "Anomaly Detection in Weakly Supervised Videos Using Multistage Graphs and General Deep Learning Based Spatial-Temporal Feature Enhancement", IEEE Access, 12, pp.65213~65227 (2024)[DOI]
  71. N. Simmler, "A Survey of Un-, Weakly-, and Semi-Supervised Learning Methods for Noisy, Missing and Partial Labels in Industrial Vision Applications", 2021 8th Swiss Conference on Data Science (SDS), pp.26~31 (2021)
  72. S. Park, "Normality Guided Multiple Instance Learning for Weakly Supervised Video Anomaly Detection", Proc. the 2023 IEEE, 2023, 2664~2673
  73. Z. Xu, "Improved Anomaly Detection in Surveillance Videos with Multiple Probabilistic Models Inference", Intell. Autom. Soft Comput, 31(3), pp.1703~1717 (2022)[DOI]
  74. W. Ullah, "Intelligent dual stream CNN and echo state network for anomaly detection", Knowl. Based Syst, 253 (2022)[DOI]

관련 논문AI추천

Abnormal Behavior Recognition Based on Spatio-temporal ContextGang Liu, Lin Li, Yuanfeng Yang, Zhaobin Liu한국정보처리학회 · JIPS(Journal of Information Processing Systems)2020 · 제16권 제3호Crowd Behavior Detection using Convolutional Neural Network튜 유 민 울라, 와셈 울라, 백성욱, 이미영한국차세대컴퓨팅학회 · 한국차세대컴퓨팅학회 논문지2019 · 제15권 제6호Violent Activity Detection Using Deep Convolutional Neural Networks in Video Surveillance파트 유 민 울라, 백성욱, 이미영, 이자즈 울 하크, 알타프 후세인한국차세대컴퓨팅학회 · 한국차세대컴퓨팅학회 논문지2021 · 제17권 제4호Secure Object Detection Based on Deep Learning김건형, 정임영한국정보처리학회 · JIPS(Journal of Information Processing Systems)2021 · 제17권 제3호딥 러닝 기반의 CCTV를 활용한 이상 행동 감지 시스템 설계이용주한국지식정보기술학회 · 한국지식정보기술학회 논문지2022 · 제17권 제2호감시 영상에서 비정상적인 인체 행동 탐지를 위한 시공간 트랜스포머 네트워크김성익, 조완섭, Saravit Soeng사)한국빅데이터학회 · 한국빅데이터학회 학회지2025 · 제10권 제2호

이 간행물의 다른 논문 (7건)

Navigating #StopAsianHate Upon Atlanta Shooting Incident: Focused on Emotion and Sentiment Analysis on Twitter (X)장안리, 이승은pp. 1-13Innovative 3D Character Model Texture Mapping Solution Based on Artificial Intelligence Image Generation ModelJiawei Yin, 송밝음pp. 14-21Promoting Tax Compliance in the Democratic Republic of Congo: Lessons Learned from the Republic of Korea as Case Study조관필, Augustin Pambi Tadiamba, Jacob Ngwaba, Junior Ntiama, Justin MBIMBI, 유송희, 김윤선, 유기동, Joseph BIMBALA GWABApp. 22-37A Basic Study on the New Creative Thinking Process: Modified from the KJ Method이성필, Jiahui Gupp. 38-47A Case Study on Success Factors of a Technology Convergence Arts Startup from an ERIS Perspective: Focusing on Yeolmae Company Inc.이상연, 김기창, 김선영pp. 48-67
원문 보기