다중 객체검출 기반의 온라인 비디오 요약 알고리즘
Online Video Synopsis via Multiple Object Detection
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
In this paper, an online video summarization algorithm based on multiple object detection is proposed. As crime has been on the rise due to the recent rapid urbanization, the people’s appetite for safety has been growing and the installation of surveillance cameras such as a closed-circuit television(CCTV) has been increasing in many cities. However, it takes a lot of time and labor to retrieve and analyze a huge amount of video data from numerous CCTVs. As a result, there is an increasing demand for intelligent video recognition systems that can automatically detect and summarize various events occurring on CCTVs. Video summarization is a method of generating synopsis video of a long time original video so that users can watch it in a short time. The proposed video summarization method can be divided into two stages. The object extraction step detects a specific object in the video and extracts a specific object desired by the user. The video summary step creates a final synopsis video based on the objects extracted in the previous object extraction step. While the existed methods do not consider the interaction between objects from the original video when generating the synopsis video, in the proposed method, new object clustering algorithm can effectively maintain interaction between objects in original video in synopsis video. This paper also proposed an online optimization method that can efficiently summarize the large number of objects appearing in long-time videos. Finally, Experimental results show that the performance of the proposed method is superior to that of the existing video synopsis algorithm.
Abstract
In this paper, an online video summarization algorithm based on multiple object detection is proposed. As crime has been on the rise due to the recent rapid urbanization, the people’s appetite for safety has been growing and the installation of surveillance cameras such as a closed-circuit television(CCTV) has been increasing in many cities. However, it takes a lot of time and labor to retrieve and analyze a huge amount of video data from numerous CCTVs. As a result, there is an increasing demand for intelligent video recognition systems that can automatically detect and summarize various events occurring on CCTVs. Video summarization is a method of generating synopsis video of a long time original video so that users can watch it in a short time. The proposed video summarization method can be divided into two stages. The object extraction step detects a specific object in the video and extracts a specific object desired by the user. The video summary step creates a final synopsis video based on the objects extracted in the previous object extraction step. While the existed methods do not consider the interaction between objects from the original video when generating the synopsis video, in the proposed method, new object clustering algorithm can effectively maintain interaction between objects in original video in synopsis video. This paper also proposed an online optimization method that can efficiently summarize the large number of objects appearing in long-time videos. Finally, Experimental results show that the performance of the proposed method is superior to that of the existing video synopsis algorithm.
참고문헌 (38)
- H. W. Kang, "Space-time video montage", 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, pp.1331~1338 (2006)
- M. Xu, "A set theoretical method for video synopsis", Proceedings of the 1st ACM international conference on Multimedia information retrieval, pp.366~370 (2008)
- Y. Nie, "Compact video synopsis via global spatiotemporal optimization", IEEE transactions on visualization and computer graphics, 19(10), pp.1664~1676 (2013)
- C. R. Huang, "Online surveillance video synopsis", 2012 IEEE International Symposium on Circuits and Systems, pp.1843~1846 (2012)
- E. Bennett, "Computational time-lapse video", ACM Transactions on Graphics, 26(3), pp.102~ (2007)
- N. Petrovic, "Adaptive video fast forward", Multimedia Tools and Applications, 26(3), pp.327~344 (2005)
- J. Nam, "Video abstract of video", 1999IEEE Third Workshop on Multimedia Signal Processing(Cat. No. 99TH8451), pp.117~122 (1999)
- X. Zhu, "Exploring video content structure for hierarchical summarization", Multimedia Systems, 10(2), pp.98~115 (2004)
- T. Liu, "Shot reconstruction degree : a novel criterion for key frame selection", Pattern recognition letters, 25(12), pp.1451~1457 (2004)
- B. T. Truong, "Video abstraction:A systematic review and classification", ACM transactions on multimedia computing, communications, and applications, 3(1) (2007)
- C. Gianluigi, "An innovative algorithm for key frame extraction in video summarization", Journal of Real-Time Image Processing, 1(1), pp.69~88 (2006)
- H. Liu, "Key frame extraction of online video based on optimized frame difference", 20129th International Conference on Fuzzy Systems and Knowledge Discovery, pp.1238~1242 (2012)
- C. M. Taskiran, "Automated video program summarization using speech transcripts", IEEE Transactions on Multimedia, 8(4), pp.775~791 (2006)
- Y. F. Ma, "A generic framework of user attention model and its application in video summarization", IEEE Transaction on multimedia, 7(5), pp.907~919 (2005)
- X. Zhu, "Video synopsis by heterogeneous multi-source correlation", Proceedings of the IEEE International Conference on Computer Vision, pp.81~88 (2013)
- S. Benini, "Hidden Markov models for video skim generation", Eighth International Workshop on Image Analysis for Multimedia Interactive Services, pp.6~6 (2007)
- S. Benini, "A statistical framework for video skimming based on logical story units and motion activity", 2007 International Workshop on Content-Based Multimedia Indexing, pp.152~156 (2007)
- Y. Pritch, "Nonchronological video synopsis and indexing", IEEE Trans. on pattern analysis and machine intelligence, 30(11), pp.1971~1984 (2008)
- Y. Pritch, "Webcam synopsis: Peeking around the world", 2007 IEEE 11th International Conference on Computer Vision, pp.1~8 (2007)
- A. Rav-Acha, "Making a long video short: Dynamic video synopsis", 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, pp.435~441 (2006)
- C. R. Huang, "Maximuma PosterioriProbability Estimation for Online Surveillance Video Synopsis", IEEE Transactions on circuits and systems for video technology, 24(8), pp.1417~1429 (2014)
- S. Feng, "Online content-aware video condensation", 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp.2082~2087 (2012)
- J. Zhu, "High-performance video condensation system", IEEE Transactions on Circuits and Systems for Video Technology, 25(7), pp.1113~1124 (2014)
- L. Sun, "A tracking based fast online complete video synopsis approach", Proceedings of the 21st international conference on pattern recognition, pp.1956~1959 (2012)
- M. Lu, "Generating fluent tubes in video synopsis", 2013 IEEE international conference on acoustics, speech and signal processing, pp.2292~2296 (2013)
- W. Fu, "Online video synopsis of structured motion", Neurocomputing, 135(5), pp.155~162 (2014)
- R. Zhong, "Fast synopsis for moving objects using compressed video", IEEE signal processing letters, 21(7), pp.834~838 (2014)
- W. Lin, "Summarizing surveillance videos with local-patch-learning-based abnormality detection, blob sequence optimization, and type-based synopsis", Neurocomputing, 155(1), pp.84~98 (2015)
- J. Redmon, "YOLO9000: better, faster, stronger", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.7263~7271 (2017)
- T. Y. Lin, "Focal loss for dense object detection", Proceedings of the IEEE international conference on computer vision, pp.2980~2988 (2017)
- K. He, "Mask r-cnn", Proceedings of the IEEE international conference on computer vision, pp.2961~2969 (2017)
- S. Zhang, "Single-shot refinement neural network for object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.4203~4212 (2018)
- Q. ZHAO, "M2det : A single-shot object detector based on multi-level feature pyramid network", Proceedings of the AAAI Conference on Artificial Intelligence, 33(1), pp.9259~9266 (2019)
- J. Jin, "Online video synopsis method through simple tube projection strategy", 2016 8th International Conference on Wireless Communications & Signal Processing, pp.1~5 (2016)
- P. Pérez, "Poisson image editing", ACM Transactions on graphics, 22(3), pp.313~318 (2003)
- A. Agarwala, "Efficient gradient-domain compositing using quadtrees", ACM Transactions on Graphics, 26(3) (2007)
- Z. Fangneng, "Adaptive Composition GAN towards Realistic Image Synthesis", arXiv preprint, arXiv:1905.04693, ~ (2019)
- H. Wu, "Gp-gan:Towards realistic high-resolution image blending", arXiv preprint, arXiv:1703.07195, ~ (2019)