A study on Movie Scene Understanding for Indexing and Summarization
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Over the past decade, advancements in the entertainment industry have caused exponential growth in video data production especially movies, leading to tiresome indexing, recommendation making, and genre identification. To overcome these challenges, analyzing and understanding the movie contents is the primary step. Therefore, in this paper we proposed an intelligent framework to analyze movies from two different aspects i.e., film grammar and contextual analysis using vision and textual modality. The proposed framework first preprocessed the movie for scene segmentation. Next, the cinematographic transitions are detected such as zoom, background and text related scenes. For contextual analysis, we used customized deep models for object and text detection. Finally, the detailed description of each scene over the movie timestamp is recorded which can be use for various application such as indexing and summarization etc. The proposed framework is evaluated using movie scripts and subjective evaluation.
Abstract
Over the past decade, advancements in the entertainment industry have caused exponential growth in video data production especially movies, leading to tiresome indexing, recommendation making, and genre identification. To overcome these challenges, analyzing and understanding the movie contents is the primary step. Therefore, in this paper we proposed an intelligent framework to analyze movies from two different aspects i.e., film grammar and contextual analysis using vision and textual modality. The proposed framework first preprocessed the movie for scene segmentation. Next, the cinematographic transitions are detected such as zoom, background and text related scenes. For contextual analysis, we used customized deep models for object and text detection. Finally, the detailed description of each scene over the movie timestamp is recorded which can be use for various application such as indexing and summarization etc. The proposed framework is evaluated using movie scripts and subjective evaluation.
참고문헌 (20)
- 윤경목, "유전알고리즘과 신경망 파라미터 값들의 변동에 의한 영화 추천시스템 성능향상", 한국차세대컴퓨팅학회 논문지, 10(5), pp.56~64 (2014), 한국차세대컴퓨팅학회
- 울라 아민, "Action Recognition in Movie Scenes using Deep Features of Keyframes", 한국차세대컴퓨팅학회 논문지, 13(3), pp.7~14 (2017), 한국차세대컴퓨팅학회
- 후세인 탄베르, "Structural similarity based efficient keyframes extraction from multi-view videos", 한국차세대컴퓨팅학회 논문지, 14(6), pp.7~14 (2018), 한국차세대컴퓨팅학회
- Hanjalic, A., "Affective video content representation and modeling", IEEE transactions on multimedia, 7(1), pp.143~154 (2005)
- Pang, L., "Deep multimodal learning for affective analysis and retrieval", IEEE Transactions on Multimedia, 17(11), pp.2008~2020 (2015)
- Vasconcelos, N., "Towards semantically meaningful feature spaces for the characterization of video content", Proceedings of International Conference on Image Processing (1997), IEEE
- Nam, J., "Audio-visual content-based violent scene characterization", Proceedings 1998 International Conference on Image Processing. ICIP98 (1998), IEEE
- Sajjad, M., "Multi-grade brain tumor classification using deep CNN with extensive data augmentation", Journal of computational science, 30, pp.174~182 (2019)
- Adams, B., "Study of shot length and motion as contributing factors to movie tempo", Proceedings of the eighth ACM international conference on Multimedia (2000)
- Haq, I.U., "DeepStar : Detecting starring characters in movies", IEEE Access, 7, pp.9265~9272 (2019)
- Haq, I. U., "Movie scene segmentation using object detection and set theory", International Journal of Distributed Sensor Networks, 15(6), pp.1550147719845277~ (2019)
- Ul Haq, I., "Personalized movie summarization using deep cnn-assisted facial expression recognition", Complexity, 2019 (2019)
- Chen, H.-W., "Action movies segmentation and summarization based on tempo analysis", Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval (2004)
- Haq, I. U., "QuickLook: Movie summarization using scene-based leading characters with psychological cues fusion", Information Fusion, 76, pp.24~35 (2021)
- Castellano, B., "Scene Detection" (2017)
- Redmon, J., "You only look once: Unified, real-time object detection", Proceedings of the IEEE conference on computer vision and pattern recognition (2016)
- Zhou, X., "East: an efficient and accurate scene text detector", Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (2017)
- Lin, T. -Y., "Microsoft coco: Common objects in context", European conference on computer vision (2014), Springer
- Këpuska, V., "Comparing speech recognition systems (Microsoft API, Google API and CMU Sphinx)", Int. J. Eng. Res. Appl, 7(03), pp.20~24 (2017)
- Park, S., "Multiresolution DenseNet based acoustic models for reverberant speech recognition", Phonetics and Speech Sciences, 10(1), pp.33~38 (2018)