심층 CNN을 활용한 영상 분위기 분류 및 이를 활용한 동영상 자동 생성
Image Mood Classification Using Deep CNN and Its Application to Automatic Video Generation
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
본 연구에서는 영상의 분위기를 심층 합성곱 신경망을 통해 8 가지로 분류하고, 이에 맞는 배경 음악을 적용하여 동영상을 자동적으로 생성하였다. 수집된 이미지 데이터를 바탕으로 다층퍼셉트론을 사용하여 분류 모델을 학습한다. 이를 활용하여 다중 클래스 분류를 통해 동영상 생성에 사용할 이미지의 분위기를 예측하며, 미리 분류된 음악을 매칭시켜 동영상을 생성한다. 10겹 교차 검증의 결과, 72.4%의 정확도를 얻을 수 있었고, 실제 영상에 대한 실험에서 64%의 오차 행렬 정확도를 얻을 수 있었다. 오답의 경우, 주변의 비슷한 분위기로 분류하여 동영상에서 나오는 음악과 크게 위화감이 없음을 확인하였다.
Abstract
In this paper, the mood of images was classified into eight categories through a deep convolutional neural network and video was automatically generated using proper background music. Based on the collected image data, the classification model is learned using a multilayer perceptron (MLP). Using the MLP, a video is generated by using multi-class classification to predict image mood to be used for video generation, and by matching pre-classified music. As a result of 10-fold cross-validation and result of experiments on actual images, each 72.4% of accuracy and 64% of confusion matrix accuracy was achieved. In the case of misclassification, by classifying video into a similar mood, it was confirmed that the music from the video had no great mismatch with images.
참고문헌 (20)
- Y. Yan, "Deep learning for imbalanced multimedia data classification", 2015 IEEE International Symposium on Multimedia, pp.483~488 (2015)
- M. K. Lee, "Emotion recognition system based deep learning", Journal of the Korean Society Of Broad Engineers, pp.16~18 (2017)
- S. H. Kim, "Sentiment classification for videos using deep learning algorithms", Master, pp.2016
- James A. Russell, "A circumplex model of affect.", Journal of Personality and Social Psychology, 39(6), pp.1161~1178 (1980), American Psychological Association (APA)[DOI]
- D. H. Ko, "Face Verification based on Deep Convolutional Nerual Network", Journal of The Korean Institute of Information Scientists and Engineers (2017)
- 이동규, "합성곱 신경망을 사용한 화물차의 차종분류", 융합정보논문지, 8(6), pp.375~380 (2018), 중소기업융합학회[DOI]
- 마렌드라, "Predicting Employment Earning using Deep Convolutional Neural Networks", 디지털융복합연구, 16(6), pp.151~161 (2018), 한국디지털정책학회[DOI]
- J. Y. Lee, "Music crawler for mood-based music classification and retrieval systems", Journal of Korea Information Science Society, pp.699~701 (2018)
- C. W. Lee, "Development of automatic synchronization tool for scene and background music" (2005), INET
- Christopher Olston, "Web Crawling", Foundations and Trends® in Information Retrieval, 4(3), pp.175~246 (2010), Now Publishers[DOI]
- Guang-Bin Huang, "Extreme Learning Machine for Regression and Multiclass Classification", IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(2), pp.513~529 (2012), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Martin Riedmiller, "Advanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithms", Computer Standards & Interfaces, 16(3), pp.265~278 (1994), Elsevier BV[DOI]
- Alex Krizhevsky, "ImageNet classification with deep convolutional neural networks", Communications of the ACM, 60(6), pp.84~90 (2017), Association for Computing Machinery (ACM)[DOI]
- Sepp Hochreiter, "The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions", International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 06(02), pp.107~116 (2011), World Scientific Pub Co Pte Lt[DOI]
- R. A. Dunne, "On the pairing of the softmax activation and cross-entropy penalty functions and the derivation of the softmax activation function", Proc. 8th Aust. Conf. on the Neural Networks (1997)
- N. Srivastava, "Dropout: a simple way to prevent neural networks from overfitting", The journal of machine learning research, 15(1), pp.1929~1958 (2014)[DOI]
- A. Krogh, "Advances in neural information processing systems", pp.231~238 (1995), MITPress
- Marina Sokolova, "A systematic analysis of performance measures for classification tasks", Information Processing & Management, 45(4), pp.427~437 (2009), Elsevier BV[DOI]
- T. Kincl, "Getting inside the minds of the customers: automated sentiment analysis", ECMLG2013-Proceedings For the 9th European Conference on Management Leadership and Governance: ECMLG 2013, pp.122~128 (2013), Academic Conferences Limited
- V. Gajarla, "Emotion detection and sentiment analysis of images" (2015), Georgia Institute of Technology