이미지의 인지적 특징 정량화를 통한 CNN-ViT 하이브리드 미학 평가 모델
CNN-ViT Hybrid Aesthetic Evaluation Model Based on Quantification of Cognitive Features in Images
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 논문에서는 이미지의 지역적 및 전역적 특징을 결합하여 이미지의 미학적 품질을 자동으로 평가할 수 있는 CNN-ViT 하이브리드 모델을 제안한다. 이 접근 방식에서는 CNN을 사용하여 색상 및 객체 배치와 같은 지역적 특징을 추출하고, ViT를 통해 전역적 특징을 반영하여 이미지의 미학적 가치를 분석한다. Color composition은 입력 이미지에서 주요 색상을 추출해 생성한 컬러팔레트를 CNN에 통과시켜 얻은 값이며, Rule of Third는 이미지 속 오브젝트가 삼등분할점에 얼마나 근접한지를 정량적으로 평가한 점수로 사용된다. 이러한 값들은 모델에 이미지의 주요 평가 요소인 색채와 공간 균형에 대한 정보를 제공한다. 모델은 이를 바탕으로 이미지의 점수와 색상, 공간의 균형 간에 연관성을 분석하며, 인간의 평가 분포와 유사한 점수를 추측하도록 설계되었다. 실험 결과, AADB 이미지 데이터베이스에서 스피어만순위상관계수(SRCC)에서는 0.716을 기록하여 순위 예측에서 더 일관된 결과를제공 했으며, 피어슨상관계수(LCC)에서도 0.72을 기록하여 기존 연구 모델보다 2~4% 정도 향상된 결과를 보였다.
Abstract
This paper proposes a CNN-ViT hybrid model that automatically evaluates the aesthetic quality of images bycombining local and global features. In this approach, CNN is used to extract local features such as color andobject placement, while ViT is employed to analyze the aesthetic value of the image by reflecting global features. Color composition is derived by extracting the primary colors from the input image, creating a color palette, andthen passing it through the CNN. The Rule of Thirds is quantified by calculating how closely objects in the imageare positioned near the thirds intersection points. These values provide the model with critical information aboutthe color balance and spatial harmony of the image. The model then analyzes the relationship between thesefactors to predict scores that align closely with human judgment. Experimental results on the AADB image databaseshow that the proposed model achieved a Spearman's Rank Correlation Coefficient (SRCC) of 0.716, indicatingmore consistent rank predictions, and a Pearson Correlation Coefficient (LCC) of 0.72, which is 2~4% higher thanexisting models.
참고문헌 (34)
- Lo, K. Y., "Assessment of photo aesthetics with efficiency", Proc. of the 21st International Conference on Pattern Recognition (ICPR2012), pp.2186~2189 (2012), IEEE
- Stepanova, E., "The impact of color palettes on the prices of paintings", Empirical Economics, 56(2), pp.755~773 (2019)
- Karp, A, "The elements of color", Leonardo, 5(2), pp.180~ (1972)
- O’Donovan, P., "Color compatibility from large datasets", ACM Transactions on Graphics, 30(4), pp.1~12 (2011)[DOI]
- Tan, J. C., "Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry", ACM Transactions on Graphics, 37(6), pp.1~10 (2018)[DOI]
- Lu, P., "Deep conditional color harmony model for image aesthetic assessment", Proc. of the 2018 24th International Conference on Pattern Recognition (ICPR), pp.2845~2850 (2018), IEEE
- Schloss, K. B., "Aesthetics of color combinations", SPIE Proc. of Human Vision and Electronic Imaging XV (2010), SPIE
- Leder, H., "A model of aesthetic appreciation and aesthetic judgments", Brit. J. Psychol, 95(4), pp.489~508 (2004)[DOI]
- Schultze, S., "Explaining Image Aesthetics Assessment : An Interactive Approach", Proc. of the 2023 ACM International Conference on Multimedia Retrieval, pp.20~28 (2023)
- Perona, F. R., "An application for aesthetic quality assessment in photography with interpretability features", Entropy, 23(11) (2021)[DOI]
- Deng, Y., "Image aesthetic assessment : An experimental survey", IEEE Signal Process. Mag, 34(4), pp.80~106 (2017)[DOI]
- Schwarz, K., "Will people like your image? Learning the aesthetic space", Proc. of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.2048~2057 (2018), IEEE
- Tong, J., "An interpretable approach for automatic aesthetic assessment of remote sensing images", Front. Comput. Neurosci., 16, pp.1077439~ (2022)[DOI]
- Li, C., "Aesthetic visual quality assessment of paintings", IEEE Journal of Selected Topics in Signal Processing, 3(2), pp.236~252 (2009)[DOI]
- Kong, S., "Photo aesthetics ranking network with attributes and content adaptation", Proc. of the 14th European Conference on Computer Vision (ECCV 2016), pp.662~679 (2016), Springer
- Droste, M., "Bauhaus 1919~1933, Dansk produktion: Book Service I", S, 1919
- Nishiyama, M., "Aesthetic quality classification of photographs based on color harmony", Proc. of CVPR 2011, pp.33~40 (2011), IEEE
- Krizhevsky, A., "ImageNet classification with deep convolutional neural networks", Commun. ACM, 60, pp.84~90 (2017)[DOI]
- Lu, X., "RAPID : Rating pictorial aesthetics using deep learning", Proc. of the 22nd ACM International Conference on Multimedia, pp.457~466 (2014)
- Szegedy, C., "Going deeper with convolutions", Proc. of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.1~9 (2015), IEEE
- Talebi, H., "NIMA : Neural image assessment", IEEE Transactions on Image Processing, 27(8), pp.3998~4011 (2018)[DOI]
- Wu, O., "Learning to predict the perceived visual quality of photos", Proc. of the 2011 International Conference on Computer Vision, pp.225~232 (2011), IEEE
- Kong, S., "Photo aesthetics ranking network with attributes and content adaptation", Proc. of the 14th European Conference on Computer Vision (ECCV 2016), pp.662~679 (2016), Springer
- Gao, F., "Style-adaptive photo aesthetic rating via convolutional neural networks and multi-task learning", Neurocomputing, 395, pp.247~254 (2020)[DOI]
- Murray, N., "AVA : A large-scale database for aesthetic visual analysis", Proc. of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp.2408~2415 (2012), IEEE
- "Color Thief"[URL]
- Ke, Y., "Image aesthetics assessment using composite features from transformer and CNN", Multimedia Systems, 29(5), pp.2483~2494 (2023)
- Li, Xuewei, "A novel feature fusion method for computing image aesthetic quality", IEEE access, 8, pp.63043~63054 (2020)[DOI]