고해상도 텍스처 맵 생성을 위한 딥러닝 기반 초해상도 기법들의 비교 분석 연구
Comparative analysis of the deep-learning-based super-resolution methods for generating high-resolution texture maps
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
디스플레이의 해상도의 증가에 따라 고해상도 텍스처 맵을 내장한 앱들도 함께 증가하는 추세에 있다. 최근 딥러닝 기반이미지 초해상도 기법들의 발전은 이러한 고해상도 텍스처 생성 작업을 자동화할 수 있는 가능성을 만들고 있다. 하지만이러한 적용 사례에 대해 심층적으로 분석한 연구는 아직 부족한 상태이다. 그래서 본 논문에서는 최신 초해상도 기법들중 BSRGAN, Real-ESRGAN, SwinIR(classical 및 real-world image SR)을 택하여 텍스처 맵의 업스케일링(upscaling)에 적용한 후 그 결과를 정량적, 정성적으로 비교, 분석하였다. 실험 결과 업스케일링 후 나타나는 다양한 아티팩트(artifact)들을발견할 수 있었으며, 이를 통해 기존 초해상도 기법들을 텍스처 맵 업스케일링에 곧바로 적용하기에는 일부 미흡한 부분이존재한다는 점을 확인하였다.
Abstract
As display resolution increases, many apps also tend to include high-resolution texture maps. Recent advancements in deeplearning-based image super-resolution techniques make it possible to automate high-resolution texture generation. However, there is still a lack of comprehensive analysis of the application of these techniques to texture maps. In this paper, we selected three recent super-resolution techniques, namely BSRGAN, Real-ESRGAN, and SwinIR (classical and real-world image SR), and applied them to upscale texture maps. We then conducted a quantitative and qualitative analysis of the experimental results. The findings revealed various artifacts after upscaling, which indicates that there are still limitations in directly applying superresolution techniques to texture-map upscaling
참고문헌 (35)
- "New Cyberpunk 2077 HD texture packs improve the textures of NPCs faces, hair & beards" (2023)[URL]
- "Arkham city redux" (2023)[URL]
- L. Williams, "Pyramidal parametrics", Proceedings of the 10th annual conference on Computer graphics and interactive techniques (SIGGRAPH), pp.1~11 (1983)
- S. Takemura, "Optimize deep super-resolution and denoising for compressed textures", ACM SIGGRAPH Asia 2018Posters (2018)
- J.-H. Nah, "TexSR: Image super-resolution for high-quality texture mapping", ACM SIGGRAPH Asia 2022 Posters (2022)
- K. Zhang, "Designing a practical degradation model for deep blind image superresolution", Proceedings of the International Conference on Computer Vision(ICCV), pp.4791~4800 (2021)
- X. Wang, "Real-ESRGAN : Training real-world blind super-resolution with pure synthetic data", 2021 IEEE, 2021, 1905~1191
- J. Liang, "SwinIR : Image restoration using swin transformer", 2021 IEEE, 2021, 1833~1844
- P. Andersson, "FLIP : A difference evaluator for alternating images", Proceedings of the ACM on Computer Graphics and Interactive Techniques(HPG), 3(2), pp.1~23 (2020)
- "Deep learning super sampling (DLSS)" (2023)[URL]
- C. Dong, "Image superresolution using deep convolutional networks", Proceedings of European Conference on Computer Vision(ECCV)2014, pp.184~199 (2014)
- C. Ledig, "Photo-realistic single image super-resolution using a generative adversarial network", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.105~114 (2017)
- X. Wang, "ESRGAN: Enhanced super-resolution generative adversarial networks", Computer Vision – ECCV 2018 Workshops, pp.63~79 (2019)
- X. Ji, "Realworld super-resolution via kernel estimation and noise injection", 2020 IEEE, 2020, 1914~1923
- A. Vaswani, "Attention is all you need", ~ (2017)
- Z. Liu, "Swin transformer: Hierarchical vision transformer using shifted windows", 2021 IEEE, 2021, 9992~10002
- 나재호, "PSNR 값 기반의자동화된 ASTC 블록 크기 결정 방법", 한국컴퓨터그래픽스학회논문지, 28(2), pp.21~28 (2022), (사)한국컴퓨터그래픽스학회[DOI]
- 조우영, "A Comparative Study on OCR using Super-Resolution for Small Fonts", The International Journal of Advanced Smart Convergence, 8(3), pp.95~101 (2019), 한국인터넷방송통신학회[DOI]
- 성상하, "딥 러닝 기법을 활용한 이미지 내 한글 텍스트 인식에 관한 연구", 한국융합학회논문지, 11(6), pp.1~6 (2020), 한국융합학회[DOI]
- Y. Li, "A review of the deep learning methods for medical images super resolution problems", IRBM, 42(2), pp.120~133 (2021)
- S. Jia, "Multiattention generative adversarial network for remote sensing image superresolution", IEEE Transactions on Geoscience and Remote Sensing, 60, pp.1~15 (2022)
- Z. Wang, "Image quality assessment : from error visibility to structural similarity", IEEE Transactions on Image Processing, 13(4), pp.600~612 (2004)
- J. -H. Nah, "QuickETC2 : How to finish ETC2 compression within 1 ms", ACM SIGGRAPH 2020 Talks (2020)
- J.-H. Nah, "QuickETC2: Fast ETC2 texture compression using luma differences", ACM Transactions on Graphics (SIGGRAPH Asia 2020), 39(6), pp.10~ (2020)
- "Real- ˜time normal map DXT compression" (2008)[URL]
- "Image datasets - a reposi- ˜tory of public image data sets" (2021)[URL]
- E. Agustsson, "NTIRE 2017 challenge on single image super-resolution: Dataset and study", 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.1122~1131 (2017)
- "QualityScaler - image/video AI upscaler app (BSRGAN)" (2023)[URL]
- 육심언, "SNS 메신저 환경에서의 썸네일 이미지 기반의 새로운 스테가노그래피 통신 기법 연구", 인터넷정보학회논문지, 22(6), pp.151~162 (2021), 한국인터넷정보학회
- J. Nilsson, "Understanding SSIM", ArXiv e-prints, 2020, arXiv:2006.13846, ~ (2020)
- "DLSS - download and get started" (2023)[URL]
- J.-H. Nah, "Classified texture resizing for mobile devices", ACM SIGGRAPH 2018Talks (2018)
- "α9 Gen5 AI Processor" (2022)[URL]
- Karthik Vaidyanathan, "Random-Access Neural Compression of Material Textures", ACM Transactions on Graphics, 42(4), pp.1~25 (2023), Association for Computing Machinery (ACM)[DOI]
- J.-H. Nah, "Addition of an adaptive block-size determination feature to astcenc, the reference ASTC encoder", Software Impacts, 17 (2023)