멀티모달 텍스트-이미지 정렬을 활용한 블라인드 폭우얼굴 영상 복원
Blind Heavy Rain Face Image Restoration Using Multimodal Text-Image Alignment
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
이 논문에서는 폭우 환경에서 얼굴 인식 개선을 위한 영상 복원 기술을 제안하고자 한다. 최근 주목받고 있는 블라인드 생성형얼굴 영상 복원은 랜덤 벡터에서 통계적 특징 변환을 통해 고해상도 영상을 생성한다. 하지만, 기상 조건을 고려하지 않고 단순히해상도 개선에 초점을 두기에 폭우 환경에서 복구된 영상에서 텍스처 왜곡이나 디테일 부족 현상이 발생한다. 따라서 본 연구에서는사전 학습된 텍스트-영상 정렬 모델을 활용하여 텍스트 정보로부터 통계적 특징 분포를 추출하는 얼굴 영상 복원 기법을 제안하고자한다. 실험 결과를 통해, 제안한 멀티모달 기반 영상 복원 모델이 텍스트 정보를 활용함으로써 최종 복원 영상의 텍스처 및 색상 왜곡 문제를 해소하고 디테일 표현 능력을 개선할 수 있음을 확인하였다.
Abstract
In this paper, we propose an image restoration technique for improving face recognition in a heavy rainenvironment. Recently, blind generative face image restoration, which has been attracting attention, generates ahigh-resolution image through statistical feature transformation from a random vector. However, since it simplyfocuses on improving the resolution without considering the weather conditions, the images restored in a heavyrain environment have texture distortion and lack of detail. Therefore, in this study, we propose a face imagerestoration technique that extracts statistical feature transformation from text information by utilizing a pretrainedtext-image alignment model. Through the experimental results, we confirm that the proposed multimodal-basedimage restoration model can solve the texture and color distortion problems of the final restored image andimprove the detail expression capability by utilizing text information.
참고문헌 (15)
- C. -H. Son, "Near-infrared fusion via color regularization for haze and color distortion removals", IEEE Transactions on Circuits and Systems for Video Technology, 28(11), pp.3111~3126 (2018)[DOI]
- C. H. Son, "Rain removal via shrinkage of sparse codes and learned rain dictionary", Proc. IEEE International Conference on Multimedia and Expo Workshop (2016)
- L. W. Kang, "Automatic single-image-based rain steaks removal via image decomposition", IEEE Transactions on Image Processing, 21(4), pp.1742~1755 (2012)[DOI]
- R. Li, "Heavy rain image restoration : Integrating physics model and conditional adversarial learning", Proc. IEEE International Conference on Computer Vision and Pattern Recognition, pp.1633~1642 (2019)
- X. Wang, "Esrgan : Enhanced superresolution generative adversarial networks", Proc. European Conference on Computer Vision, pp.63~79 (2018)
- C. Chen, "Progressive semantic-aware style transformation for blind face restoration", Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp.11896~11905 (2021)
- T. Karras, "Analyzing and improving the image quality of stylegan", Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp.8110~8119 (2020)
- W. Xia, "Tedigan : Text-guided diverse face image generation and manipulation", Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp.2256~2265 (2020)
- L. Yang, "Hifacegan : Face renovation via collaborative suppression and replenishment", Proc. ACM International onference on Multimedia, pp.1551~1560 (2020)
- Y. Chen, "FSRNet : End-to-end learning face super-resolution with facial priors", Proc. IEEE International Conference on Computer Vision and Pattern Recognition, pp.2492~2501 (2018)
- Z. Wang, "Image quality assessment : from error visibility to structural similarity", IEEE Transactions on Image Processing, 13(4), pp.600~612 (2004)[DOI]
- R. Zhang, "The Unreasonable effectiveness of deep features as a perceptual metric", IEEE Conference on Computer Vision and Pattern Recognition, pp.586~595 (2018)