피부영상 털 제거를 위한 트랜스포머 기반 인페인팅 기법:기존 방법의 적용성 평가
Transformer-based Image Inpainting Approach to Hair Removal in Skin Images: Applicability Evaluation of Existing Methods
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
피부영상에서 털 제거는 자동화된 피부진단을 위한 필수적인 전처리 과정이다. 기존의 털 제거 기법은 대부분 영상처리에 기반한 방식이었지만, 털이 많은 피부에는 성능이 크게 저하되는 문제점이 있다. 다른 접근방식으로서 심층학습 모델에 기반한 인페인팅 기법을 고려할 수 있는데, 털이 많은 피부영상 학습 데이터의 부족으로 인하여 심층학습 기반 털 제거 모델은 거의 보고된 바 없다. 본 논문에서는 피부영상에서 털 제거를 위한 영상 인페인팅 기법으로서 트랜스포머 기반 심층학습 모델인 BAT-Fill의 적용성을 평가한다. BAT-Fill은 양방향 자기회귀 트랜스포머(BAT) 기반의 거친 네트워크와 생성적 적대 신경망(GAN) 기반의 정제 네트워크로 구성되어 있다. 인페인팅은 일반적으로 이웃 픽셀의 특징으로부터 점차적으로 채워가는 방식으로 이루어지는 반면, 가느다란 털이 많이 분포되어 있는 피부에서는 이웃 피부의 특성을 정확하게 추출하기 어렵다. 트랜스포머는 서로 다른 영역 사이의 유사성을 학습하여 피부를 복원하는 방식으로서, 전역적인 피부 특징으로부터 지역적인 피부 특성을 정확하게 복원하는 것을 가능하게 한다. 모델에 대한 성능 평가 결과, BAT-Fill이 기존 방법에 비해서 PSNR 및 SSIM에서는 유의미한 차이가 없었으나 FID에서는 피부영상 털 제거에서 실감성과 인지적 품질이 향상되었음을 보여준다.
Abstract
Hair removal in skin images is the essential preprocessing task for automatized skin cancer detection. Most existing methods for hair removal are based on image processing, but their performance tends to be significantly degraded in hairy skin images. As an alternative, an image inpainting approach based on deep learning can be taken into account, while few deep learning models for hair removal have been reported due to the lack of datasets. In this paper, we evaluate the applicability of BAT-Fill which is a transformer-based deep learning model as an image inpainting method for hair removal in skin images. BAT-Fill consists of not only the coarse network based on bidirectional autoregressive transformers but also the refinement network based on generative adversarial network (GAN). While a typical process of image inpainting is to progressively fill the inner hidden block using the properties of neighbor pixels, it is challenging to accurately extract the skin properties of neighbor pixels when hairs are largely distributed. The transformer is a solution for reconstructing the skin texture by learning the similarity of pixel properties between two distant areas, where local skin textures can be accurately predicted using global skin properties. The model assessment shows that BAT-Fill is effective to enhance the perceptual quality of the hair-removed image as it exhibits significant improvement in FID although it does not in PSNR and SSIM.
참고문헌 (31)
- Q. Abbas, "Hair removal methods : A comparative study for dermoscopy images", Biomedical Signal Processing and Control, 6(4), pp.395~404 (2011)
- Salem, Nermin Mohamed Fawzy, "A Survey on Various Image Inpainting Techniques", Future Engineering Journal, 11 (2021)
- Lee T, "Dullrazor®: a software approach to hair removal from images" (1997), Comput Biol Med
- Xie F-Y, "PDE-based unsupervised repair of hair-occluded information in dermoscopy images of melanoma", Comput Med Imaging Graph, ~ (2009)
- Kiani K, "E-shaver : an improved DullRazor® for digitally removing dark and light-colored hairs in dermoscopic images" (2011), Comput Biol Med
- Fiorese M, "VirtualShave : automated hair removal from digital dermatoscopic images", 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2011)
- Abbas Q, "Hair removal methods : a comparative study for dermoscopy images", Biomed Signal Process Control (2011)
- Toossi MTB, "An effective hair removal algorithm for dermoscopy images", Skin Res Technol (2013)
- Huang A, "A robust hair segmentation and removal approach for clinical images of skin lesions", 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2013)
- Xie F, "No-reference hair occlusion assessment for dermoscopy images based on distribution feature", Comput Biol Med (2015)
- Koehoorn J, "Automated digital hair removal by threshold decomposition and morphological analysis", International symposium on mathematical morphology and its applications to signal and image processing (2015)
- Bibiloni P, "Skin hair removal in dermoscopic images using soft color morphology", Conference on artificial intelligence in medicine in Europe (2017)
- Z. Yan, "Shift-net: Image inpainting via deep feature rearrangement" (2018), European Conference on Computer Vision
- J. Yu, "Free-form image inpainting with gated convolution", International Conference on Computer Vision (2019)
- G. Liu, "Image Inpainting for Irregular Holes Using Partial Convolutions", European Conference on Computer Vision (2018)
- Y. Wang, "Image inpainting via generative multi-column convolutional neural networks", Neural Information Processing Systems (2018)
- C. Xie, "Image inpainting with learnable bidirectional attention maps", International Conference on Computer Vision (2019)
- Talavera-Martínez L, "Hair segmentation and removal in dermoscopic images using deep learning", IEEE Access (2020)
- Dalal Bardou, "Hair removal in dermoscopy images using variational autoencoders", Skin Research & Technology, 1 (2022)
- Ashish Vaswani, "Attention Is All You Need", Neural Information Processing Systems, 6 (2017)
- Alexey Dosovitskiy, "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale", International Conference on Learning Representations (2020)
- Yingchen Yu, "Diverse Image Inpainting with Bidirectional and Autoregressive Transformers", ACM Multimedia, 4 (2021)
- Taesung Park, "Semantic Image Synthesis with Spatially-Adaptive Normalization", Computer Vision and Pattern Recognition Conference, 3 (2019)
- Martin Heusel, "GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium", Neural Information Processing Systems, 6 (2017)
- Youngchan Lee, "Crossdomain Knowledge Distillation Approach to Hair Segmentation in Pet Skin Images for Skin Disease Detection", Korea Computer Congress 2022 (2022)
- Youngchan Lee, "An Exploratory Study for Hair Segmentation in Pet Skin Images Using LadderNet", Autumn Annual Conference of IEIE (2021)
- Youngchan Lee, "Deep Learning Approach to Hair Removal in Skin Images Using Synthetic Data Augmentation:Applicability Evaluation of Existing Methods", The 34th Workshop on Image Processing and Image Understanding (IPIU 2022) (2022)
- Ilya Loshchilov, "Decoupled Weight Decay Regularization", International Conference on Learning Representations (2019)
- Diederik P. Kingma, "Adam: A Method for Stochastic Optimization", International Conference on Learning Representations (2015)
- Zhou Wang, "Image quality assess ment: From error visibility to structural similarity", IEEE Transactions on Image Processing (2004)
- Rotemberg, V., "A patient-centric dataset of images and metadata for identifying melanomas using clinical context", Sci Data, 8, pp.34~ (2021)