Few-Shot Image Synthesis using Noise-Based Deep Conditional Generative Adversarial Nets
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In recent years research on automatic font generation with machine learning mainly focus on using transformation-based methods, in comparison, generative model-based methods of font generation have received less attention. Transformation-based methods learn a mapping of the transformations from an existing input to a target. This makes them ambiguous because in some cases a single input reference may correspond to multiple possible outputs. In this work, we focus on font generation using the generative model-based methods which learn the buildup of the characters from noise-to-image. We propose a novel way to train a conditional generative deep neural model so that we can achieve font style control on the generated font images. Our research demonstrates how to generate new font images conditioned on both character class labels and character style labels when using the generative model-based methods. We achieve this by introducing a modified generator network which is given inputs noise, character class, and style, which help us to calculate losses separately for the character class labels and character style labels. We show that adding the character style vector on top of the character class vector separately gives the model rich information about the font and enables us to explicitly specify not only the character class but also the character style that we want the model to generate.
Abstract
In recent years research on automatic font generation with machine learning mainly focus on using transformation-based methods, in comparison, generative model-based methods of font generation have received less attention. Transformation-based methods learn a mapping of the transformations from an existing input to a target. This makes them ambiguous because in some cases a single input reference may correspond to multiple possible outputs. In this work, we focus on font generation using the generative model-based methods which learn the buildup of the characters from noise-to-image. We propose a novel way to train a conditional generative deep neural model so that we can achieve font style control on the generated font images. Our research demonstrates how to generate new font images conditioned on both character class labels and character style labels when using the generative model-based methods. We achieve this by introducing a modified generator network which is given inputs noise, character class, and style, which help us to calculate losses separately for the character class labels and character style labels. We show that adding the character style vector on top of the character class vector separately gives the model rich information about the font and enables us to explicitly specify not only the character class but also the character style that we want the model to generate.
참고문헌 (22)
- I. J. Goodfellow, "Generative adversarial networks", NIPS, Proc. of Neural Information Processing Systems, pp.2672~2680 (2014)
- S. Reed, "Generative adversarial text to image synthesis", arXiv preprint 1605. 05396, ~ (2016)
- P. Isola, "Image-to-Image translation with conditional adversarial networks", CVPR, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
- "Tips for Training Stable Generative Adversarial Networks"[URL]
- Y. Choi, "StarGAN : Unified generative adversarial networks for multi-domain image-to-image translation", CVPR, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
- Y. Choi, "StarGAN v2 : Diverse image synthesis for multiple domains", arXiv preprint arXiv : 1912. 01865, ~ (2019)
- L. A. Gatys, "Image style transfer using convolutional neural networks", CVPR, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2414~2423 (2016)
- G. Atarsaikhan, "Neural font style transfer", ICDAR, Proc. of 14th International Conference on Document Analysis and Recognition, pp.51~56 (2017)
- "Rewrite: Neural style transfer for Chinese fonts"[URL]
- "Zi2Zi: Master Chinese calligraphy with conditional adversarial networks"[URL]
- Y. Jiang, "DCFont : An end-to-end deep Chinese font generation system", SIGGRAPH Asia, Proc. of SIGGRAPH Asia Technical Briefs (2017)
- Y. Jiang, "SCFont : Structure-guided Chinese font generation via deep stacked networks", AAAI, Proc. of 33rd AAAI Conference on Artificial Intelligence (2019)
- H. Hayashi, "GlyphGAN : Style-consistent font generation based on generative adversarial networks", arXiv preprint arXiv : 1905. 12502, ~ (2019)
- M. Arjovsky, "Wasserstein generative adversarial networks", Machine Learning, Proc. of 34th International Conference on Machine Learning, pp.214~223 (2017)
- A. Radford, "Unsupervised representation learning with deep convolutional generative adversarial networks", arXiv preprint arXiv:1511.06434, ~ (2015)
- M. Mirza, "Conditional generative adversarial nets", arXiv preprint arXiv : 1411. 1784, ~ (2014)
- A. Krizhevsky, "ImageNet classification with deep convolutional neural networks", NIPS, Proc. of Neural Information Processing Systems, pp.1097~1105 (2012)
- 노순국, "인공지능 기반 구글넷 딥러닝과 IoT를 이용한 의류 분류", 스마트미디어저널, 9(3), pp.41~45 (2020), (사)한국스마트미디어학회[DOI]
- 김영광, "자율주행 환경에서 이미지 객체 분할을 위한 강화된 DFCN 알고리즘 성능연구", 스마트미디어저널, 9(4), pp.9~16 (2020), (사)한국스마트미디어학회[DOI]
- S. Ioffe, "Batch Normalization: Accelerating deep network training by reducing internal covariate shift", ICML, Proc. of International Conference on Machine Learning (2015)
- "Handwritten Korean Character Recognition with TensorFlow and Android"[URL]
- J. Nilsson, "Understanding SSIM", arXiv preprint arXiv:2006.13846, 20, ~