FillingGAN: A Deep Learning-based Automatic Coloring Technique from Line Drawings for Developing Coloring Game Contents
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In this paper, we contribute to the field of game by presenting FillingGAN, an automatic coloring framework using a generative adversarial network (GAN). FillingGAN is devised to generate a coloring image from a line drawing by filling empty regions between the lines in the line drawing image from the coloring styles learned from sample coloring images. Our model consists of two style extracting modules and a GAN model. The style extracting modules are designed as an auto-encoder that extracts feature from input images. One module extracts color styles from coloring sample images and the other extracts structure from a line drawing. FillingGAN executes coloring process by applying the coloring styles from coloring samples to a line drawing. It determines the similarity between the generated coloring image and the input line drawing, and calculate the perceptual loss to preserve the structural similarity. FillingGAN generates coloring images with preserved details by adjusting the weights between the structural feature vectors and the style feature vectors. As a result, it generates a stylized image without distorting the unique features of a line drawing. Our framework can be improved to apply various styles including artistic media strokes to line drawings.
Abstract
In this paper, we contribute to the field of game by presenting FillingGAN, an automatic coloring framework using a generative adversarial network (GAN). FillingGAN is devised to generate a coloring image from a line drawing by filling empty regions between the lines in the line drawing image from the coloring styles learned from sample coloring images. Our model consists of two style extracting modules and a GAN model. The style extracting modules are designed as an auto-encoder that extracts feature from input images. One module extracts color styles from coloring sample images and the other extracts structure from a line drawing. FillingGAN executes coloring process by applying the coloring styles from coloring samples to a line drawing. It determines the similarity between the generated coloring image and the input line drawing, and calculate the perceptual loss to preserve the structural similarity. FillingGAN generates coloring images with preserved details by adjusting the weights between the structural feature vectors and the style feature vectors. As a result, it generates a stylized image without distorting the unique features of a line drawing. Our framework can be improved to apply various styles including artistic media strokes to line drawings.
참고문헌 (20)
- 황인화, "노인을 위한 포토 컬러링북 개발 연구", Journal of Integrated Design Research, 17(1), pp.65~77 (2018), 디자인연구소[DOI]
- Kang, H., "Flow-based image abstraction", IEEE Transactions on Visualization and Computer Graphics, 15(1), pp.62~76 (2008)
- Kyprianidis, J., E, "State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video", IEEE Transactions on Visualization and Computer Graphics, 19(5), pp.866~885 (2012)
- Kingma D. P., "Auto-Encoding Variational Bayes", arXiv : 1312. 6114, ~ (2013)
- Goodfellow, I., "Generative Adversarial Nets", Proc. of NIPS 2014, pp.2672~2680 (2014)
- Radford, A., "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", arXiv : 1511. 06434, ~ (2015)
- Chen, X., "InfoGAN : Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets", arXiv : 1606. 03657, ~ (2016)
- Mao, X., "Least Squares Generative Adversarial Networks", Proc. of ICCV, pp.2794~2802 (2017)
- Yi, Z., "DualGAN: Unsupervised Dual Learning for Image-to-Image Translation", Proc. of ICCV 2017, pp.2849~2857 (2017)
- Isola, P., "Image-to-image translation with conditional adversarial networks", Proc. of CVPR 2017, pp.1125~1134 (2017)
- Zhu, J. Y., "Unpaired image-to-image translation using cycle-consistent adversarial networks", Proc. of CVPR 2017, pp.2223~2232 (2017)
- Liu, M. Y., "Unsupervised image-to-image translation net-works", arXiv : 1703. 00848, ~ (2016)
- Huang, X., "Multimodal unsupervised image-to-image translation", Proc. of ECCV 2018, pp.172~189 (2018)
- Kim, J., "U-gat-it : unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation", arXiv : 1907. 10830, ~ (2019)
- 양희경, "선택적 다중스케일 구조의 게임배경을 생성하는 영상 추상화 기법", 한국컴퓨터게임학회논문지, (19), pp.131~140 (2009), (사)한국컴퓨터게임학회
- 강동화, "CUDA를 이용한 실시간 감성 동영상 추상화 구현", 한국컴퓨터게임학회논문지, 24(3), pp.125~134 (2011), (사)한국컴퓨터게임학회
- Simonyan, K., "Very Deep Convolutional Networks for Large-Scale Image Recognition", arXiv : 1409. 1556, ~ (2015)
- Lee, H., M, "Coloring book on beautiful girls 1" (2020), Bookpin
- Lee, H., M, "Coloring book on beautiful girls 2 THE MOMENT" (2020), Bookpin