An RCCL-based GAN for Illustrative Sketch Generation from Game Scenes
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
We present a GAN-based framework for producing illustrative sketches from game scenes. This framework includes a relaxed cycle consistency loss (RCCL) module that estimates the difference between the edges of game scenes and resulting sketches extracted by the holistically-nested edge detection (HED) scheme. We compare the two edges using the learned perceptual image patch similarity (LPIPS) metric to focus on clear line representations. We also employ an attention map that focuses on semantic areas to identify areas that need to be intensively transformed. Our framework consists of a style extraction module, a generator module, a discriminator module, and a relaxed cycle consistency loss module. First, We extract the styles of illustrative sketch images using the style extraction module. Next, we generate illustrative sketches via style attention maps extracted from game scenes using the generator module. We then process the generated sketch images to the discriminator module to obtain the probability that the generated sketch satisfies the quality of illustrative sketch. We also apply RCCL to maintain the structure of the game scenes in the generated illustrative sketches. We demonstrate the superiority of our framework by illustrative sketches from various game scenes.
Abstract
We present a GAN-based framework for producing illustrative sketches from game scenes. This framework includes a relaxed cycle consistency loss (RCCL) module that estimates the difference between the edges of game scenes and resulting sketches extracted by the holistically-nested edge detection (HED) scheme. We compare the two edges using the learned perceptual image patch similarity (LPIPS) metric to focus on clear line representations. We also employ an attention map that focuses on semantic areas to identify areas that need to be intensively transformed. Our framework consists of a style extraction module, a generator module, a discriminator module, and a relaxed cycle consistency loss module. First, We extract the styles of illustrative sketch images using the style extraction module. Next, we generate illustrative sketches via style attention maps extracted from game scenes using the generator module. We then process the generated sketch images to the discriminator module to obtain the probability that the generated sketch satisfies the quality of illustrative sketch. We also apply RCCL to maintain the structure of the game scenes in the generated illustrative sketches. We demonstrate the superiority of our framework by illustrative sketches from various game scenes.
참고문헌 (15)
- Kang, H., "Flow-based image abstraction", IEEE transactions on visualization and computer graphics, 15(1), pp.62~76 (2008)
- 양희경, "게임 캐릭터의 사실적인 연필 렌더링 기법", 한국컴퓨터게임학회논문지, (23), pp.49~59 (2010), (사)한국컴퓨터게임학회
- 양희경, "게임 영상에 대한 가속화된 컬러 연필 렌더링 기법", 한국컴퓨터게임학회논문지, 24(4), pp.127~136 (2011), (사)한국컴퓨터게임학회
- Kingma, D. P., "Auto-Encoding Variational Bayes", arXiv : 1312. 6114, ~ (2013)
- Goodfellow, I., "Generative Adversarial Nets", Proc. of NIPS 2014, pp.2672~2680 (2014)
- Mirza, M., "Conditional Generative Adversarial Nets", arXiv : 1411. 1784, ~ (2014)
- Xie, S., "Holistically-Nested Edge Detection", Proc. of ICCV 2015, pp.1395~1403 (2015)
- Isola, P., "Image-To-Image Translation With Conditional Adversarial Networks", Proc. of CVPR 2017, pp.1125~1134 (2017)
- Zhu, J. -Y., "Unpaired image-to-image translation using cycle-consistent adversarial networks", Proc. of ICCV 2017, pp.2223~2232 (2017)
- Yi, Z., "DualGAN: Unsupervised Dual Learning for Image-to-Image Translation", Proc. of ICCV 2017, pp.2849~2857 (2017)
- Zhang, R., "The Unreasonable Effectiveness of Deep Features as a Perceptual Metric", Proc. of CVPR 2018, pp.586~595 (2018)
- Huang, X., "Multimodal unsupervised imageto-image translation", Proc. of ECCV 2018, pp.172~189 (2018)
- Kim, J., "U-gat-it : unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation", arXiv : 1907. 10830, ~ (2019)
- Yi, R., "Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping", Proc. of CVPR 2020, pp.8217~8225 (2020)
- Kampelmuhler, M., "Synthesizing human-like sketches from natural images using a conditional convolutional decoder", Proc. of WACV 2020, pp.3203~3211 (2020)