A GAN-based face reenactment technique for non-human game characters using coloring module
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Recently, many progresses on deep learning-based facial expression reenactment are released. Applying the facial reenactment techniques to game characters should address a challenge of extracting action units (AUs) from non-human game characters. To extract AUs from non-human game characters effectively, we devise a coloring module that adjusts the face colors of the character to that of a normal human. Our framework is based on GAN. Our framework consists of a coloring module, two generators, two discriminators, and an identity preserving module. In the first, we adjust the face color using the coloring module and then generate a neutral face from the first generator according to the input AU. The second generator produces the reenacted face images from the neutral face. Finally, the coloring module applies the original face color to the reenacted facial images. We measure the quality of the generated image through the discriminators and preserve the identity through the identity preserving module. We generate reenacted images from various game characters and prove the excellency of our method.
Abstract
Recently, many progresses on deep learning-based facial expression reenactment are released. Applying the facial reenactment techniques to game characters should address a challenge of extracting action units (AUs) from non-human game characters. To extract AUs from non-human game characters effectively, we devise a coloring module that adjusts the face colors of the character to that of a normal human. Our framework is based on GAN. Our framework consists of a coloring module, two generators, two discriminators, and an identity preserving module. In the first, we adjust the face color using the coloring module and then generate a neutral face from the first generator according to the input AU. The second generator produces the reenacted face images from the neutral face. Finally, the coloring module applies the original face color to the reenacted facial images. We measure the quality of the generated image through the discriminators and preserve the identity through the identity preserving module. We generate reenacted images from various game characters and prove the excellency of our method.
참고문헌 (20)
- Ganin, Y., "Deepwarp: Photorealistic image resynthesis for gaze manipulation", Proc. of ECCV 2016, pp.311~326 (2016)
- Chen, Y. -C., "Facelet-bank for fast portrait manipulation", Proc. of CVPR 2018, pp.3541~3549 (2018)
- Chung, J. S., "You said that?", arXiv : 1705. 02966, ~ (2017)
- Zhixin, S., "Deforming autoencoders: Unsupervised disentangling of shape and appearance", Proc. of ECCV 2018, pp.650~665 (2018)
- Wiles, O., "X2face: A network for controlling face generation by using images, audio, and pose codes", Proc. of ECCV 2018, pp.670~686 (2018)
- Goodfellow, I., "Generative adversarial nets", Proc. of NIPS 2014, pp.2672~2680 (2014)
- Zhang, J., "Freenet:Multi-identity face reenactment", Proc. of CVPR 2020, pp.5326~5335 (2020)
- Huang, P. -H., "Learning identity-invariant motion representations for cross-ID face reenactment", Proc. of CVPR 2020, pp.7084~7092 (2020)
- Ha, S., "Marionette: Few-shot face reenactment preserving identity of unseen targets", Proc. of AAAI 2020, pp.10893~10900 (2020)
- Zeng, X., "Realistic face reenactment via self-supervised disentangling of identity and pose", Proc. of AAAI 2020, pp.12757~12764 (2020)
- Kim, H., "Deep video portraits", ACM ToG, 37(4), pp.1~14 (2018)
- Wu, W., "Reenactgan: Learning to reenact faces via boundary transfer", Proc. of ECCV 2018, pp.603~619 (2018)
- Mirza, M., "Conditional gener-ative adversarial nets", arXiv : 1411. 1784, ~ (2014)
- Tripathy, S., "Icface: Interpretable and controllable face reenactment using GANs", Proc. of WACV 2020, pp.3385~3394 (2020)
- Baltrusaitis, T., "Openface 2.0: Facial behavior analysis toolkit", Proc. of FG 2018, pp.59~66 (2018)
- Baltruaitis, T., "Cross-dataset learning and person-specific normalisation for automatic action unit detection", Proc. of FG 2015, pp.1~6 (2015)
- Pumarola, A., "Ganimation: Anatomically-aware facial animation from a single image", Proc. of ECCV 2018, pp.818~833 (2018)
- Xu, R., "Face transfer with generative adversarial network", arXiv:1710.06090, ~ (2017)
- He, K., "Deep residual learning for image recognition", Proc. of CVPR 2016, pp.770~778 (2016)
- Wu, X., "A light CNN for deep face representation with noisy labels", Proc, of IEEE Transactions on Information Forensics and Security 2018, pp.2884~2896 (2018)