A Study of Implementing a Real-Time Digital Human System Using Generative AI and Unreal Engine
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study explores the integration of generative AI with Unreal Engine's MetaHuman technology to create an advanced real-time digital human system. We analyze existing implementations combining these technologies, including Inworld AI's NPC system and NVIDIA's Avatar Cloud Engine, to inform our research. We propose a novel architecture that leverages MetaHuman's visual fidelity and generative AI's dynamic behavior. Our methodology covers character creation, AI model integration, and optimization for real-time performance. The system is evaluated based on rendering quality, AI response accuracy, and user experience. Results demonstrate the potential of this integration for creating more responsive and lifelike digital humans. We discuss implications for gaming, education, and virtual assistants, addressing current limitations and proposing future research directions. This study contributes to the field by providing a framework for combining advanced AI capabilities with state-of-the-art real-time rendering, advancing the development of intelligent virtual characters.
Abstract
This study explores the integration of generative AI with Unreal Engine's MetaHuman technology to create an advanced real-time digital human system. We analyze existing implementations combining these technologies, including Inworld AI's NPC system and NVIDIA's Avatar Cloud Engine, to inform our research. We propose a novel architecture that leverages MetaHuman's visual fidelity and generative AI's dynamic behavior. Our methodology covers character creation, AI model integration, and optimization for real-time performance. The system is evaluated based on rendering quality, AI response accuracy, and user experience. Results demonstrate the potential of this integration for creating more responsive and lifelike digital humans. We discuss implications for gaming, education, and virtual assistants, addressing current limitations and proposing future research directions. This study contributes to the field by providing a framework for combining advanced AI capabilities with state-of-the-art real-time rendering, advancing the development of intelligent virtual characters.
참고문헌 (10)
- Cao, C., "Displaced dynamic expression regression for real-time facial animation", ACM Transactions on Graphics, 33(4), pp.1~10 (2014)
- Guo, J., "Recent advances in real-time rendering of digital humans : A survey", IEEE Transactions on Visualization and Computer Graphics, 27(11), pp.4125~4146 (2021)
- Karras, T., "A style-based generator architecture for generative adversarial networks", Proceedings of the IEEE, 2019, 4401~4410
- Radford, A., "Language models are unsupervised multitask learners", OpenAI blog, 1(8), pp.9~ (2019)
- Seymour, M., "Interactive facial animation with deep neural networks", IET Computer Vision, 12(3), pp.213~222 (2018)
- Thies, J., "Face2face: Real-time face capture and reenactment of rgb videos", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2387~2395 (2016)
- Vaswani, A., "Advances in neural information processing systems", pp.5998~6008 (2017)
- Zhu, J. Y., "Unpaired image-to-image translation using cycle-consistent adversarial networks", Proceedings of the IEEE international conference on computer vision, pp.2223~2232 (2017)