2D증명사진기반 3D아바타 페이스 변환방법에 대한 연구
Study on 3D Avatar Face Conversion Method Based on 2D ID Photo
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구는 2차원 증명사진을 기반으로 3차원 아바타의 얼굴을 실시간으로 교체하는 기술을 제안한다. 3D 저작 도구인 블렌더를 활용하여 기존 아바타의 얼굴 메쉬를 2차원 이미지의 특징점과 매핑하고, 얼굴 표정 변형에 따른 자연스러운 변화를 구현하기 위해 리깅(rigging) 작업을 수행한다. 이후 FBX 형식으로 모델을 export하여 유니티 게임 엔진으로 가져와 실시간 렌더링 환경을 구축한다. 특히, 얼굴 피부의 질감과 색상을 정확하게 표현하기 위해 머티리얼 매칭 기법을 적용하여 현실감 있는 아바타를 생성한다. 본 연구는 이미지 생성 모델과의 연동을 통해 더욱 다양한 얼굴 특징을 가진 아바타 생성 가능성을 제시하며, 메타버스 환경에서의 개인 맞춤형 아바타 생성 기술의 발전에 기여할 것으로 기대된다.
Abstract
This research proposes a technique for real-time face replacement of a 3D avatar based on a 2D profile picture. By utilizing Blender, a 3D authoring tool, we map the facial mesh of an existing avatar to the feature points of a 2D image and perform rigging to implement natural changes according to facial expression variations. Afterward, the model is exported in FBX format and imported into the Unity game engine to establish a real-time rendering environment. Particularly, we apply a material matching technique to accurately represent the texture and color of the facial skin, resulting in a realistic avatar. This research suggests the possibility of generating avatars with a wider variety of facial features through integration with image generation models and is expected to contribute to the advancement of personalized avatar generation technology in metaverse environments.
참고문헌 (16)
- Y. Taigman, "Deepface : Closing the gap to human-level performance in face verification", Proceedings of the IEEE conference on computer vision and pattern recognition, pp.1701~1708 (2014)
- Paysan, P., "A 3D face model for pose and illumination invariant face recognition", 2009 sixth IEEE international conference on advanced video and signal based surveillance, pp.296~301 (2009), Ieee
- L. Wei, "Improved Multi-pose 2D Face Recognition Using 3D Face Model with Camera Pose Estimation Approach and nD-PCA Recognition Algorithm", 2008Third International Conference on Convergence and Hybrid Information Technology, 2, pp.728~736 (2008), IEEE
- M. Marques, "3D face recognition from multiple images: A shape-from-motion approach", 2008 8th IEEE International Conference on Automatic Face &Gesture Recognition, pp.1~6 (2008), IEEE
- A. F. Abate, "Fast 3D face recognition based on normal map", IEEE International Conference on Image Processing 2005, pp.II~946 (2005), IEEE
- H. Tang, "Real-time conversion from a single 2D face image to a 3D text-driven emotive audio-visual avatar", 2008 IEEE International Conference on Multimedia and Expo, pp.1205~1208 (2008), IEEE
- D. Kumarapeli, "Emotional avatars: Effect of uncanniness in identifying emotions using avatar expressions", 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp.650~651 (2022), IEEE
- S. Morishima, "Face image analysis and synthesis for human-computer interaction", WCC 2000-ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000, 2, pp.1213~1220 (2000), IEEE
- S. Morishima, "Realtime face analysis and synthesis using neural network", Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No. 00TH8501), 1, pp.13~22 (2000), IEEE
- S. Gurbuz, "Real-time synthesis of natural head motion on a 3D avatar from reconstructed 3D frontal face data", 2008 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, pp.277~280 (2008), IEEE
- S. Bader, "SID-avatar database: A 3D Avatar Dataset for virtual world research", International Image Processing, Applications and Systems Conference, pp.1~5 (2014), IEEE
- C. H. Huang, "Using multiple cameras to construct 3D avatar from 2D video based on thinning and tracking algorithm", 2008 First IEEE International Conference on Ubi-Media Computing, pp.249~254 (2008), IEEE
- H. Boujut, "A fully automatic framework for building 3d animated avatars", 2014 IEEE Fourth International Conference on Consumer Electronics Berlin (ICCE-Berlin), pp.36~40 (2014), IEEE
- B. Mu, "Implementation of the interactive gestures of virtual avatar based on a multi-user virtual learning environment", 2009 International Conference on Information Technology and Computer Science, 1, pp.613~617 (2009), IEEE
- M. Joachimczak, "Creating 3D personal avatars with high quality facial expressions for telecommunication and telepresence", 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp.856~857 (2022), IEEE
- R. M. Paweroi, "3D Avatar Animation Optimization in Metaverse by Differential Evolution Algorithm", 2023International Conference on Intelligent Metaverse Technologies & Applications (iMETA), pp.1~7 (2023)