High-Quality Depth Map Generation of Humans in Monocular Videos
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The quality of 2D-to-3D conversion depends on the accuracy of the assigned depth to scene objects. Manual depth painting for given objects is labor intensive as each frame is painted. Specifically, a human is one of the most challenging objects for a high-quality conversion, as a human body is an articulated figure and has many degrees of freedom (DOF). In addition, various styles of clothes, accessories, and hair create a very complex silhouette around the 2D human object. We propose an efficient method to estimate visually pleasing depths of a human at every frame in a monocular video. First, a 3D template model is matched to a person in a monocular video with a small number of specified user correspondences. Our pose estimation with sequential joint angular constraints reproduces a various range of human motions (i.e., spine bending) by allowing the utilization of a fully skinned 3D model with a large number of joints and DOFs. The initial depth of the 2D object in the video is assigned from the matched results, and then propagated toward areas where the depth is missing to produce a complete depth map. For the effective handling of the complex silhouettes and appearances, we introduce a partial depth propagation method based on color segmentation to ensure the detail of the results. We compared the result and depth maps painted by experienced artists. The comparison shows that our method produces viable depth maps of humans in monocular videos efficiently.
Abstract
The quality of 2D-to-3D conversion depends on the accuracy of the assigned depth to scene objects. Manual depth painting for given objects is labor intensive as each frame is painted. Specifically, a human is one of the most challenging objects for a high-quality conversion, as a human body is an articulated figure and has many degrees of freedom (DOF). In addition, various styles of clothes, accessories, and hair create a very complex silhouette around the 2D human object. We propose an efficient method to estimate visually pleasing depths of a human at every frame in a monocular video. First, a 3D template model is matched to a person in a monocular video with a small number of specified user correspondences. Our pose estimation with sequential joint angular constraints reproduces a various range of human motions (i.e., spine bending) by allowing the utilization of a fully skinned 3D model with a large number of joints and DOFs. The initial depth of the 2D object in the video is assigned from the matched results, and then propagated toward areas where the depth is missing to produce a complete depth map. For the effective handling of the complex silhouettes and appearances, we introduce a partial depth propagation method based on color segmentation to ensure the detail of the results. We compared the result and depth maps painted by experienced artists. The comparison shows that our method produces viable depth maps of humans in monocular videos efficiently.
참고문헌 (18)
- M. Guttmann, "Semi-automatic stereo extraction from video footage", Computer Vision, 2009 IEEE 12th International Conference, pp.136~142 (2009), IEEE
- O. Wang, "Stereobrush: interactive 2d to 3d conversion using discontinuous warps", International Symposium on Sketch- Based Interfaces and Modeling (2011)
- X. Yan, "Depth map generation for 2d-to-3d conversion by limited user inputs and depth propagation", 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), pp.1~4 (2011), IEEE
- R. B. i. Ribera, "QVideo panorama for 2d to 3d conversion", Computer Graphics Forum, 31(7pt2), pp.2213~2222 (2012)
- L.-M. Po, "Automatic 2d-to-3d video conversion technique based on depth-from-motion and color segmentation", Signal Processing (ICSP), 2010 IEEE 10th International Conference, pp.1000~1003 (2010)
- A. McKenzie, "Terrain geometry from monocular image sequences", Journal of Computing Science and Engineering, 2(1), pp.98~108 (2008)
- H. Hwang, "Stereoscopic image generation of background terrain scenes", Computer Animation and Virtual Worlds (2011)
- B. Ward, "Depth director: A system for adding depth to movies", Computer Graphics and Applications, IEEE, 31(1), pp.36~48 (2011)
- D. Anguelov, "Scape: shape completion and animation of people", ACM Transactions on Graphics (TOG), 24(3), pp.408~416 (2005)
- A. Jain, "Moviereshape: Tracking and reshaping of humans in videos", ACM Transactions on Graphics (TOG), 29(6), pp.148~ (2010)
- S. Zhou, "Parametric reshaping of human bodies in images", ACM Transactions on Graphics (TOG), 29(4), pp.126~ (2010)
- A. Agarwal, "Recovering 3d human pose from monocular images", Pattern Analysis and Machine Intelligence, IEEE Transactions, 28(1), pp.44~58 (2006)
- D. DiFranco, "Reconstruction of 3d figure motion from 2d correspondences", Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference, 1, pp.1~307 (2001), IEEE
- G. Loy, "Monocular 3d reconstruction of human motion in long action sequences", Computer Vision-ECCV 2004, pp.442~455 (2004)
- X. Wei, "Videomocap: Modeling physically realistic human motion from monocular video sequences", ACM Transactions on Graphics (TOG), 29(4), pp.42~ (2010)
- "levmar: Levenberg-marquardt nonlinear least squares algorithms in C/C++,”" (2004)[URL]
- D. Comaniciu, "Mean shift: A robust approach toward feature space analysis", Pattern Analysis and Machine Intelligence, IEEE Transactions, 24(5), pp.603~619 (2002)
- K. Hormann, "Mean value coordinates for arbitrary planar polygons", ACM Transactions on Graphics (TOG), 25(4), pp.1424~1441 (2006)