Image-based Virtual Try-On System: Full Automatic System Design and its Performance
0
조회
0
다운로드
7
인용
0
참고문헌
키워드
초록
We present an 2D image-based Virtual Try-On (VTON) system applicable to online shopping. The proposed method and system eliminate the need for the 3-D information requirement of clothes and humans, leveraging the recent advance in DNN-based image processing and computer graphics technology. In the first step, the cloth images are parsed into clothes and parts of a human body with a semantic segmentation network. And the pose or joint information of the cloth and human images are extracted using a pose estimation network. In the second step, the segmented cloth and joint information are jointly used for matching and deforming the cloth images onto a target human image. From our full implementation and preliminary trial, many successful result with from simple shaped and posed cloths and pose images demonstrates the feasibility of VTON system. However, we also observed many failed cases due to the diverse cloth and poses. We discuss the causes of failures to identify the applicable area of the current state-of-the art technology for commercial level VTON, and long term key problems for a future VTON system.
Abstract
We present an 2D image-based Virtual Try-On (VTON) system applicable to online shopping. The proposed method and system eliminate the need for the 3-D information requirement of clothes and humans, leveraging the recent advance in DNN-based image processing and computer graphics technology. In the first step, the cloth images are parsed into clothes and parts of a human body with a semantic segmentation network. And the pose or joint information of the cloth and human images are extracted using a pose estimation network. In the second step, the segmented cloth and joint information are jointly used for matching and deforming the cloth images onto a target human image. From our full implementation and preliminary trial, many successful result with from simple shaped and posed cloths and pose images demonstrates the feasibility of VTON system. However, we also observed many failed cases due to the diverse cloth and poses. We discuss the causes of failures to identify the applicable area of the current state-of-the art technology for commercial level VTON, and long term key problems for a future VTON system.
참고문헌 (16)
- K. Yamaguchi, "Parsing clothing in fashion photographs", Proc. of Computer Vision and Pattern Recognition (CVPR) 2012 (2012)
- K. Yamaguchi, "Paper doll parsing: Retrieving similar styles to parse clothing items", Proc. of International Conference Computer Vision (ICCV), 2013 (2013)
- J. Kim, "Adoption of virtual try-on technology for online apparel shopping", Journal of Interactive Marketing, 22(2), pp.45~59 (2008)
- X. Liang, "Deep human parsing with active template regression", IEEE Tran. on pattern analysis and machine intelligence, 37(12), pp.2402~2414 (2015)
- S. Johnson, "Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation", Proc. of the 21st British Machine Vision Conference (BMVC2010 (2010)
- M. Andriluka, "2D Human Pose Estimation: New Benchmark and State of the Art Analysis", IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
- J. Long, "Fully convolutional networks for semantic segmentation", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.3431~3440 (2015)
- B. Vijay, "Segnet: A deep convolutional encoder-decoder architecture for image segmentation", IEEE transactions on pattern analysis and machine intelligence, 39(12), pp.2491~2495 (2017)
- P. Krahenbuhl, "Efficient inference in fully connected crfs with gaussian edge potentials, Advances in neural information processing systems", ~ (2011)
- E. Insafutdinov, "DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model", European Conference on Computer Vision (ECCV) (2016)
- E. Insafutdinov, "ArtTrack: Articulated Multi-person Tracking in the Wild", Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
- F. L. Bookstein, "Principal warps: Thin-plate splines and the decomposition of deformations", IEEE Transactions on pattern analysis and machine intelligence, 11(6), pp.567~585 (1989)
- T. Igarashi, "As-rigid-as-possible shape manipulation", ACM transactions on Graphics, 24(3), pp.1134~1141 (2005)
- N. Kim, "Image-based dress up system", ACM SIGGRAPH'09, pp.35~ (2009)
- D. Pathak, "Context encoders: Feature learning by inpainting", Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
- X. Han, "VITON: An Image-based Virtual Try-on Network", Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)