2D+Gan: Generating 2D+ Images using GAN-based a Depth map and an Image Segmentation Estimation
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
2D puzzles are popular board games. The technique of completing a 2D puzzle has been studied a lot. However, there is a limitation in that it is difficult to effectively express objects only with 2D. We present a 2D+ Lego puzzle generation algorithm that converts an input image into a 2D Lego puzzle with several bricks that represent heights. We employ a depth map estimation scheme and a semantic segmentation scheme for our purpose. We need to properly process the depth and segmentation information of various objects to apply to the 2D+ puzzle. For this reason, we apply deep learning models to the model to extract a depth map and a segmentation map. To extract the segmentation map, we adopted BiseNet learned with CelebAMask-HQ dataset. And DenseDepth learned with NYU Depth V2 dataset was used to obtain the depth map. We downsample the input image to a low-resolutional image and arrange the color information of the image according to the color palette of the allowable Lego bricks. We build a 2D+ pixel art image by corporating the low-resolutional images with the depth map and segmentation map. 2D+ lego puzzle is constructed from this 2D+ pixel art image by applying a greedy algorithm that seeks a largest Lego brick that can fit the candidate space.
Abstract
2D puzzles are popular board games. The technique of completing a 2D puzzle has been studied a lot. However, there is a limitation in that it is difficult to effectively express objects only with 2D. We present a 2D+ Lego puzzle generation algorithm that converts an input image into a 2D Lego puzzle with several bricks that represent heights. We employ a depth map estimation scheme and a semantic segmentation scheme for our purpose. We need to properly process the depth and segmentation information of various objects to apply to the 2D+ puzzle. For this reason, we apply deep learning models to the model to extract a depth map and a segmentation map. To extract the segmentation map, we adopted BiseNet learned with CelebAMask-HQ dataset. And DenseDepth learned with NYU Depth V2 dataset was used to obtain the depth map. We downsample the input image to a low-resolutional image and arrange the color information of the image according to the color palette of the allowable Lego bricks. We build a 2D+ pixel art image by corporating the low-resolutional images with the depth map and segmentation map. 2D+ lego puzzle is constructed from this 2D+ pixel art image by applying a greedy algorithm that seeks a largest Lego brick that can fit the candidate space.
참고문헌 (13)
- Gower, R., "LEGO: Automated model construction", Proc. of 32nd Eur. Study Group Ind, pp.81~94 (1998)
- Petrovic, P., "Solving LEGO brick layout problem using evolutionary algorithms", Proc. of Norwegian Conf. Comput. Sci, pp.87~97 (2001)
- Testuz, R., "Automatic generation of constructable brick sculptures", Proc. of Eurographics, pp.81~84 (2013)
- Kyungha Min, "Legorization from silhouette-fitted voxelization", KSII Transactions on Internet and Information Systems, 12(6), pp.2782~2805 (2018), 한국인터넷정보학회[DOI]
- 박철성, "Automatic Generation of LEGO Model Using Multi-height Bricks", 한국컴퓨터게임학회논문지, 30(1), pp.49~56 (2017), (사)한국컴퓨터게임학회[DOI]
- Zhang, M., "Blocklizer: Interactive design of stable mini block artwork", Proc. of ACM SIGGRAPH Posters, pp.18~ (2014)
- Kuo, M. -H., "Pixel2Brick : Constructing brick sculptures from pixel art", Journal of Comput. Graph. Forum, 34(7), pp.339~348 (2015)
- Park, C., "Generating 2D Lego Compatible Puzzles Using Reinforcement Learning", Proc. of IEEE Access, pp.180394~180410 (2020)
- Z. Wang, "Dueling network architectures for deep reinforcement learning", Proc. of Int. Conf. Mach. Learn, pp.1995~2003 (2016)
- Alhashim, I., "High quality monocular depth estimation via transfer learning", arXiv preprint arXiv:1812.11941, ~ (2018)
- Yu, C., "Bisenet: Bilateral segmentation network for real-time semantic segmentation", Proc. of the ECCV, pp.325~341 (2018)
- Chollet, F., "Xception: Deep learning with depthwise separable convolutions", Proc. of the CVPR, pp.1251~1258 (2017)
- Lennon, K., "Image2Lego: Customized LEGO Set Generation from Images", arXiv preprint arXiv:2108.08477, ~ (2021)