Gaussian blending: 모델 경량화를 위한 3D Gaussian Splatting의 Density Control 개선 연구
Gaussian Blending: Improved 3D Gaussian Splatting for Model Light-Weighting and Deep Learning-Based Performance Enhancement
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
NVS는 여러 각도와 위치에서 수집한 이미지를 이용해 3차원 공간을 재현하는 연구 분야로, 증강현실, 가상현실, 자율주행, 로봇 네비게이션 등에서 중요성이 커지고 있다. 최근 주목받는 3D-GS 방법론은 기존 NeRF 보다 고품질 장면 생성, 빠른 학습 시간, 실시간 렌더링이 가능하지만, Gaussian points의 밀도 조정 과정에서 전체 Gaussian points 수의 증가로 메모리 소모량 상승과 렌더링 속도가저하되는 문제가 있다. 이를 개선하기 위해 본 논문에서는 불필요한 Gaussian points를 제거하여 메모리 효율성을 높이는 Gaussian blending 기법과 Gaussian points 감소로 인한 표현력 손실을 최소화하는깊이 정보 반영 손실 함수를 제안하여 모델의 성능을 보완한다. 실험 결과, Tanks & Temples 벤치마크 데이터셋에서 성능을 유지하면서 Gaussian points 수를 최대 4% 감소시키는 효과를 확인하였다. 따라서 본 논문에서 제안한 방법론은 3D-GS 모델의 경량화 가능성을 실험적으로 증명하였다.
Abstract
NVS (Novel View Synthesis) is a field in computer vision that reconstructs new views of a scene from a set of input views. Real-time rendering and high performance are essential for NVS technology to be effectively utilized in various applications. Recently, 3D-GS (3D Gaussian Splatting) has gained popularity due to its faster training and inference times compared to those of NeRF (Neural Radiance Fields)-based methodologies. However, since 3D-GS reconstructs a 3D (Three-Dimensional) scene by splitting and cloning (Density Control) Gaussian points, the number of Gaussian points continuously increases, causing the model to become heavier as training progresses. To address this issue, we propose two methodologies: 1) Gaussian blending, an improved density control methodology that removes unnecessary Gaussian points, and 2) a performance enhancement methodology using a depth estimation model to minimize the loss in representation caused by the blending of Gaussian points. Experiments on the Tanks and Temples Dataset show that the proposed methodologies reduce the number of Gaussian points by up to 4% while maintaining performance.
참고문헌 (27)
- Y-J. Lee, "A Design of Anomaly Behavior Detection System Based on Deep Learning Model Using CCTV", Journal of Knowledge Information Technology and Systems(JKITS), 17(2), pp.183~191 (2022)[DOI]
- J-N. Heo, "Pothole Detection Using Deep Learning and Domain-based Image Preprocessing Methods", Journal of Knowledge Information Technology and Systems(JKITS), 18(5), pp.1331~1343 (2023)[DOI]
- D-H Kim, "A Research on Cylindrical Pill Bottle Recognition with YOLOv8 and ORB", Journal of the Korea Society of Computer and Information(JKSCI), 29(2), pp.13~20 (2024)[DOI]
- Zhang, Jiahui, "FreGS : 3d gaussian splatting with progressive frequency regularization", Proceedings of the IEEE, 2024, 21424~21433
- Qin, Minghan, "Langsplat : 3d language gaussian splatting", Proceedings of the IEEE, 2024, 20051~20060
- Zhou, Xiaoyu, "Drivinggaussian : Composite gaussian splatting for surrounding dynamic autonomous driving scenes", Proceedings of the IEEE, 2024, 21634~21643
- Mildenhall, Ben, "Nerf : Representing scenes as neural radiance fields for view synthesis", Communications of the ACM, 65(1), pp.99~106 (2021)[DOI]
- Lu, Tao, "Scaffold-gs : Structured 3d gaussians for view-adaptive rendering", Proceedings of the IEEE, 2024, 20654~20664
- Yu, Alex, "pixelnerf : Neural radiance fields from one or few images", Proceedings of the IEEE, 2021, 4578~4587
- Xu, Qiangeng, "Point-nerf : Point-based neural radiance fields", Proceedings of the IEEE, 2022, 5438~5448
- Barron, Jonathan T., "Mip-nerf : A multiscale representation for anti-aliasing neural radiance fields", Proceedings of the IEEE, 2021, 5855~5864
- B. Kerbl, "3D Gaussian Splatting for Real-Time Radiance Field Rendering", ACM Transactions on Graphics, 42(4) (2023)[DOI]
- J. L. Schonberger, "Structure-from-motion revisited", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.4104~4113 (2016)
- Lee, Joo Chan, "Compact 3d gaussian representation for radiance field", Proceedings of the IEEE, 2024, 21719~21728
- Barron, Jonathan T., "Zip-nerf : Anti-aliased grid-based neural radiance fields", Proceedings of the IEEE, 2023, 19697~19705
- Müller, Thomas, "Instant neural graphics primitives with a multiresolution hash encoding", ACM Transactions on Graphics, 41(4), pp.1~15 (2022)[DOI]
- Hu, Wenbo, "Tri-miprf : Tri-mip representation for efficient anti-aliasing neural radiance fields", Proceedings of the IEEE, 2023, 19774~19783
- Niedermayr, Simon, "Compressed 3d gaussian splatting for accelerated novel view synthesis", Proceedings of the IEEE, 2024, 10349~10358
- Zhu, Shengjie, "LightedDepth : Video depth estimation in light of limited inference view angles", Proceedings of the IEEE, 2023, 5003~5012
- Wang, Zhou, "Image quality assessment : from error visibility to structural similarity", IEEE Transactions on Image Processing, pp.600~612 (2004)[DOI]
- Arno Knapitsch, "Tanks and temples : Benchmarking large-scale scene reconstruction", ACM Transactions on Graphics, 36(4) (2017)[DOI]