SPAD과 CNN의 특성을 반영한 ToF 센서와 스테레오카메라 융합 시스템
Fusion System of Time-of-Flight Sensor and Stereo Cameras Considering Single Photon Avalanche Diode and Convolutional Neural Network
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
3D depth perception has played an important role in robotics, and many sensory methods have also proposed for it. As a photodetector for 3D sensing, single photon avalanche diode (SPAD) is suggested due to sensitivity and accuracy. We have researched for applying a SPAD chip in our fusion system of time-of-fight (ToF) sensor and stereo camera. Our goal is to upsample of SPAD resolution using RGB stereo camera. Currently, we have 64 x 32 resolution SPAD ToF Sensor, even though there are higher resolution depth sensors such as Kinect V2 and Cube-Eye. This may be a weak point of our system, however we exploit this gap using a transition of idea. A convolution neural network (CNN) is designed to upsample our low resolution depth map using the data of the higher resolution depth as label data. Then, the upsampled depth data using CNN and stereo camera depth data are fused using semi-global matching (SGM) algorithm. We proposed simplified fusion method created for the embedded system.
Abstract
3D depth perception has played an important role in robotics, and many sensory methods have also proposed for it. As a photodetector for 3D sensing, single photon avalanche diode (SPAD) is suggested due to sensitivity and accuracy. We have researched for applying a SPAD chip in our fusion system of time-of-fight (ToF) sensor and stereo camera. Our goal is to upsample of SPAD resolution using RGB stereo camera. Currently, we have 64 x 32 resolution SPAD ToF Sensor, even though there are higher resolution depth sensors such as Kinect V2 and Cube-Eye. This may be a weak point of our system, however we exploit this gap using a transition of idea. A convolution neural network (CNN) is designed to upsample our low resolution depth map using the data of the higher resolution depth as label data. Then, the upsampled depth data using CNN and stereo camera depth data are fused using semi-global matching (SGM) algorithm. We proposed simplified fusion method created for the embedded system.
참고문헌 (19)
- 김성환, "무인 배송용 드론 설계 및 시험비행", 제어.로봇.시스템학회 논문지, 22(3), pp.204~209 (2016), 제어·로봇·시스템학회[DOI]
- 조성욱, "물류수송을 위한 이종 협업 무인 시스템 개발", 제어.로봇.시스템학회 논문지, 20(12), pp.1181~1188 (2014), 제어·로봇·시스템학회[DOI]
- V. Magnanimo, "Safeguarding a mobile manipulator using dynamic safety fields", 2016 IEEE, 2016, 2972~2977
- 문준호, "데이터베이스 기반 GPS 위치 보정 시스템", 로봇학회 논문지, 7(3), pp.205~215 (2012), 한국로봇학회[DOI]
- 송창호, "다양한 환경에서 강건한 RGB-Depth-Thermal 카메라 기반의 차량 탑승자 점유 검출", 로봇학회 논문지, 13(1), pp.31~37 (2018), 한국로봇학회[DOI]
- D. Shin, "Structure variation effects on device reliability of single photon avalanche diodes", Microelectronics Reliability, 76-77, pp.610~613 (2014)
- Y. Choi, "Thermal image enhancement using convolutional neural network", 2016IEEE, 2016, 223~230
- S. Wang, "Accelerating magnetic resonance imaging via deep learning", 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp.514~517 (2016)
- B. Zhu, "Image reconstruction by domain-transform manifold learning", Nature, 555, pp.487~492 (2018)
- J. Park, "High quality depth map upsampling for 3d-tof cameras", 2011International Conference on Computer Vision, pp.1623~1630 (2011)
- R. Nair, "High accuracy tof and stereo sensor fusion at interactive rates", European Conference on Computer Vision, pp.1~11 (2012)
- G. Agresti, "Deep learning for confidence information in stereo and TOF data fusion", 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp.697~705 (2017)
- Z. Zhang, "A flexible new technique for camera calibration", IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), pp.1330~1334 (2000)
- J. M. Lee, "A study on heterogeneous sensor calibration between ToF sensor and RGB camera", Korean Society for Precision Engineering Spring Conference, pp.108~109 (2016)
- D. Y. Kim, "System and method for concurrent calibration of camera and depth sensor", Korea Patent, Application No. 10-2015-0095091, ~ (2015)
- D. Y. Kim, "System and method for 3-dimensional calibration of camera and depth sensor, and calibration device and calibration jig thereof", Korea Patent, Registration No. 10-1803705, ~ (2017)
- D. Hernandez-Juarez, "Embedded real-time stereo estimation via semi-global matching on the GPU", Procedia Computer Science, 80, pp.143~153 (2016)
- C. Dong, "Accelerating the super-resolution convolutional neural network", European Conference on Computer Vision, pp.391~407 (2016)