Multi-vessel target tracking with camera fusion for unmanned surface vehicles
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
With the decreasing availability of sailors, there has been an increasing focus on the development of autonomous ships. Among the various components of autonomous ships, automatic recognition systems that can replace human vision are a crucial area of research. While ongoing studies utilize traditional perception sensors such as RADAR (RAdio Detection And Ranging) and AIS (Automatic Identification System), they have limitations such as blind spots and a restricted detection range. To address these limitations, this paper proposes a new recognition method that utilizes multiple cameras, including electro-optical and infrared radiation cameras, to supplement traditional perception sensors. This method aims to detect maritime obstacles accurately and estimate their dynamic motion using a tracking process. Initially, real-sea images were collected for maritime obstacle detection, and a deep-learning-based detection model was trained on them. The detection results were then employed in an adaptive tracking filter, which allowed the precise motion estimation of the obstacles. Furthermore, to compensate for the limitations of using individual cameras as sensors, this study introduces the simultaneous fusion of tracked data from multiple cameras. This fusion process enhances tracking results in various ways. In field tests using multiple Unmanned Surface Vehicles (USVs), the proposed method successfully converged tracking results within the range of GPS errors. In addition, the fusion of tracked data from multiple cameras significantly improved the tracking results obtained from a single camera.
Abstract
With the decreasing availability of sailors, there has been an increasing focus on the development of autonomous ships. Among the various components of autonomous ships, automatic recognition systems that can replace human vision are a crucial area of research. While ongoing studies utilize traditional perception sensors such as RADAR (RAdio Detection And Ranging) and AIS (Automatic Identification System), they have limitations such as blind spots and a restricted detection range. To address these limitations, this paper proposes a new recognition method that utilizes multiple cameras, including electro-optical and infrared radiation cameras, to supplement traditional perception sensors. This method aims to detect maritime obstacles accurately and estimate their dynamic motion using a tracking process. Initially, real-sea images were collected for maritime obstacle detection, and a deep-learning-based detection model was trained on them. The detection results were then employed in an adaptive tracking filter, which allowed the precise motion estimation of the obstacles. Furthermore, to compensate for the limitations of using individual cameras as sensors, this study introduces the simultaneous fusion of tracked data from multiple cameras. This fusion process enhances tracking results in various ways. In field tests using multiple Unmanned Surface Vehicles (USVs), the proposed method successfully converged tracking results within the range of GPS errors. In addition, the fusion of tracked data from multiple cameras significantly improved the tracking results obtained from a single camera.
참고문헌 (26)
- Bewley, A., "Simple online and real-time tracking", Proceedings of the International Conference on Image Processing (2016)
- Bovcon, B., "MODS-a USV-oriented object detection and obstacle segmentation benchmark", IEEE Trans. Intell. Transport. Syst, 23(8), pp.13403~13418 (2021)
- Fr¨anken, D., "Improved fast covariance intersection for distributed data fusion", Proceedings of the 7th International Conference on Information Fusion (2005)
- Fu, C., "Visual tracking with online structural similaritybased weighted multiple instance learning", Inf. Sci, pp.481~ (2019)[DOI]
- Han, J., "Autonomous collision detection and avoidance for ARAGON USV : development and field tests", J. Field Robot, 37, pp.987~1002 (2020)[DOI]
- Han, Y. S., "Assessment of ship-block stacking status based on images obtained from drone and geographic information system data", Int. J. Nav. Archit, 16 (2024)[DOI]
- Hwang, I. K., "Wave height measurement scheme using wave detector based on convolutional neural network and PPM calculator with ocean wave images", Int. J. Nav. Archit, 15 (2023)[DOI]
- "YOLOv5" (2021)[URL]
- Kim, T., "Extended Kalman filter (Ekf) design for vehicle position tracking using reliability function of radar and LiDAR", Sensors, 20 (2020)[DOI]
- Kim, D. H., "CNN-based fire detection method on autonomous ships using composite channels composed of RGB and IR data", Int. J. Nav. Archit, 14 (2022)[DOI]
- Kim, J. H., "Prediction of the superiority of the hydrodynamic performance of hull forms using deep learning", Int. J. Nav. Archit, 14 (2022)[DOI]
- Lee, W. J., "Detection and tracking for the awareness of surroundings of a ship based on deep learning", J. Comput. Des. Eng, 8 (2021)[DOI]
- Liu, X., "Self-adaptive dynamic obstacle avoidance and path planning for USV under complex maritime environment", IEEE Access, 7, pp.114945~114954 (2019)
- Mitchell, H. B., "Multi-Sensor Data Fusion : an Introduction" (2007), Springer International Publishing
- Park, J. H., "Multi-video based obstacle detection and tracking method", Proceedings of the Annual Autumn Conference, pp.256~ (2021), SNAK
- Raj, T., "A survey on LiDAR scanning mechanisms", Electronics, 9(5), pp.741~ (2020)
- Redmon, J., "YOLOv3: an Incremental Improvement", ~ (2018)
- Rezaeianaran, F., "Seeking similarities over differences: similarity-based domain alignment for adaptive object detection", Proceedings of the IEEE International Conference on Computer Vision (2021)
- Song, L., "Interference intention classification of moving obstacles used for USV collision avoidance", Int. J. Nav. Archit, 14 (2022)[DOI]
- Wang, Q., "ECA-Net: efficient channel attention for deep convolutional neural networks", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2020)
- Wang, C. Y., "YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors", Proceedings of the IEEE, 2023, 7464~7475
- Woo, S., "CBAM: convolutional block attention module", Lect. Notes Comput. Sci (2018)[DOI]
- Yu, B. J., "Fender segmentation in unmanned aerial vehicle images based on densely connected receptive field block", Int. J. Nav. Archit, 14 (2022)[DOI]
- Zhang, W., "A object detection and tracking method for security in intelligence of unmanned surface vehicles", J. Ambient Intell. Hum. Comput, 13 (2022)[DOI]
- Zhu, L., "Improving YOLOv5 with attention mechanism for detecting boulders from planetary images", Rem. Sens, 13 (2021)[DOI]