작물 수확 자동화를 위한 시각 언어 모델 기반의 환경적응형 과수 검출 기술
Domain Adaptive Fruit Detection Method based on a Vision-Language Model for Harvest Automation
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Recently, mobile manipulators have been utilized in agriculture industry for weed removal and harvest automation. This paper proposes a domain adaptive fruit detection method for harvest automation, by utilizing OWL-ViT model which is an open-vocabulary object detection model. The vision-language model can detect objects based on text prompt, and therefore, it can be extended to detect objects of undefined categories. In the development of deep learning models for real-world problems, constructing a large-scale labeled dataset is a time-consuming task and heavily relies on human effort. To reduce the labor-intensive workload, we utilized a large-scale public dataset as a source domain data and employed a domain adaptation method. Adversarial learning was conducted between a domain discriminator and feature extractor to reduce the gap between the distribution of feature vectors from the source domain and our target domain data. We collected a target domain dataset in a real-like environment and conducted experiments to demonstrate the effectiveness of the proposed method. In experiments, the domain adaptation method improved the AP50 metric from 38.88% to 78.59% for detecting objects within the range of 2m, and we achieved 81.7% of manipulation success rate.
Abstract
Recently, mobile manipulators have been utilized in agriculture industry for weed removal and harvest automation. This paper proposes a domain adaptive fruit detection method for harvest automation, by utilizing OWL-ViT model which is an open-vocabulary object detection model. The vision-language model can detect objects based on text prompt, and therefore, it can be extended to detect objects of undefined categories. In the development of deep learning models for real-world problems, constructing a large-scale labeled dataset is a time-consuming task and heavily relies on human effort. To reduce the labor-intensive workload, we utilized a large-scale public dataset as a source domain data and employed a domain adaptation method. Adversarial learning was conducted between a domain discriminator and feature extractor to reduce the gap between the distribution of feature vectors from the source domain and our target domain data. We collected a target domain dataset in a real-like environment and conducted experiments to demonstrate the effectiveness of the proposed method. In experiments, the domain adaptation method improved the AP50 metric from 38.88% to 78.59% for detecting objects within the range of 2m, and we achieved 81.7% of manipulation success rate.
참고문헌 (18)
- D. R. Vincent, "Sensors Driven AI-based Agriculture Recommendation Model for Assessing Land Suitability", Sensors, 19(17), pp.3667~ (2019)
- M. D. Bah, "Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images", Sensors, 10(11), pp.1690~ (2018)
- L. Li, "Plant Disease Detection and Classification by Deep Learning—A Review", IEEE Access, 9, pp.56683~56698 (2021)
- Y. Onishi, "An Automated Fruit Harvesting Robot by Using Deep Learning", Robomech Journal, 6(1), pp.1~8 (2019)
- 남창우, "작물 수확 자동화를 위한 환경적응형 과수 검출 알고리즘", 2023 대한임베디드공학회 추계학술대회 (2023)
- J. Redmon, "You Only Look Oonce: Unified, Real-time Object Detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.779~788 (2016)
- N. Carion, "End-to-end Object Detection with Transformers", Proceedings of the European Conference on Computer Vision, pp.213~229 (2020)
- A. Radford, "Learning Transferable Visual Models From Natural Language Supervision", International Conference on Machine Learning, pp.8748~8763 (2021), PMLR
- M. Minderer, "Simple Open-Vocabulary Object Detection with Vision Transformers", European Conference on Computer Vision, pp.728~755 (2022)
- N. Häni, "MinneApple : a Benchmark Dataset for Apple Detection and Segmentation", IEEE Robotics and Automation Letters, 5(2), pp.852~858 (2020)
- "Sugar Content-Quality Data of Apple in Jeonbuk Jangsu"[URL]
- P. Chu, "Deep Learning-based Apple Detection Using a Suppression Mask R-CNN", Pattern Recognition Letters, 147, pp.206~211 (2021)
- Y. Ganin, "Domain-Adversarial Training of Neural Networks", The Journal of Machine Learning Research, 17(59), pp.1~35 (2016)
- E. Tzeng, "Adversarial Discriminative Domain Adaptation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.7167~7176 (2017)
- B. T. Polyak, "Acceleration of Stochastic Approximation by Averaging", SIAM Journal on Control and Optimization, 30(4), pp.838~855 (1992)