멀티 모달 지도 대조 학습을 이용한 농작물 병해 진단 예측 방법
Multimodal Supervised Contrastive Learning for Crop Disease Diagnosis
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
With the wide spread of smart farms and the advancements in IoT technology, it is easy to obtain additional data in addition to crop images. Consequently, deep learning-based crop disease diagnosis research utilizing multimodal data has become important. This study proposes a crop disease diagnosis method using multimodal supervised contrastive learning by expanding upon the multimodal self-supervised learning. RandAugment method was used to augment crop image and time series of environment data. These augmented data passed through encoder and projection head for each modality, yielding low-dimensional features. Subsequently, the proposed multimodal supervised contrastive loss helped features from the same class get closer while pushing apart those from different classes. Following this, the pretrained model was fine-tuned for crop disease diagnosis. The visualization of t-SNE result and comparative assessments of crop disease diagnosis performance substantiate that the proposed method has superior performance than multimodal self-supervised learning.
Abstract
With the wide spread of smart farms and the advancements in IoT technology, it is easy to obtain additional data in addition to crop images. Consequently, deep learning-based crop disease diagnosis research utilizing multimodal data has become important. This study proposes a crop disease diagnosis method using multimodal supervised contrastive learning by expanding upon the multimodal self-supervised learning. RandAugment method was used to augment crop image and time series of environment data. These augmented data passed through encoder and projection head for each modality, yielding low-dimensional features. Subsequently, the proposed multimodal supervised contrastive loss helped features from the same class get closer while pushing apart those from different classes. Following this, the pretrained model was fine-tuned for crop disease diagnosis. The visualization of t-SNE result and comparative assessments of crop disease diagnosis performance substantiate that the proposed method has superior performance than multimodal self-supervised learning.
참고문헌 (21)
- K. P. Ferentinos, "Deep Learning Models for Plant Disease Detection and Diagnosis", Computers and Electronics in Agriculture, 145, pp.311~318 (2018)
- J. A. Pandian, "Plant Disease Detection Using Deep Convolutional Neural Network", Applied Sciences, 12(14), pp.6982~ (2022)
- H. Yu, "Corn Leaf Diseases Diagnosis Based on K-means Clustering and Deep Learning", IEEE Access, 9, pp.143824~143835 (2021)
- 윤협상, "농작물 질병분류를 위한 전이학습에 사용되는 기초 합성곱신경망 모델간 성능 비교", 한국산업경영시스템학회지, 44(3), pp.33~38 (2021), 한국산업경영시스템학회
- J. Li, "An Improved YOLOv5-based Vegetable Disease Detection Method", Computers and Electronics in Agriculture, 202, pp.107345~ (2022)
- L. Alves, "System to Predict Diseases in Vineyards and Olive Groves using Data Mining and Geolocation", ICSOFT (2018)
- S. Sannakki, "A Neural Network Approach for Disease Forecasting in Grapes Using Weather Parameters", 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT). IEEE, pp.1~5 (2013)
- J. Deng, "Imagenet : A Large-scale Hierarchical Image Database", 2009 IEEE Conference on Computer Vision and Pattern Recognition. Ieee, pp.248~255 (2009)
- J. Ngiam, "Multimodal Deep Learning", Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp.689~696 (2011)
- J. Venugopalan, "Multimodal Deep Learning Models for Early Detection of Alzheimer’s Disease Stage", Scientific reports, 11(1), pp.3254~ (2021)
- W. Liu, "Emotion Recognition Using Multimodal Deep Learning", Neural Information Processing: 23rd International Conference, ICONIP 2016, (Part II 23), pp.521~529 (2016), Springer International Publishing
- T. G. Kim, "A Multimodal Deep Learning Method for Android Malware Detection Using Various Features", IEEE Transactions on Information Forensics and Security, 14(3), pp.773~788 (2018)
- T. Chen, "A Simple Framework for Contrastive Learning of Visual Representations", International Conference on Machine Learning. PMLR, pp.1597~1607 (2020)
- J. Zhang, "A Class-aware Supervised Contrastive Learning Framework for Imbalanced Fault Diagnosis", Knowledge-Based Systems, 252, pp.109437~ (2022)
- A. Taleb, "Contig : Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics", Proceedings of the IEEE, 2022, 20908~20921
- P. Hager, "Best of Both Worlds : Multimodal Contrastive Learning with Tabular and Imaging Data", Proceedings of the IEEE, 2023, 23924~23935
- P. Khosla, "Supervised Contrastive Learning", Advances in Neural Information Processing Systems, 33, pp.18661~18673 (2020)
- "DACON AI Challenge Website"[URL]
- E. D. Cubuk, "Randaugment : Practical Automated Data Augmentation with a Reduced Search Space", Proceedings of the IEEE, 2020, 702~703
- K. He, "Deep Residual Learning for Image Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
- B. K. Iwana, "An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks", Plos one, 16(7), pp.e0254841~ (2021)