제한된 의료 데이터셋에서 효과적인 트랜스포머와 합성곱 신경망 융합
Effective Fusion of Transformer and Convolutional Neural Networks on Limited Medical Datasets
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
최근 인공지능 연구에서 트랜스포머 모델이 주로 사용되고 있다. 트랜스포머 구조는 합성곱 신경망에 비해 더 많은 가중치를 학습시킬 수 있으므로 다양한 연구에서 기본 구조로 사용되고 있다. 그러나 트랜스포머는 대규모 데이터를 활용하여 학습해야 한다는 한계가 있다. 그로 인해 데이터가 한정적인 의료 분야에서는 트랜스포머 구조 적용에 어려움이 있다. 따라서, 본 연구에서는 제한된 의료 데이터셋으로도 트랜스포머를 학습할 수 있는 융합 구조를 제안한다. 융합 구조는 트랜스포머 인코더와 합성곱 신경망 디코더를 결합함으로써, 학습 과정에서 트랜스포머 인코더가 안정적으로 수렴할 수 있게 하고, 영상 내의 전역적 특징과 지역적 특징을 모두 학습할 수 있게 한다. 또한, 빠르고 안정적인 영상 분할 학습을 위해 세 가지의 손실함수를 결합한 손실함수를 이용한다. 본 연구에서는 두 개의 대장내시경 용종 데이터셋을 사용하여 제안하는 구조의 우수성과 안정성을 검증한다. 융합 구조는 용종 분할과 같은 의료 영상 분할 외에도 의료 영상 분석, 의료 영상 내 질병 탐지 등 여러 분야에서 활용될 수 있을 것이라 조망된다.
Abstract
Transformer models have been increasingly used in artificial intelligence research. Transformer architectures are used as a basic structure in various research because they can learn more weights than convolutional neural networks. However, transformers have the limitation of requiring large datasets to learn. As a result, it is difficult to apply transformer architectures in medical fields where available datasets are limited. Therefore, this study proposes a fusion architecture that can train transformers even with limited medical datasets. The fusion architecture combines a transformer encoder and a convolutional neural network decoder, which allows the transformer encoder to converge stably and learn both global and local features in the image. In addition, a loss function that combines three loss functions is used for fast and stable image segmentation learning. In this study, the superiority and stability of the proposed architecture are validated using two colonoscopic polyp datasets. The fusion architecture is expected to be used in various fields, such as medical image segmentation, medical image analysis, and disease detection in medical images, in addition to polyp segmentation.
참고문헌 (27)
- Vaswani, Ashish, "Attention is all you need", Adv Neural Inf Process Syst, 30 (2017)
- Devlin, Jacob, "Bert: Pre-training of deep bidirectional transformers for language understanding" (2018)
- Brown, Tom, "Language models are few-shot learners", Adv Neural Inf Process Syst, 33, pp.1877~1901 (2020)
- Dosovitskiy, Alexey, "An image is worth 16x16 words: Transformers for image recognition at scale" (2020)
- J. Deng, "ImageNet: A Large-Scale Hierarchical Image Database", 2009 IEEE conference on computer vision and pattern recognition, 1-4, pp.248~255 (2009)
- C. Sun, "Revisiting Unreasonable Effectiveness of Data in Deep Learning Era", Proceedings of the IEEE international conference on computer vision, pp.843~852 (2017)
- Y. Lecun, "Gradient-based learning applied to document recognition", Proceedings of the IEEE, 86(11), pp.2278~2324 (1998)
- C. Y. Choi, "TECD: Combined Transform and Convolutional Neural Network for Polyp Segmentation", Proceedings. of 2023KING Spring Conference, pp.169~171 (2023)
- C. -Y. Choi, "Enhanced Polyp Segmentation with a Hybrid Approach Integrating Transformers and Convolutional Neural Networks", Proceedings of the 2023 International Conference on Research in Adaptive and Convergent Systems (2023)
- O. Ronneberger, "U-Net: Convolutional Networks for Biomedical Image Segmentation", Medical Image Computing and Computer-Assisted Intervention, Pt Iii, 9351, pp.234~241 (2015)
- T. T. Xiao, "Unified Perceptual Parsing for Scene Understanding", Proceedings of the European conference on computer vision, 11209, pp.432~448 (2018)
- T. Y. Lin, "Focal Loss for Dense Object Detection", Proceedings of the IEEE international conference on computer vision, pp.2999~3007 (2017)
- D. Jha, "Kvasir-SEG: A Segmented Polyp Dataset", Multimedia Modeling, 11962, pp.451~462 (2020)
- J. Bernal, "WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians", Comput Med Imag Grap, 43, pp.99~111 (2015)
- O. Oktay, "Attention U-Net : Learning where to look for the pancreas" (2018)
- Z. W. Zhou, "UNet plus plus: A Nested U-Net Architecture for Medical Image Segmentation", Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 11045, pp.3~11 (2018)
- F. I. Diakogiannis, "ResUNet-a : A deep learning framework for semantic segmentation of remotely sensed data", Isprs J Photogramm, 162, pp.94~114 (2020)
- F. Milletari, "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation", Int Conf 3d Vision, pp.565~571 (2016)
- S. H. Woo, "CBAM: Convolutional Block Attention Module", Computer Vision - Eccv 2018, 11211, pp.3~19 (2018)
- E. Z. Xie, "SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers", Adv Neural Inf Process Syst, 34 (2021)
- W. H. Wang, "Pyramid Vision Trans former: A Versatile Backbone for Dense Prediction without Convolutions", 2021 Ieee, 2021, 548~558
- Z. Liu, "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows", 2021 Ieee, 2021, 9992~10002
- Z. Liu, "Swin Transformer V2: Scaling Up Capacity and Resolution", 2022 Ieee, 2022, 11999~12009
- T. Y. Lin, "Feature Pyramid Networks for Object Detection", 30th Ieee Conference on Computer Vision and Pattern Recognition, pp.936~944 (2017)
- B. L. Zhou, "Semantic Understanding of Scenes Through the ADE20K Dataset", Int J Comput Vision, 127(3), pp.302~321 (2019)
- K. M. He, "Deep Residual Learning for Image Recognition", Proceedings of the IEEE international conference on computer vision, pp.770~778 (2016)
- D. P. Kingma, "Adam: A method for stochastic optimization" (2014)