임베디드 엣지 플랫폼에서의 경량 비전 트랜스포머 성능 평가
Performance Evaluation of Efficient Vision Transformers on Embedded Edge Platforms
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Recently, on-device artificial intelligence (AI) solutions using mobile devices and embedded edge devices have emerged in various fields, such as computer vision, to address network traffic burdens, low-energy operations, and security problems. Although vision transformer deep learning models have outperformed conventional convolutional neural network (CNN) models in computer vision, they require more computations and parameters than CNN models. Thus, they are not directly applicable to embedded edge devices with limited hardware resources. Many researchers have proposed various model compression methods or lightweight architectures for vision transformers; however, there are only a few studies evaluating the effects of model compression techniques of vision transformers on performance. Regarding this problem, this paper presents a performance evaluation of vision transformers on embedded platforms. We investigated the behaviors of three vision transformers: DeiT, LeViT, and MobileViT. Each model performance was evaluated by accuracy and inference time on edge devices using the ImageNet dataset. We assessed the effects of the quantization method applied to the models on latency enhancement and accuracy degradation by profiling the proportion of response time occupied by major operations. In addition, we evaluated the performance of each model on GPU and EdgeTPU-based edge devices. In our experimental results, LeViT showed the best performance in CPU-based edge devices, and DeiT-small showed the highest performance improvement in GPU-based edge devices. In addition, only MobileViT models showed performance improvement on EdgeTPU. Summarizing the analysis results through profiling, the degree of performance improvement of each vision transformer model was highly dependent on the proportion of parts that could be optimized in the target edge device. In summary, to apply vision transformers to on-device AI solutions, either proper operation composition and optimizations specific to target edge devices must be considered.
Abstract
Recently, on-device artificial intelligence (AI) solutions using mobile devices and embedded edge devices have emerged in various fields, such as computer vision, to address network traffic burdens, low-energy operations, and security problems. Although vision transformer deep learning models have outperformed conventional convolutional neural network (CNN) models in computer vision, they require more computations and parameters than CNN models. Thus, they are not directly applicable to embedded edge devices with limited hardware resources. Many researchers have proposed various model compression methods or lightweight architectures for vision transformers; however, there are only a few studies evaluating the effects of model compression techniques of vision transformers on performance. Regarding this problem, this paper presents a performance evaluation of vision transformers on embedded platforms. We investigated the behaviors of three vision transformers: DeiT, LeViT, and MobileViT. Each model performance was evaluated by accuracy and inference time on edge devices using the ImageNet dataset. We assessed the effects of the quantization method applied to the models on latency enhancement and accuracy degradation by profiling the proportion of response time occupied by major operations. In addition, we evaluated the performance of each model on GPU and EdgeTPU-based edge devices. In our experimental results, LeViT showed the best performance in CPU-based edge devices, and DeiT-small showed the highest performance improvement in GPU-based edge devices. In addition, only MobileViT models showed performance improvement on EdgeTPU. Summarizing the analysis results through profiling, the degree of performance improvement of each vision transformer model was highly dependent on the proportion of parts that could be optimized in the target edge device. In summary, to apply vision transformers to on-device AI solutions, either proper operation composition and optimizations specific to target edge devices must be considered.
참고문헌 (46)
- W. Ahmad, "Cyber Security in Iot-based Cloud Computing: A Comprehensive Survey", Electronics, 11(1), pp.16~ (2022)
- M. Ham, "NNStreamer: Efficient and Agile Development of On-Device AI Systems", Proc. of the IEEE, 2021, 198~207
- S. Dhar, "A Survey of On-device Machine Learning: An Algorithms and Learning Theory Perspective", ACM Transactions on Internet of Things, 2(3), pp.1~49 (2021)
- D. Kong, "Science Driven Innovations Powering Mobile Product: Cloud AI vs. Device AI Solutions on Smart Device", ~ (2017)
- "Nvidia Embedded Systems for Next-Gen Autonomous Machines"[URL]
- "Edge TPU - run Inference at the Edge | Google Cloud"[URL]
- W. Vijitkunsawat, "Comparison of Machine Learning Algorithm’s on Self-driving Car Navigation Using Nvidia Jetson Nano", Proc. of the International Conference on Electrical Engineering, 2020, 201~204
- A. Basulto-Lantsova, "Performance comparative of OpenCV Template Matching method on Jetson TX2 and Jetson Nano Developer Kits", Proc. of the Annual Computing and Communication Workshop and Conference (CCWC), pp.0812~0816 (2020)
- K. Alibabaei, "Real-Time Detection of Vine Trunk for Robot Localization Using Deep Learning Models Developed for Edge TPU Devices", Future Internet, 14(7), pp.199~ (2022)
- Y. H. Tseng, "Combination of Computer Vision Detection and Segmentation for Autonomous Driving", Proc. of the IEEE, 2018, 1047~1052
- M. Bojarski, "End to end Learning for Self-driving Cars" (2016)
- D. N. N. Tran, "Object Detection with Deep Learning on Drive PX2", Proc. of the IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pp.1~4 (2020)
- K. He, "Deep Residual Learning for Image Recognition", Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
- M. Tan, "Efficientnet: Rethinking Model Scaling for Convolutional Neural Networks", Proc. of the International Conference on Machine Learning, pp.6105~6114 (2019)
- A. Howard, "Searching for Mobilenetv3", Proc. of the IEEE, 2019, 1314~1324
- A. Dosovitskiy, "An Image is Worth 16x16 Words:Transformers for Image Recognition at Scale", ~ (2020)
- M. Wortsman, "Model Soups: Averaging Weights of Multiple Fine-tuned Models Improves Accuracy Without Increasing Inference Time", Proc. of the International Conference on Machine Learning, pp.23965~23998 (2022)
- X. Zhai, "Scaling Vision Transformers", Proc. of the IEEE, 2022, 12104~12113
- H. Bao, "Beit: Bert Pre-training of Image Transformers", ~ (2021)
- Z. Liu, "Post-training Quantization for Vision Transformer", Advances in Neural Information Processing Systems, 34, pp.28092~28103 (2021)
- Y. Tang, "Patch Slimming for Efficient Vision Transformers", Proc. of the IEEE, 2022, 12165~12174
- L. Song, "Dynamic Grained Encoder for Vision Transformers", Advances in Neural Information Processing Systems, 34, pp.5770~5783 (2021)
- B. Chen, "Psvit: Better Vision Transformer Via Token Pooling and Attention Sharing", ~ (2021)
- B. Graham, "Levit: A Vision Transformer in Convnet’s Clothing for Faster Inference", Proc. of the IEEE, 2021, 12259~12269
- S. Mehta, "Mobilevit: Light-weight, General-purpose, and Mobile-friendly Vision Transformer", ~ (2021)
- O. Russakovsky, "Imagenet Large Scale Visual Recognition Challenge", International Journal of Computer Vision, 115, pp.211~252 (2015)
- H. Touvron, "Training Data-efficient Image Transformers & Distillation Through Attention", Proc. of the International Conference on Machine Learning, pp.10347~10357 (2021)
- B. Pan, "IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformers", Advances in Neural Information Processing Systems, 34, pp.24898~24911 (2021)
- A. Ignatov, "Ai Benchmark: Running Deep Neural Networks on Android Smartphones", Proc. of the European Conference on Computer Vision (ECCV)Workshops (2018)
- A. Ignatov, "Ai Benchmark:All About Deep Learning on Smartphones in 2019", Proc. of the IEEE, 2019, 3617~3635
- A. A. Süzen, "Benchmark Analysis of Jetson tx2, Jetson Nano and Raspberry pi Using Deep-cnn", Proc. of the International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp.1~5 (2020)
- P. Kang, "Benchmarking Modern Edge Devices for Ai Applications", IEICE TRANSACTIONS on Information and Systems, 104(3), pp.394~403 (2021)
- X. Wang, "Towards Efficient Vision Transformer Inference: A First Study of Transformers on Mobile Devices", Proc. of the Annual International Workshop on Mobile Computing Systems and Applications, pp.1~7 (2022)
- T. Xiao, "Early Convolutions Help Transformers See Better", Advances in Neural Information Processing Systems, 34, pp.30392~30400 (2021)
- M. Sandler, "Mobilenetv2: Inverted Residuals and Linear Bottlenecks", Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp.4510~4520 (2018)
- "Higher Accuracy on Vision Models with EfficientNet-Lite"[URL]
- R. Wightman, "PyTorch Image Models" (2019), GitHub
- "Raspberry pi 4 Model B Specifications"[URL]
- "CUDA Toolkit Documentation, CUDA Toolkit Documentation v12.0 - landing 12.0 documentation" (2022)[URL]
- "Jetson Nano Developer Kit" (2022)[URL]
- "EdgeTPU USB Accelerator"[URL]
- "Tensorflow"[URL]
- "Tensorflow Lite"[URL]
- "TensorFlow-TensorRT (TF-TRT)"[URL]
- "Edge Tpu Compiler"[URL]
- T. Sheng, "A Quantization-friendly Separable Convolution for Mobilenets", Proc. of the Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2), pp.14~18 (2018)