FPGA기반 뉴럴네트워크 가속기에서 2차 타일링 기반 행렬 곱셈 최적화
Optimizing 2-stage Tiling-based Matrix Multiplication in FPGA-based Neural Network Accelerator
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The acceleration of neural networks has become an important topic in the field of computer vision. An accelerator is absolutely necessary for accelerating the lightweight model. Most accelerator-supported operators focused on direct convolution operations. If the accelerator does not provide GEMM operation, it is mostly replaced by CPU operation. In this paper, we proposed an optimization technique for 2-stage tiling-based GEMM routines on VTA. We improved performance of the matrix multiplication routine by maximizing the reusability of the input matrix and optimizing the operation pipelining. In addition, we applied the proposed technique to the DarkNet framework to check the performance improvement of the matrix multiplication routine. The proposed GEMM method showed a performance improvement of more than 2.4 times compared to the non-optimized GEMM method. The inference performance of our DarkNet framework has also improved by at least 2.3 times.
Abstract
The acceleration of neural networks has become an important topic in the field of computer vision. An accelerator is absolutely necessary for accelerating the lightweight model. Most accelerator-supported operators focused on direct convolution operations. If the accelerator does not provide GEMM operation, it is mostly replaced by CPU operation. In this paper, we proposed an optimization technique for 2-stage tiling-based GEMM routines on VTA. We improved performance of the matrix multiplication routine by maximizing the reusability of the input matrix and optimizing the operation pipelining. In addition, we applied the proposed technique to the DarkNet framework to check the performance improvement of the matrix multiplication routine. The proposed GEMM method showed a performance improvement of more than 2.4 times compared to the non-optimized GEMM method. The inference performance of our DarkNet framework has also improved by at least 2.3 times.
참고문헌 (25)
- 배주원, "엣지 컴퓨팅 환경에서 적용 가능한 딥러닝 기반 라벨 검사 시스템 구현", 대한임베디드공학회논문지, 17(2), pp.77~83 (2022), 대한임베디드공학회[DOI]
- 최재협, "수술 도구의 세분화와 행동 인식 기능이 탑재된 AI 서비스 개발", 대한임베디드공학회논문지, 16(2), pp.51~57 (2021), 대한임베디드공학회[DOI]
- 김현돈, "임베디드 연산을 위한 잡음에서 음성추출 U-Net 설계", 대한임베디드공학회논문지, 15(5), pp.227~234 (2020), 대한임베디드공학회[DOI]
- 김현진, "Analysis of Reduced-Width Truncated Mitchell Multiplication for Inferences Using CNNs", 대한임베디드공학회논문지, 15(5), pp.235~242 (2020), 대한임베디드공학회[DOI]
- J. M. Lee, "Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for fast Deployment", Future Generation Computer Systems, 132, pp.124~135 (2022)
- 권구윤, "Cycle-accurate NPU 시뮬레이터 및 데이터 접근 방식에 따른 NPU 성능평가", 대한임베디드공학회논문지, 17(4), pp.217~228 (2022), 대한임베디드공학회[DOI]
- T. Moreau, "A Hardware–software Blueprint for Flexible Deep Learning Specialization", IEEE Micro, 39(5), pp.8~16 (2019)
- S. Chetlur, "cudnn : Efficient Primitives for Deep Learning", arXiv preprint arXiv : 1410. 0759, ~ (2014)
- E. Wang, "High-Performance Computing on the Intel® Xeon Phi ™", pp.167~188 (2014), Springer
- K. He, "Deep Residual Learning for Image Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
- A. Anderson, "Low-memory Gemm-based Convolution Algorithms for Deep Neural Networks", arXiv preprint arXiv : 1709. 03395, ~ (2017)
- J. S. Park, "mGEMM : Low-latency Convolution with Minimal Memory Overhead Optimized for Mobile Devices", Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services, pp.222~234 (2022)
- M. S. Cho, "MEC: Memory-efficient Convolution for Deep Neural Network", International Conference on Machine Learning. PMLR, pp.815~824 (2017)
- M. Dukhan, "The Indirect Convolution Algorithm", arXiv preprint arXiv : 1907. 02129, ~ (2019)
- C. Nugteren, "CLBlast: A Tuned OpenCL BLAS Library", Proceedings of the International Workshop on OpenCL, pp.1~10 (2018)
- K. Goto, "High-performance Implementation of the Level-3 BLAS", ACM Transactions on Mathematical Software (TOMS), 35(1), pp.1~14 (2008)