Cycle-accurate NPU 시뮬레이터 및 데이터 접근 방식에 따른 NPU 성능평가
Cycle-accurate NPU Simulator and Performance Evaluation According to Data Access Strategies
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Currently, there are increasing demands for applying deep neural networks (DNNs) in the embedded domain such as classification and object detection. The DNN processing in embedded domain often requires custom hardware such as NPU for acceleration due to the constraints in power, performance, and area. Processing DNN models requires a large amount of data, and its seamless transfer to NPU is crucial for performance. In this paper, we developed a cycle-accurate NPU simulator to evaluate diverse NPU microarchitectures. In addition, we propose a novel technique for reducing the number of memory accesses when processing convolutional layers in convolutional neural networks (CNNs) on the NPU. The main idea is to reuse data with memory interleaving, which recycles the overlapping data between previous and current input windows. Data memory interleaving makes it possible to quickly read consecutive data in unaligned locations. We implemented the proposed technique to the cycle-accurate NPU simulator and measured the performance with LeNet-5, VGGNet-16, and ResNet-50. The experiment shows up to 2.08x speedup in processing one convolutional layer, compared to the baseline.
Abstract
Currently, there are increasing demands for applying deep neural networks (DNNs) in the embedded domain such as classification and object detection. The DNN processing in embedded domain often requires custom hardware such as NPU for acceleration due to the constraints in power, performance, and area. Processing DNN models requires a large amount of data, and its seamless transfer to NPU is crucial for performance. In this paper, we developed a cycle-accurate NPU simulator to evaluate diverse NPU microarchitectures. In addition, we propose a novel technique for reducing the number of memory accesses when processing convolutional layers in convolutional neural networks (CNNs) on the NPU. The main idea is to reuse data with memory interleaving, which recycles the overlapping data between previous and current input windows. Data memory interleaving makes it possible to quickly read consecutive data in unaligned locations. We implemented the proposed technique to the cycle-accurate NPU simulator and measured the performance with LeNet-5, VGGNet-16, and ResNet-50. The experiment shows up to 2.08x speedup in processing one convolutional layer, compared to the baseline.
참고문헌 (20)
- H. Esmaeilzadeh, "Neural Acceleration for General-purpose Approximate Programs", 2012 45th Annual IEEE, 012, 449~460
- Y. Chen, "A Survey of Accelerator Architectures for Deep Neural Networks", Engineering, 6(3), pp.264~274 (2020)
- A. Skillman, "A Technical Overview of Cortex-m55 and Ethos-u55: Arm’s most Capable Processors for Endpoint ai", 2020 IEEE Hot Chips 32Symposium (HCS), pp.1~20 (2020)
- J. Choquette, "NVIDIA A100 Tensor core GPU:Performance and Innovation", IEEE Micro, 41(2), pp.29~35 (2021)
- "ttp://deepx.musigndm.com/ product/"[URL]
- J. W. Jang, "Sparsity-aware and Re-configurable npu Architecture for Samsung Flagship Mobile soc", 2021ACM, 2021, 15~28
- Y. Wang, "PL-NPU:An Energy-Efficient Edge-Device DNN Training Processor With Posit-Based Logarithm-Domain Computing" (2022), IEEE Transactions on Circuits and Systems I: Regular Papers
- L. Lu, "Evaluating fast Algorithms for Convolutional Neural Networks on FPGAs", 2017 IEEE 25th Annual International Symposium on Field-programmable Custom Computing Machines (FCCM), pp.101~108 (2017)
- Y. H. Chen, "Eyeriss:An Energy-efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks", IEEE Journal of Solid-state Circuits, 52(1), pp.127~138 (2016)
- Y. H. Chen, "Eyeriss v2:A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices", IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(2), pp.292~308 (2019)
- K. Prabhu, "CHIMERA: A 0.92-TOPS, 2.2-TOPS", W Edge AI Accelerator With 2-MByte On-Chip Foundry Resistive RAM for Efficient Training and Inference, 2022, pp.57(4), 1013~1026
- "IEEE Signal Processing Magazine", 2012, 141~142
- J. Deng, "Imagenet: A Large-scale Hierarchical Image Database", 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.248~255 (2009)
- "LeNet-5, Convolutional Neural Networks"[URL]
- K. Simonyan, "Very Deep Convolutional Networks for Large-scale Image Recognition", arXiv preprint arXiv:1409.1556, ~ (2014)
- K. He, "Deep Residual Learning for Image Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770~778 (2016)
- H. Wu, "Integer Quantization for Deep Learning Inference:Principles and Empirical Evaluation", arXiv preprint arXiv:2004.09602, ~ (2020)