데이터 프리패치를 이용한 하드웨어 가속기의 병목 현상 해결
Resolving Memory Bottlenecks in Hardware Accelerators with Data Prefetch
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
최근 다양한 분야에서 딥러닝이 사용되면서, 더 빠르고 정확한 결과를 내는 딥러닝이 더욱 중요해졌다. 이를 위해서는 많은 양의 저장 공간이 필요하고, 대용량 연산을 진행해야 한다. 이에따라 여러 연구는 빠르고 정확하게 연산 처리가 가능한 하드웨어 가속기를 이용한다. 하지만 하드웨어 가속기는 CPU와 하드웨어 사이를 이동하면서 병목현상이 발생하게 된다. 따라서 본 논문에서는 하드웨어 가속기의 병목현상을 효율적으로 줄일 수 있는 데이터 프리패치 전략을 제안한다. 데이터 프리패치 전략의 핵심 아이디어는 Matrix Multiplication Unit(MMU)가 연산을 진행하는동안 다음 연산에 필요한 데이터를 예측하여 로컬 메모리로 올려 병목현상을 줄인다. 또한, 이 전략은 듀얼 버퍼를 이용하여 읽고 쓰는 두 가지 동작을 동시에 진행하여 처리율을 높인다. 이를통해 데이터 전송의 지연시간 및 실행 시간을 감소시킨다. 시뮬레이션을 통해 듀얼 버퍼를 이용한 병렬 프로세싱과 데이터 프리패치를 이용한 메모리 간 병목현상을 최대한 감소시켜 하드웨어가속기의 성능이 24% 향상함을 알 수 있다.
Abstract
Deep learning with faster and more accurate results requires large amounts of storage space and large computations. Accordingly, many studies are using hardware accelerators for quick and accurate calculations. However, the performance bottleneck is due to data movement between the hardware accelerators and the CPU. In this paper, we propose a data prefetch strategy that can efficiently reduce such operational bottlenecks. The core idea of the data prefetch strategy is to predict the data needed for the next task and upload it to local memory while the hardware accelerator (Matrix Multiplication Unit, MMU) performs a task. This strategy can be enhanced by using a dual buffer to perform read and write operations simultaneously. This reduces latency and execution time of data transfer. Through simulations, we demonstrate a 24% improvement in the performance of hardware accelerators by maximizing parallel processing with dual buffers and bottlenecks between memories with data prefetch.
참고문헌 (18)
- Y. LeCun, "Deep Learning", Nature, 521(7553), pp.436~444 (2015)[DOI]
- Zou, Z., "Object detection in 20 years: A survey", Proceedings of the IEEE, 111(3), pp.257~276 (2023)
- Zhao, Zhong-Qiu, "Object detection with deep learning: A review", IEEE transactions on neural networks and learning systems, 30(11), pp.3212~3232 (2019)[DOI]
- C. Chen, "DeepDriving : Learning Affordance for Direct Perception in Autonomous Driving", 2015 IEEE International Conference on Computer Vision (ICCV), pp.2722~2730 (2015)
- Silver, David, "Mastering the game of Go with deep neural networks and tree search", nature, 529(7587), pp.484~489 (2016)[DOI]
- Krizhevsky, Alex, "ImageNet classification with deep convolutional neural networks", Communications of the ACM, 60(6), pp.84~90 (2017)[DOI]
- Struharik, Rastislav JR, "CoNNa–Hardware accelerator for compressed convolutional neural networks", Microprocessors and Microsystems, 73 (2020)[DOI]
- Chen, Tao, "Efficient data supply for hardware accelerators with prefetching and access", execute decoupling, ACM International Symposium on Microarchitecture (MICRO), pp.2016[DOI]
- Chen, Yunji, "DianNao family : energy-efficient hardware accelerators for machine learning", Communications of the ACM, 59(11), pp.105~112 (2016)[DOI]
- Jouppi, Norman P., "In-datacenter performance analysis of a tensor processing unit", Proceedings of the 44th annual international symposium on computer architecture (2017)
- Park, Jong-Hyeon, "Hardware Accelerator for Artificial Neural Network Operations Latest Research Trends", Communications of the Korean Institute of Information Scientists and Engineers, 34(9), pp.21~26 (2016)
- Oyama, Yosuke, "Accelerating deep learning frameworks with micro-batches", 2018 IEEE International Conference on Cluster Computing (CLUSTER) (2018), IEEE
- JUN, Hong shin, "Hbm (high bandwidth memory) dram technology and architecture", 2017 IEEE International Memory Workshop (IMW), pp.1~4 (2017), IEEE
- Marchisio, Alberto, "Capsacc: An efficient hardware accelerator for capsulenets with data reuse", 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE) (2019), IEEE
- Han, Song, "EIE: Efficient inference engine on compressed deep neural network", ACM SIGARCH Computer Architecture News, 44(3), pp.243~254 (2016)[DOI]
- Hyo-chan Lee, "The Design of Hardware Accelerator for Binary Segmentation Algorithm for Object Detection in Maritime", Journal of the Korea Institute Of Information and Communication Engineering, 24(1), pp.157~160 (2020)[DOI]
- Chen, Y. H., "Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices", IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(2), pp.292~308 (2019)[DOI]
- Chi, Ping, "Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory", ACM SIGARCH Computer Architecture News, 44(3), pp.27~39 (2016)[DOI]