New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
On-chip caches of graphics processing units (GPUs) have contributed to improved GPU performance byreducing long memory access latency. However, cache efficiency remains low despite the facts that recentGPUs have considerably mitigated the bottleneck problem of L1 data cache. Although the cache miss rate is areasonable metric for cache efficiency, it is not necessarily proportional to GPU performance. In this study, weintroduce a second key determinant to overcome the problem of predicting the performance gains from L1 datacache based on the assumption that miss rate only is not accurate. The proposed technique estimates the benefitsof the cache by measuring the balance between cache efficiency and throughput. The throughput of the cacheis predicted based on the warp occupancy information in the warp pool. Then, the warp occupancy is used fora second bypass phase when workloads show an ambiguous miss rate. In our proposed architecture, the L1 datacache is turned off for a long period when the warp occupancy is not high. Our two-level bypassing techniquecan be applied to recent GPU models and improves the performance by 6% on average compared to thearchitecture without bypassing. Moreover, it outperforms the conventional bottleneck-based bypassingtechniques.
Abstract
On-chip caches of graphics processing units (GPUs) have contributed to improved GPU performance byreducing long memory access latency. However, cache efficiency remains low despite the facts that recentGPUs have considerably mitigated the bottleneck problem of L1 data cache. Although the cache miss rate is areasonable metric for cache efficiency, it is not necessarily proportional to GPU performance. In this study, weintroduce a second key determinant to overcome the problem of predicting the performance gains from L1 datacache based on the assumption that miss rate only is not accurate. The proposed technique estimates the benefitsof the cache by measuring the balance between cache efficiency and throughput. The throughput of the cacheis predicted based on the warp occupancy information in the warp pool. Then, the warp occupancy is used fora second bypass phase when workloads show an ambiguous miss rate. In our proposed architecture, the L1 datacache is turned off for a long period when the warp occupancy is not high. Our two-level bypassing techniquecan be applied to recent GPU models and improves the performance by 6% on average compared to thearchitecture without bypassing. Moreover, it outperforms the conventional bottleneck-based bypassingtechniques.
참고문헌 (17)
- W. Jia, "MRPB: memory request prioritization for massively parallel processors", Proceedings of 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA), pp.272~283 (2014)
- "NVIDIA Tesla P100: GP100 Pascal Architecture" (2016)[URL]
- C. T. Do, "Application characteristics-aware sporadic cache bypassing for high performance GPGPUs", Journal of Parallel and Distributed Computing, 122, pp.238~250 (2018)
- J. Zhang, "Memory-aware TLP throttling and cache bypassing for GPUs", Cluster Computing, 22(1), pp.871~883 (2019)
- "NVIDA Tesla V100 GPU architecture" (2017)[URL]
- M. Gebhart, "Unifying primary cache, scratch, and register file memories in a throughput processor", Proceedings of 2012 45th Annual IEEE, 2012, 96~106
- X. Xie, "Coordinated static and dynamic cache bypassing for GPUs", Proceedings of 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), pp.76~88 (2015)
- C. T. Do, "Early miss prediction based periodic cache bypassing for high performance GPUs", Microprocessors and Microsystems, 55, pp.44~54 (2017)
- X. Chen, "Adaptive cache management for energy-efficient GPU computing", Proceedings of 2014 47th Annual IEEE, 2014, 343~355
- A. Sethia, "Mascar: speeding up GPU warps by reducing memory pitstops", Proceedings of 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), pp.174~185 (2015)
- G. Koo, "Access pattern-aware cache management for improving data utilization in GPU", Proceedings of the 44th Annual International Symposium on Computer Architecture, pp.307~319 (2017)
- J. Fang, "Miss-aware LLC buffer management strategy based on heterogeneous multi-core", The Journal of Supercomputing, 75(8), pp.4519~4528 (2019)
- M. Khairy, "A detailed model for contemporary GPU memory systems", Proceedings of 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp.141~142 (2019)
- "NVIDIA GeForce GTX 1080" (2016)[URL]
- "Dissecting the NVIDIA Volta GPU architecture via microbenchmarking" (2018)[URL]
- "Is data placement optimization still relevant on newer GPUs?" (2018)[URL]
- A. Karki, "Tango: a deep neural network benchmark suite for various accelerators", Proceedings of 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp.137~138 (2019)