비트 분리 가능한 곱셈기를 활용한 인공지능 가속기의 출력 활성화 희소성 최적화
Optimization of Output Activation Sparsity in AI AcceleratorUsing Bit-Separable Multiplier
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Artificial intelligence (AI) has become pervasive in enhancing human productivity and has evolved significantly based on specific areas of interest. The strength of AI lies in its computational speed and versatility, surpassing human capabilities, due to the redundancy of parameters in AI models. However, this redundancy imposes constraints on computational power and memory bandwidth in the hardware that processes AI. Consequently, edge devices face latency issues, making it challenging to operate AI models independently without cloud assistance. We propose a novel approach that utilizes output activation sparsity for efficient computing and memory usage. By implementing a simple, flexible structure with a bit-separable multiplier that splits the multiplier structure in half and a dynamic range decoder to interpret output range, we optimize AI accelerators to skip computations when zero activations are predicted in output activation values. This method is applicable to state-of-the-art models, including convolutional neural networks (CNNs) and Transformers. Simulation results from ASIC-synthesized accelerators demonstrated a 46.9% improvement in speed and a 68% reduction in power consumption. Additionally, memory access was reduced by an average of 25.5% in CNN models and 41.5% in Transformer models.
Abstract
Artificial intelligence (AI) has become pervasive in enhancing human productivity and has evolved significantly based on specific areas of interest. The strength of AI lies in its computational speed and versatility, surpassing human capabilities, due to the redundancy of parameters in AI models. However, this redundancy imposes constraints on computational power and memory bandwidth in the hardware that processes AI. Consequently, edge devices face latency issues, making it challenging to operate AI models independently without cloud assistance. We propose a novel approach that utilizes output activation sparsity for efficient computing and memory usage. By implementing a simple, flexible structure with a bit-separable multiplier that splits the multiplier structure in half and a dynamic range decoder to interpret output range, we optimize AI accelerators to skip computations when zero activations are predicted in output activation values. This method is applicable to state-of-the-art models, including convolutional neural networks (CNNs) and Transformers. Simulation results from ASIC-synthesized accelerators demonstrated a 46.9% improvement in speed and a 68% reduction in power consumption. Additionally, memory access was reduced by an average of 25.5% in CNN models and 41.5% in Transformer models.