통신 실패에 강인한 분산 뉴럴 네트워크 분할 및 추론 정확도 개선 기법
Communication Failure Resilient Improvement of Distributed Neural Network Partitioning and Inference Accuracy
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Recently, it is increasingly necessary to run high-end neural network applications with huge computation overhead on top of resource-constrained embedded systems, such as wearable devices. While the huge computational overhead can be alleviated by distributed neural networks running on multiple separate devices, existing distributed neural network techniques suffer from a large traffic between the devices; thus are very vulnerable to communication failures. These drawbacks make the distributed neural network techniques inapplicable to wearable devices, which are connected with each other through unstable and low data rate communication medium like human body communication. Therefore, in this paper, we propose a distributed neural network partitioning technique that is resilient to communication failures. Furthermore, we show that the proposed technique also improves the inference accuracy even in case of no communication failure, thanks to the improved network partitioning. We verify through comparative experiments with a real-life neural network application that the proposed technique outperforms the existing state-of-the-art distributed neural network technique in terms of accuracy and resiliency to communication failures.
Abstract
Recently, it is increasingly necessary to run high-end neural network applications with huge computation overhead on top of resource-constrained embedded systems, such as wearable devices. While the huge computational overhead can be alleviated by distributed neural networks running on multiple separate devices, existing distributed neural network techniques suffer from a large traffic between the devices; thus are very vulnerable to communication failures. These drawbacks make the distributed neural network techniques inapplicable to wearable devices, which are connected with each other through unstable and low data rate communication medium like human body communication. Therefore, in this paper, we propose a distributed neural network partitioning technique that is resilient to communication failures. Furthermore, we show that the proposed technique also improves the inference accuracy even in case of no communication failure, thanks to the improved network partitioning. We verify through comparative experiments with a real-life neural network application that the proposed technique outperforms the existing state-of-the-art distributed neural network technique in terms of accuracy and resiliency to communication failures.
참고문헌 (14)
- 허성필, "사물인터넷 기반 헬스케어 서비스 기술 동향", 대한임베디드공학회논문지, 10(4), pp.221~230 (2015), 대한임베디드공학회[DOI]
- J. Yuan, "Privacy Preserving Back-propagation Neural Network Learning Made Practical with Cloud Computing", IEEE Transactions on Parallel and Distributed Systems, 25(1), pp.212~221 (2013)
- L. Lai, "Cmsis-nn : Efficient Neural Network Kernels for Arm Cortex-m Cpus", arXiv preprint arXiv : 1801. 06601, ~ (2018)
- 정종훈, "자원제약 내장형 시스템을 위한 컨볼루션 뉴럴 네트워크 모델 자동 경량화 프레임워크", 정보과학회논문지, 47(2), pp.136~146 (2020), 한국정보과학회[DOI]
- S. Kim, "IB-MAC : Transmission Latency-aware MAC for Electro-magnetic Intra-body Communications", Sensors, 19(2), pp.341~ (2019)
- Electronics and Telecommunications Trends, "2013", 28(2)
- J. Mao, "Modnn : Local Distributed Mobile Computing System for Deep Neural Network", Design, Automation & Test in Europe Conference & Exhibition(DATE) (2017)
- K. Bhardwaj, "Memory-and Communication-aware Model Compression for Distributed Deep Learning Inference on iot", ACM Transactions on Embedded Computing Systems (TECS), 18(5s), pp.1~22 (2019)
- G. Hinton, "Distilling the Knowledge in a Neural Network", arXiv preprint arXiv : 1503. 02531, ~ (2015)
- N. Komodakis, "Paying more Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer" (2017)
- M. E. J. Newman, "Modularity and Community Structure in Networks", Proceedings of the national academy of sciences, 103(23), pp.8577~8582 (2006)
- J. H. Jeong, "Optimization of Distributed Neural Network on Wearable Devices with Low Data Rate Communication", Proceedings of Korean Institute of Communications and Information Sciences Summer Conference, pp.661~662 (2020)
- S. Zagoruyko, "Wide Residual Networks", arXiv preprint arXiv : 1605. 07146, ~ (2016)
- K. Simonyan, "Very Deep Convolutional Networks for Large-scale Image Recognition", arXiv preprint arXiv : 1409. 1556, ~ (2014)