효과적인 2차 최적화 적용을 위한 Minibatch 단위 DNN 훈련 관점에서의 CNN 구현
Implementation of CNN in the view of mini-batch DNN training for efficient second order optimization
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This paper describes some implementation schemes of CNN in view of mini-batch DNN training for efficient second order optimization. This uses same procedure updating parameters of DNN to train parameters of CNN by simply arranging an input image as a sequence of local patches, which is actually equivalent with mini-batch DNN training. Through this conversion, second order optimization providing higher performance can be simply conducted to train the parameters of CNN. In both results of image recognition on MNIST DB and syllable automatic speech recognition, our proposed scheme for CNN implementation shows better performance than one based on DNN.
Abstract
This paper describes some implementation schemes of CNN in view of mini-batch DNN training for efficient second order optimization. This uses same procedure updating parameters of DNN to train parameters of CNN by simply arranging an input image as a sequence of local patches, which is actually equivalent with mini-batch DNN training. Through this conversion, second order optimization providing higher performance can be simply conducted to train the parameters of CNN. In both results of image recognition on MNIST DB and syllable automatic speech recognition, our proposed scheme for CNN implementation shows better performance than one based on DNN.
참고문헌 (13)
- Abdel-Hamid, O., "Convolutional neural networks for speech recognition", IEEE, 2014, pp.22(10), 1533~1545
- Sak, H., "Long short-term recurrent neural network architectures for large scale acoustic modeling", Interspeech 2014, pp.338~342 (2014)
- Chellapilla, K., "High performance convolutional neural networks for document processing", Proceedings of International Workshop on Frontiers in Handwriting Recognition (2006)
- Jia, Y., "Caffe:convolutional architecture for fast feature embedding", Proceedings of the 22nd ACM International Conference on Multimedia, pp.675~678 (2014)
- "cuDNN: efficient primitives for deep learning" (2014)[URL]
- Ren, J., "On vectorization of deep convolutional neural networks for vision tasks", Proceedings of the 29th AAAI Conference on Artificial Intelligence, pp.1840~1846 (2015)
- Song, H. J., "A study of CNN training based on various filter structures and feature normalization methods", Proceedings 2015 International Conference on Speech Sciences, pp.243~244 (2015)
- Lecun, Y., "Gradient-based learning applied to document recognition", Proceedings of the IEEE, 86(11), pp.2278~2324 (1998)
- Amari, S., "Natural gradient works efficiently in learning", Neural Computation, 10, pp.251~276 (1998)
- Povey, D., "Parallel training of DNNs with natural gradient and parameter averaging", Proceedings of International Conference on Learning Representations 2015 (2015)
- 송화전, "다양한Pretraining 방법에 따른 DNN 훈련 방법에 대한 고찰", 한국음성학회 2015 봄학술대회 논문집, pp.169~170 (2015)
- "MNIST" (2013)[URL]
- "Tensorflow" (2015)[URL]