Effects of Hyper-parameters and Dataset on CNN Training
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
The purpose of training a convolutional neural network (CNN) is to obtain weight factors that give high classification accuracies. The initial values of hyper-parameters affect the training results, and it is important to train a CNN with a suitable hyper-parameter set of a learning rate, a batch size, the initialization of weight factors, and an optimizer. We investigate the effects of a single hyper-parameter while others are fixed in order to obtain a hyper-parameter set that gives higher classification accuracies and requires shorter training time using a proposed VGG-like CNN for training since the VGG is widely used. The CNN is trained for four datasets of CIFAR10, CIFAR100, GTSRB and DSDL-DB. The effects of the normalization and the data transformation for datasets are also investigated, and a training scheme using merged datasets is proposed.
Abstract
The purpose of training a convolutional neural network (CNN) is to obtain weight factors that give high classification accuracies. The initial values of hyper-parameters affect the training results, and it is important to train a CNN with a suitable hyper-parameter set of a learning rate, a batch size, the initialization of weight factors, and an optimizer. We investigate the effects of a single hyper-parameter while others are fixed in order to obtain a hyper-parameter set that gives higher classification accuracies and requires shorter training time using a proposed VGG-like CNN for training since the VGG is widely used. The CNN is trained for four datasets of CIFAR10, CIFAR100, GTSRB and DSDL-DB. The effects of the normalization and the data transformation for datasets are also investigated, and a training scheme using merged datasets is proposed.
참고문헌 (8)
- K. Alex, "Imagenet classification with deep convolutional neural networks", Proc. of Neural Information Processing Systems, pp.1097~1105 (2012)
- Y. Bengio, "Practical recommendations for gradient-based training of deep architecture, Neural Networks: Tricks of the Trade", pp.437~478 (2012), Springer Berlin Heidelberg
- T. M. Breuel, "The Effects of Hyperparameters on SGD Training of Neural Networks", ~
- N. Ketkar, "Deep learning with Python" (2017), Apress
- M. Moller, "Supervised learning on large redundant training sets", Proc. of Neural Networks for Signal Processing, pp.79~89 (1992)
- M.D. Zeiler, "Visualizing and Understanding Convolutional Networks", Proc. of 13th European Conference on Computer Vision, pp.818~833 (2014)
- Y. Xu, "Improved relation classification by deep recurrent neural networks with data augmentation", Proc. of 26th International Conference on Computational Linguistics (COLING), pp.1461~1470 (2016)
- A. Rusiecki, "Training Neural Networks on Noisy Data", Proc. of 13th ICAISC, pp.131~142 (2014)