Performance Analysis of Different Optimizers, Batch Sizes, and Epochs on Convolutional Neural Network for Image Classification
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The important thing in the field of deep learning is to find out the appropriate hyper-parameter for image classification. In this study, the main objective is to investigate the performance of various hyper-parameters in a convolutional neural network model based on the image classification problem. The dataset was obtained from the Kaggle dataset. The experiment was conducted through different hyper-parameters. For this proposal, Stochastic Gradient Descent without momentum (SGD), Adaptive Moment Estimation (Adam), Adagrad, Adamax optimizer, and the number of batch sizes (16, 32, 64, 120), and the number of epochs (50, 100, 150) were considered as hyper-parameters to determine the losses and accuracy of a model. In addition, Binary Cross-entropy Loss Function (BCLF) was used for evaluating the performance of a model. In this study, the VGG16 convolutional neural network was used for image classification. Empirical results demonstrated that a model had minimum losses obtain by Adagrad optimizer in the case of 16 batch sizes and 50 epochs. In addition, the SGD with a 32 batch sizes and 150 epochs and the Adam with a 64 batch sizes and 50 epochs had the best performance based on the loss value during the training process. Interestingly, the accuracy was higher while performing the Adagrad and Adamax optimizer with a 120 batch sizes and 150 epochs. In this study, the Adagrad optimizer with a 120 batch sizes and 150 epochs performed slightly better among those optimizers. In addition, an increasing number of epochs can improve the performance of accuracy. It can help to create a broader scope for further experiments on several datasets to perceive the suitable hyper-parameters for the convolutional neural network. Dataset: https://www.kaggle.com/c/dogs-vs-cats/data
Abstract
The important thing in the field of deep learning is to find out the appropriate hyper-parameter for image classification. In this study, the main objective is to investigate the performance of various hyper-parameters in a convolutional neural network model based on the image classification problem. The dataset was obtained from the Kaggle dataset. The experiment was conducted through different hyper-parameters. For this proposal, Stochastic Gradient Descent without momentum (SGD), Adaptive Moment Estimation (Adam), Adagrad, Adamax optimizer, and the number of batch sizes (16, 32, 64, 120), and the number of epochs (50, 100, 150) were considered as hyper-parameters to determine the losses and accuracy of a model. In addition, Binary Cross-entropy Loss Function (BCLF) was used for evaluating the performance of a model. In this study, the VGG16 convolutional neural network was used for image classification. Empirical results demonstrated that a model had minimum losses obtain by Adagrad optimizer in the case of 16 batch sizes and 50 epochs. In addition, the SGD with a 32 batch sizes and 150 epochs and the Adam with a 64 batch sizes and 50 epochs had the best performance based on the loss value during the training process. Interestingly, the accuracy was higher while performing the Adagrad and Adamax optimizer with a 120 batch sizes and 150 epochs. In this study, the Adagrad optimizer with a 120 batch sizes and 150 epochs performed slightly better among those optimizers. In addition, an increasing number of epochs can improve the performance of accuracy. It can help to create a broader scope for further experiments on several datasets to perceive the suitable hyper-parameters for the convolutional neural network. Dataset: https://www.kaggle.com/c/dogs-vs-cats/data
참고문헌 (25)
- Babanezhad R, "Stop wasting my gradients:Practical SVRG", J. Adv Neu Info Proc Sys, pp.2251~2259 (2015)
- Bello I, "Neural optimizer search with reinforcement learning", Int Conf on Machine Learning, 1, pp.712~721 (2017)
- Daneshm H, "Starting small-learning with adaptive sample sizes", Int Conf on Machine Learning, 3, pp.2167~2186 (2016)
- "AdaBatch:Adaptive batch sizes for training deep neural networks" (2017)[URL]
- Duchi JC, "Randomized smoothing for(parallel)stochastic optimization", IEEE Int Conf on Decision and Control, 12, pp.5442~5444 (2012)
- "Accurate large minibatch" (2017)[URL]
- Hinton G, "Neural networks for machine learning", Coursera, Video Lectures, 1, pp.264~
- Hoffer E, "Closing the generalization gap in large batch training of neural networks", J. Adv Neu Info Proc Sys, pp.1732~1742 (2017)
- Huo B, "Research on novel image classification algorithm based on multi-feature extraction and modified SVM", Classifier, 9, pp.103~112 (2015)
- Hutter F, "Meta-learning", Studies in Computational Intelligence, 498, pp.233~317 (2014)
- Ioffe S, "Batch normalization accelerating deep network training by reducing internal covariate shift", Int. Machine Learning, pp.448~456 (2015)
- Keskar NS, "On large-batch training for deep learning : Generalization gap and sharp minima. Int", Int. Conference on Learning Representations, pp.1~16 (2019)
- Kingma DP, "Adam: A method for stochastic optimization", Int. Conference on Learning Representations, pp.5~15 (2015)
- Li J, "A novel large-scale multimedia image data classification algorithm based on mapping assisted deep neural network", Multi Tools and App, 76, pp.678~710 (2017)
- Mane H, "Image classification using deep learning", Int. J. Eng. Technol, 7, pp.614~617 (2018)
- Radiuk PM, "Impact of training set batch size on the performance of convolutional neural networks for diverse datasets", Info. Techno. Sci, 20, pp.20~24 (2018)
- Rangarajan AK, "Tomato crop disease classification using pre-trained deep learning algorithm", Proc Com. Sci, 33, pp.1040~1047 (2018)
- Reddy SV, "Optimization of deep learning using various optimizers, loss functions and dropout", Int. J. Recent Technol. Eng, 7, pp.448~455 (2018)
- Ruder S, "An overview of gradient descent optimization", arXiv:1609.04747, 1~14 (2016)
- Sadowski P, "Notes on backpropagation", Dept. Com. Sci. University of California Irvine, 1, pp.1~4 (2017)
- Selvaraju RR, "Visual explanations from deep networks via gradient-based localization", Proceedings of the IEEE International Conference on Computer Vision, 17, pp.331~336 (2016)
- Simonyan K, "Very deep convolutional networks for large-scale image recognition", Int. Conference on Learning Representations, pp.1~14 (2015)
- Srivastava Y, "A performance comparison of loss functions for deep face recognition", Int. Conference on Computer Vision, Pattern Recog. Image Proc. Graph, pp.322~332 (2018)
- Takáč M, "Mini-batch primal and dual methods for SVMs", Int. Conference on Machine Learning, 28, pp.2059~2067 (2013)
- "Large batch training of convolutional networks" (2017)[URL]