Contour Plots of Objective Functions for Feed-Forward Neural Networks
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Error surfaces provide us with very important information for training of feed-forward neural networks (FNNs). In this paper, we draw the contour plots of various error or objective functions for training of FNNs. Firstly, when applying FNNs to classifications, the weakness of mean-squared error is explained with the viewpoint of error contour plot. And the classification figure of merit, mean log-square error, cross-entropy error, and n-th order extension of cross-entropy error objective functions are considered for the contour plots. Also, the recently proposed target node method is explained with the viewpoint of contour plot. Based on the contour plots, we can explain characteristics of various error or objective functions when training of FNNs proceeds.
Abstract
Error surfaces provide us with very important information for training of feed-forward neural networks (FNNs). In this paper, we draw the contour plots of various error or objective functions for training of FNNs. Firstly, when applying FNNs to classifications, the weakness of mean-squared error is explained with the viewpoint of error contour plot. And the classification figure of merit, mean log-square error, cross-entropy error, and n-th order extension of cross-entropy error objective functions are considered for the contour plots. Also, the recently proposed target node method is explained with the viewpoint of contour plot. Based on the contour plots, we can explain characteristics of various error or objective functions when training of FNNs proceeds.
참고문헌 (17)
- K. Hornik, "Multilayer Feed-forward Networks are Universal Approximators", Neural Networks, 2, pp.359~366 (1989)
- K. Hornik, "Approximation Capabilities of Multilayer Feedforward Networks", Neural Networks, 4, pp.251~257 (1991)
- S. Suzuki, "Constructive Function Approximation by Three-Layer Artificial Neural Networks", Neural Networks, 11, pp.1049~1058 (1998)
- Y. Liao, "Relaxed Conditions for Radial-Basis Function Networks to be Universal Approximators", Neural Networks, 16, pp.1019~1028 (2003)
- D. E. Rumelhart, "Parallel Distributed Processing" (1986)
- C. Wang, "Training Neural Networks with Additive Noise in the Desired Signal", IEEE Trans. Neural Networks, 10, pp.1511~1517 (1999)
- J. B. Hampshare, "A Novel Objective Function for Improved Phoneme Recognition Using Time-Delay Neural Networks", IEEE Trans. Neural Networks, 1, pp.216~228 (1990)
- K. Liano, "Robust Error measure for Supervised Neural Network Learning with Outliers", IEEE Trans. Neural Networks, 7, pp.246~250 (1996)
- A. van Ooyen, "Improving the Convergence of the Backpropagation Algorithm", Neural Networks, 5, pp.465~471 (1992)
- S.-H. Oh, "Improving the Error Back-Propagation Algorithm with a Modified Error Function", IEEE Trans. Neural Networks, 8, pp.799~803 (1997)
- S.-H. Oh, "Error Back-Propagation Algorithm for Classification of Imbalanced Data", Neurocomputing, 74, pp.1058~1061 (2011)
- H. White, "Learning in Artificial Neural Networks: A Statistical Perspective", Neural Computation, 1(4), pp.425~464 (1989)
- M. D. Richard, "Neural Network Classifier Estimate Bayesian a Posteriori Probabilities", Neural Computa., 3, pp.461~483 (1991)
- S.-H. Oh, "Statistical Analyses of Various Error Functions for Pattern Classifiers", Proc. Convergence on Hybrid Information Technology, 206, pp.129~133 (2011), CCIS
- 오상훈, "A Statistical Perspective of Neural Networks for Imbalanced Data Problems", International Journal of Contents, 7(3), pp.1~5 (2011), 한국콘텐츠학회
- N. Baba, "A New Approach for Finding the Global Minimum of Error Function of Neural Networks", Neural Networks, 2, pp.367~373 (1989)
- Y. Lee, "An Analysis of Premature Saturation in Back-Propagation Learning", Neural Networks, 6, pp.719~728 (1993)