결합된 파라메트릭 활성함수를 이용한 합성곱 신경망의 성능 향상
Performance Improvement Method of Convolutional Neural Network Using Combined Parametric Activation Functions
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
합성곱 신경망은 이미지와 같은 격자 형태로 배열된 데이터를 다루는데 널리 사용되고 있는 신경망이다. 일반적인 합성곱 신경망은 합성곱층과완전연결층으로 구성되며 각 층은 비선형활성함수를 포함하고 있다. 본 논문은 합성곱 신경망의 성능을 향상시키기 위해 결합된 파라메트릭 활성함수를 제안한다. 결합된 파라메트릭 활성함수는 활성함수의 크기와 위치를 변환시키는 파라미터를 적용한 파라메트릭 활성함수들을 여러 번 더하여만들어진다. 여러 개의 크기, 위치를 변환하는 파라미터에 따라 다양한 비선형간격을 만들 수 있으며, 파라미터는 주어진 입력데이터에 의해 계산된손실함수를 최소화하는 방향으로 학습할 수 있다. 결합된 파라메트릭 활성함수를 사용한 합성곱 신경망의 성능을 MNIST, Fashion MNIST, CIFAR10그리고 CIFAR100 분류문제에 대해 실험한 결과, 다른 활성함수들보다 우수한 성능을 가짐을 확인하였다.
Abstract
Convolutional neural networks are widely used to manipulate data arranged in a grid, such as images. A general convolutional neuralnetwork consists of a convolutional layers and a fully connected layers, and each layer contains a nonlinear activation functions. Thispaper proposes a combined parametric activation function to improve the performance of convolutional neural networks. The combinedparametric activation function is created by adding the parametric activation functions to which parameters that convert the scale andlocation of the activation function are applied. Various nonlinear intervals can be created according to parameters that convert multiplescales and locations, and parameters can be learned in the direction of minimizing the loss function calculated by the given input data. As a result of testing the performance of the convolutional neural network using the combined parametric activation function on theMNIST, Fashion MNIST, CIFAR10 and CIFAR100 classification problems, it was confirmed that it had better performance than otheractivation functions.
참고문헌 (17)
- Y. Bengio, "Deep learning" (2017), MIT Press
- C. A. Charu, "Neural Networks and Deep Learning: A Textbook" (2018), Springer International Publishing AG
- 고영민, "결합된 파라메트릭 활성함수를 이용한완전연결신경망의 성능 향상", 정보처리학회 논문지, 11(1), pp.1~10 (2022), 한국정보처리학회
- 공나영, "파라메트릭 활성함수를 이용한 심층신경망의 성능향상 방법", 한국콘텐츠학회 논문지, 21(3), pp.616~625 (2021), 한국콘텐츠학회[DOI]
- 공나영, "민첩한 활성함수를 이용한 합성곱 신경망의 성능 향상", 정보처리학회 논문지, 9(7), pp.213~220 (2020), 한국정보처리학회[DOI]
- 고영민, "파라메트릭 활성함수를 이용한 기울기 소실 문제의 완화", 정보처리학회 논문지, 10(10), pp.407~420 (2021), 한국정보처리학회
- A. Apicella, "A survey on modern trainable activation functions", Neural Networks, 138, pp.14~32 (2021)
- V. Nair, "Rectified linear units improve restricted boltzmann machines", Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML), pp.807~814 (2010)
- M. Roodschild, "A new approach for the vanishing gradient problem on sigmoid activation", Springer Nature, 20(4), pp.351~360 (2020)
- S. Hochreiter, "The vanishing gradient problem during learning recurrent neural nets and problem solutions", International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(2), pp.107~116 (1998)
- S. Kong, "Hexpo : A vanishing-proof activation function", International Joint Conference on Neural Networks, pp.2562~2567 (2017)
- Y. Qin, "The optimized deep belief networkswith improved logistic Sigmoid units and their application in faultdiagnosis for planetary gearboxes of wind turbines", IEEE Transactions on Industrial Electronics, 66(5), pp.3814~3824 (2018)
- X. Wang, "ReLTanh : An activation function with vanishing gradient resistance for SAE-based DNNs and its application to rotating machinery fault diagnosis", Neurocomputing, 363, pp.88~98 (2019)
- B. Xu, "Empirical evaluation of rectified activations in convolutional network", arXiv : 1505. 00853, ~ (2015)
- S. Qian, "Adaptive activation functions in convolutional neural networks", Neurocomputing, 272, pp.204~212 (2017)
- K. He, "Delving deep into rectifiers : Surpassing human-level performance on imagenet classification", arXiv : 1502. 01852, ~ (2015)
- D. Clevert, "Fast and accurate deep network learning by exponential linear units(ELUs)", arXiv : 1511. 07289, ~ (2016)