홍수량 예측 인공신경망 모형의 활성화 함수에 따른 영향 분석
Impact of Activation Functions on Flood Forecasting Model Based on Artificial Neural Networks
0
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
The objective of this study was to analyze the impact of activation functions on flood forecasting model based on Artificial neural networks (ANNs). The traditional activation functions, the sigmoid and tanh functions, were compared with the functions which have been recently recommended for deepneural networks; the ReLU, leaky ReLU, and ELU functions. The flood forecasting model based on ANNs was designed to predict real-time runofffor 1 to 6-h lead time using the rainfall and runoff data of the past nine hours. The statistical measures such as R2, Nash-Sutcliffe Efficiency (NSE),Root Mean Squared Error (RMSE), the error of peak time (ETp), and the error of peak discharge (EQp) were used to evaluate the model accuracy. The tanh and ELU functions were most accurate with R2=0.97 and RMSE=30.1 (m3/s) for 1-h lead time and R2=0.56 and RMSE=124.6∼124.8 (m3/s)for 6-h lead time. We also evaluated the learning speed by using the number of epochs that minimizes errors. The sigmoid function had the slowestlearning speed due to the ‘vanishing gradient problem’ and the limited direction of weight update. The learning speed of the ELU function was 1.2times faster than the tanh function. As a result, the ELU function most effectively improved the accuracy and speed of the ANNs model, so it wasdetermined to be the best activation function for ANNs-based flood forecasting.
Abstract
The objective of this study was to analyze the impact of activation functions on flood forecasting model based on Artificial neural networks (ANNs). The traditional activation functions, the sigmoid and tanh functions, were compared with the functions which have been recently recommended for deepneural networks; the ReLU, leaky ReLU, and ELU functions. The flood forecasting model based on ANNs was designed to predict real-time runofffor 1 to 6-h lead time using the rainfall and runoff data of the past nine hours. The statistical measures such as R2, Nash-Sutcliffe Efficiency (NSE),Root Mean Squared Error (RMSE), the error of peak time (ETp), and the error of peak discharge (EQp) were used to evaluate the model accuracy. The tanh and ELU functions were most accurate with R2=0.97 and RMSE=30.1 (m3/s) for 1-h lead time and R2=0.56 and RMSE=124.6∼124.8 (m3/s)for 6-h lead time. We also evaluated the learning speed by using the number of epochs that minimizes errors. The sigmoid function had the slowestlearning speed due to the ‘vanishing gradient problem’ and the limited direction of weight update. The learning speed of the ELU function was 1.2times faster than the tanh function. As a result, the ELU function most effectively improved the accuracy and speed of the ANNs model, so it wasdetermined to be the best activation function for ANNs-based flood forecasting.
참고문헌 (28)
- Adamowski, J., "Comparison of machine learning methods for runoff forecasting in mountainous watersheds with limited data", Journal of Water and Land Development, 17(1), pp.89~97 (2012)[DOI]
- Aggarwal, C. C., "Neural networks and deep learning" (2018), Springer
- "Keras" (2015)[URL]
- Chollet, F., "Deep Learning with Python" (2017), Manning
- Clevert, D. A., "Fast and accurate deep network learning by exponential linear units (ELUs)", arXiv preprint. arXiv:1511.07289, ~ (2016)
- Dawson, C. W., "An artificial neural network approach to rainfall-runoff modeling", Hydrological Sciences, 43(1), pp.47~66 (1998)[DOI]
- Juan Peralta Donate, "Time series forecasting using a weighted cross-validation evolutionary artificial neural network ensemble", Neurocomputing, 109, pp.27~32 (2013), Elsevier BV[DOI]
- Glorot, X., "Understanding the difficulty of training deep feedforward neural networks", Proc. Thirteenth International Conference on Artificial Intelligence and Statistics, pp.249~256 (2010)
- Goodfellow, I., "Deep Learning" (2016), MIT press
- Kuo-lin Hsu, "Artificial Neural Network Modeling of the Rainfall-Runoff Process", Water Resources Research, 31(10), pp.2517~2530 (1995), American Geophysical Union (AGU)[DOI]
- Caihong Hu, "Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation", Water, 10(11), pp.1543~ (2018), MDPI AG[DOI]
- Minhyuk Jeung, "Evaluation of random forest and regression tree methods for estimation of mass first flush ratio in urban catchments", Journal of Hydrology, 575, pp.1099~1110 (2019), Elsevier BV[DOI]
- 강문성, "인공신경망 이론을 이용한 단기 홍수량 예측", 한국농공학회논문집, 45(2), pp.45~57 (2003), 한국농공학회
- Kang, M. S., "Application of grey model and artificial neural networks to flood forecasting", Journal of the American Water Resources Association, 42(2), pp.473~486 (2006)
- Kim, J. H., "A study on hydrologic forecasting of stream flows based on artificial neural network", Ph.D., pp.1993
- Kingma, D. P., "Adam: A method for stochastic optimization", arXiv preprint. arXiv:1412.6980, ~ (2014)
- Alex Krizhevsky, "ImageNet classification with deep convolutional neural networks", Communications of the ACM, 60(6), pp.84~90 (2017), Association for Computing Machinery (ACM)[DOI]
- Lau, M. M., "Investigation of activation functions in deep belief network", 2017 2nd International Conference on Control and Robotics Engineering (ICCRE), pp.201~206 (2017)
- Lee, G. S., "The prediction of runoff using artificial neural network in the Young-san River", Journal of Korea Water Resources Association, 33(1), pp.251~256 (2000)
- Marina Campolo, "River flood forecasting with a neural network model", Water Resources Research, 35(4), pp.1191~1197 (1999), American Geophysical Union (AGU)[DOI]
- Ministry of the Interior and Safety, "Statistical yearbook of natural disaster" (2018)
- Moriasi, D. N, "Hydrologic and Water Quality Models: Performance Measures and Evaluation Criteria", Transactions of the ASABE, 58(6), pp.1763~1785 (2015), American Society of Agricultural and Biological Engineers (ASABE)[DOI]
- Sarkar, A., "Artificial neural networks for event based rainfall-runoff modeling", Journal of Water Resource and Protection, 4(10), pp.891~ (2012)[DOI]
- Solomatine, D., "Practical Hydroinformatics", pp.17~30 (2009)
- Jung-Hun Song, "Regionalization of a Rainfall-Runoff Model: Limitations and Potentials", Water, 11(11), pp.2257~ (2019), MDPI AG[DOI]
- Xu, B., "Empirical evaluation of rectified activations in convolutional network", arXiv preprint. arXiv:1505.00853, ~ (2015)
- Chih-Chieh Young, "A physically based and machine learning hybrid approach for accurate rainfall-runoff modeling during extreme typhoon events", Applied Soft Computing, 53, pp.205~216 (2017), Elsevier BV[DOI]
- Zealnad, C. M., "Short term stream flow forecasting using artificial neural networks", Journal of Hydrology, 214, pp.32~48 (1999)[DOI]