임계치 모형과 인공신경망 모형을 이용한 실시간 저수지 수위자료의 이상치 탐지
Outlier Detection of Real-Time Reservoir Water Level Data Using Threshold Model and Artificial Neural Network Model
0
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
Reservoir water level data identify the current water storage of the reservoir, and they are utilized as primary data for management and research ofagricultural water. For the reservoir storage management, Korea Rural Community Corporation (KRC) installed water level stations at around 1,600agricultural reservoirs and has been collecting the water level data every 10 minutes. However, various kinds of outliers due to noise and erroneousproblems are frequently appearing because of environmental and physical causes. Therefore, it is necessary to detect outlier and improve the qualityof reservoir water level data to utilize the water level data in purpose. This study was conducted to detect and classify outlier and normal data usingtwo different models including the threshold model and the artificial neural network (ANN) model. The results were compared to evaluate theperformance of the models. The threshold model identifies the outlier by setting the upper/lower bound of water level data and variation data and bysetting bandwidth of water level data as a threshold of regarding erroneous water level. The ANN model was trained with prepared training datasetas normal data (T) and outlier (F), and the ANN model operated for identifying the outlier. The models are evaluated with reference data which werecollected reservoir water level data in daily by KRC. The outlier detection performance of the threshold model was better than the ANN model, butANN model showed better detection performance for not classifying normal data as outlier.
Abstract
Reservoir water level data identify the current water storage of the reservoir, and they are utilized as primary data for management and research ofagricultural water. For the reservoir storage management, Korea Rural Community Corporation (KRC) installed water level stations at around 1,600agricultural reservoirs and has been collecting the water level data every 10 minutes. However, various kinds of outliers due to noise and erroneousproblems are frequently appearing because of environmental and physical causes. Therefore, it is necessary to detect outlier and improve the qualityof reservoir water level data to utilize the water level data in purpose. This study was conducted to detect and classify outlier and normal data usingtwo different models including the threshold model and the artificial neural network (ANN) model. The results were compared to evaluate theperformance of the models. The threshold model identifies the outlier by setting the upper/lower bound of water level data and variation data and bysetting bandwidth of water level data as a threshold of regarding erroneous water level. The ANN model was trained with prepared training datasetas normal data (T) and outlier (F), and the ANN model operated for identifying the outlier. The models are evaluated with reference data which werecollected reservoir water level data in daily by KRC. The outlier detection performance of the threshold model was better than the ANN model, butANN model showed better detection performance for not classifying normal data as outlier.
참고문헌 (34)
- 안지현, "인공신경망 기법을 이용한 논에서의 지표 유출량 산정", 한국농공학회논문집, 54(4), pp.65~71 (2012), 한국농공학회[DOI]
- Ahn, S. J., "Rainfall forecasting Using neural network", Journal of The Korean Society of Civil Engineers, 20(5B), pp.711~722 (2000)
- Ahn, S. J., "Forecasting of runoff hydrograph using neural network algorithms", Journal of Korean Water Resources Association, 33(4), pp.505~515 (2000)[DOI]
- Bang, J. H., "A study of outlier detection on time series of water-level in agricultural reservoir", Proceedings of the Korean Society of Agricultural Engimeers Conefrence, 60 (2017)
- Choi, J. K., "Application of neural network to water resources", Journal of Korean National Comittee on Irrigation and Drainage, 7(2), pp.248~258 (2000)[DOI]
- Development of quality control methods for water level data and irrigation water supply estimation, "Choi, J. Y.", Korea Rural Community Corporation, 4~10 (2018)
- Song Feng, "Quality control of daily meteorological data in China, 1951–2000: a new dataset", International Journal of Climatology, 24(7), pp.853~870 (2004), Wiley[DOI]
- J. Fidel González-Rouco, "Quality Control and Homogeneity of Precipitation Data in the Southwest of Europe", Journal of Climate, 14(5), pp.964~978 (2001), American Meteorological Society[DOI]
- Günther, F., "Neuralnet : training of neural networks", The R Journal, 2(1), pp.30~38 (2010)
- D L Harrison, "Improving precipitation estimates from weather radar using quality control and correction techniques", Meteorlogical Applications, 7(2), pp.135~144 (2000), Wiley[DOI]
- Jeffery S. Horsburgh, "A relational model for environmental and water resources data", Water Resources Research, 44(5) (2008), American Geophysical Union (AGU)[DOI]
- 장수형, "홍수조절기능을 가진 저수지의 치수 및 이수목적 운영관리 시스템 구축 : Ⅰ. 실시간 저수지 유입량 예측모형 구축", 대한토목학회 논문집B, 27(2B), pp.133~140 (2007), 대한토목학회
- Jeong, G. H., "Comparing water distribution model with reservoir and water system management", Water and Future, 40(10), pp.38~43 (2007)[DOI]
- 강부식, "인공신경망과 중규모기상수치예보를 이용한 강수확률예측", 대한토목학회 논문집B, 28(5), pp.485~493 (2008), 대한토목학회
- Kang, M. G., "Efficient management of agricultural canal systems through quality management of water level and water quantity data", Rural Resources, 52(2), pp.87~96 (2010)[DOI]
- Kilonsky, B. J., "In the pursuit of high-quality sea level data", Oceans 91 Proceedings, (2), pp.669~675 (1991)
- 김창훈, "지하수위, 토양수분함량 및 수질변화를 활용한 습윤화 지역의 배수시설 효과 평가", 지질공학, 26(2), pp.251~260 (2016), 대한지질공학회
- Kim, H. G., "Correlation of deep landside occurrence and variation of groundwater level", Journal of The Korea Society of Forest Engineering and technology, 15(1), pp.1~12 (2017)[DOI]
- 김학관, "관계형 데이터베이스를 이용한 수문자료 관리시스템 개발", 한국수자원학회 논문집, 39(10), pp.855~866 (2006), 한국수자원학회
- 김수전, "신경망에 의한 레이더강우 보정 및 유출해석", 대한토목학회 논문집B, 30(2), pp.159~167 (2010), 대한토목학회
- Kim, S. W., "A study on the forecasting of daily streamflow using the multilayer neural networks model", Journal of Korean Water Resources Association, 33(5), pp.537~550 (2000)[DOI]
- Kim, S. W., "The flood water stage prediction based on neural networks method in stream gauge station", Journal of Korean Water Resources Association, 33(2), pp.247~262 (2000)[DOI]
- 이충원, "지속적 수위변동 및 월류에 따른 저수지 제체의 거동 연구", 한국지반환경공학회 논문집, 15(6), pp.41~48 (2014), 한국지반환경공학회[DOI]
- Mounce, S. R., "Novelty detection for time series data analysis in water distribution systems using support vector machines", Journal of Hydroinformatics, 13(4), pp.672~686 (2011)[DOI]
- M. Mourad, "A method for automatic validation of long time series of data in urban hydrology", Water Science and Technology, 45(4-5), pp.263~270 (2002), IWA Publishing[DOI]
- 오창렬, "신경망을 이용한 영산강의 수질예측", 대한토목학회 논문집B, 22(3-B), pp.371~382 (2002), 대한토목학회
- 오재우, "신경망을 이용한 결측 수문자료 추정 및 실시간 자료 보정", 한국수자원학회 논문집, 41(10), pp.1059~1065 (2008), 한국수자원학회
- 박지훈, "IoT 기반 지능형 수위 모니터링 플랫폼 설계 및 구현", 농촌계획, 21(4), pp.177~186 (2015), 한국농촌계획학회[DOI]
- Udo Schneider, "GPCC's new land surface precipitation climatology based on quality-controlled in situ data and its role in quantifying the global water cycle", Theoretical and Applied Climatology, 115(1-2), pp.15~40 (2014), Springer Science and Business Media LLC[DOI]
- 서영민, "기계학습모델을 이용한 저수지 수위 예측", 한국농공학회논문집, 59(3), pp.97~110 (2017), 한국농공학회[DOI]
- Shim, S. B., "Decision support system for real-time reservoir operation during flood period", Journal of Korean Water Resources Association, 30(5), pp.431~439 (1997)[DOI]
- Matthias Steiner, "Effect of bias adjustment and rain gauge data quality control on radar rainfall estimation", Water Resources Research, 35(8), pp.2487~2503 (1999), American Geophysical Union (AGU)[DOI]
- 여운기, "인공신경망과 유전자알고리즘의 결합모형을 이용한 수위예측에 관한 연구", 한국수자원학회 논문집, 43(8), pp.721~731 (2010), 한국수자원학회
- 윤강훈, "신경망을 이용한 낙동강 유역 홍수기 댐유입량 예측", 한국수자원학회 논문집, 37(1), pp.67~75 (2004), 한국수자원학회