농업용저수지의 실시간 수위 보정을 위한 Hampel Filter의 최적 Window Size 분석
Analysis of the Optimal Window Size of Hampel Filter for Calibration of Real-time Water Level in Agricultural Reservoirs
1
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
Currently, a vast amount of hydrologic data is accumulated in real-time through automatic water level measuring instruments in agricultural reservoirs. At the same time, false and missing data points are also increasing. The applicability and reliability of quality control of hydrological data must besecured for efficient agricultural water management through calculation of water supply and disaster management. Considering the characteristics ofirregularities in hydrological data caused by irrigation water usage and rainfall pattern, the Korea Rural Community Corporation is currently applyingthe Hampel filter as a water level data quality management method. This method uses window size as a key parameter, and if window size is large,distortion of data may occur and if window size is small, many outliers are not removed which reduces the reliability of the corrected data. Thus,selection of the optimal window size for individual reservoir is required. To ensure reliability, we compared and analyzed the RMSE (Root Mean SquareError) and NSE (Nash–Sutcliffe model efficiency coefficient) of the corrected data and the daily water level of the RIMS (Rural InfrastructureManagement System) data, and the automatic outlier detection standards used by the Ministry of Environment. To select the optimal window size, weused the classification performance evaluation index of the error matrix and the rainfall data of the irrigation period, showing the optimal values at3 h. The efficient reservoir automatic calibration technique can reduce manpower and time required for manual calibration, and is expected to improvethe reliability of water level data and the value of water resources.
Abstract
Currently, a vast amount of hydrologic data is accumulated in real-time through automatic water level measuring instruments in agricultural reservoirs. At the same time, false and missing data points are also increasing. The applicability and reliability of quality control of hydrological data must besecured for efficient agricultural water management through calculation of water supply and disaster management. Considering the characteristics ofirregularities in hydrological data caused by irrigation water usage and rainfall pattern, the Korea Rural Community Corporation is currently applyingthe Hampel filter as a water level data quality management method. This method uses window size as a key parameter, and if window size is large,distortion of data may occur and if window size is small, many outliers are not removed which reduces the reliability of the corrected data. Thus,selection of the optimal window size for individual reservoir is required. To ensure reliability, we compared and analyzed the RMSE (Root Mean SquareError) and NSE (Nash–Sutcliffe model efficiency coefficient) of the corrected data and the daily water level of the RIMS (Rural InfrastructureManagement System) data, and the automatic outlier detection standards used by the Ministry of Environment. To select the optimal window size, weused the classification performance evaluation index of the error matrix and the rainfall data of the irrigation period, showing the optimal values at3 h. The efficient reservoir automatic calibration technique can reduce manpower and time required for manual calibration, and is expected to improvethe reliability of water level data and the value of water resources.
참고문헌 (28)
- Bang, J. H., "A study of outlier detection on time series of water-level in agricultural reservoir", Proceedings of the Korean Society of Agricultural Engineers Coneference, pp.60~ (2017), Environmental Protection Agency
- Development of quality control methods for water level data and irrigation water supply estimation, "Choi, J. Y.", Korea Rural Community Corporation, 4~10 (2018)
- Choi, G. H., "Quality control on water-level in irrigation canal using filter techniques", Proceedings of the Korean Society of Agricultural Engineers Conference, pp.107~ (2020)
- Rob J. Hyndman, "Another look at measures of forecast accuracy", International Journal of Forecasting, 22(4), pp.679~688 (2006), Elsevier BV[DOI]
- 김학관, "관계형 데이터베이스를 이용한 수문자료 관리시스템 개발", 한국수자원학회 논문집, 39(10), pp.855~866 (2006), 한국수자원학회
- Kim, H. S., "Hydrological data quality control system for monitoring and improving the data quality", Korean Society of Civil Engineers Conference, pp.2032~2036 (2003)
- 김마가, "임계치 모형과 인공신경망 모형을 이용한 실시간 저수지 수위자료의 이상치 탐지", 한국농공학회논문집, 61(1), pp.107~120 (2019), 한국농공학회[DOI]
- Development of technology for securing reliability of water level measurement data and estimating water supply, "Korea Rural Community Corporation", ~ (2018)
- A Study on the establishment of quality control standards for hydrologic data in agricultural reservoirs and waterways, "Korea Rural Community Corporation", ~ (2019)
- Advanced quality management study of reservoir water level measurement data, "Korea Rural Community Corporation", ~ (2020)
- K-water, "K-water Data Quality Management Guidelines (Water Information)", ~ (2018)
- Lee, J. W, "Data cleansing algorithm for reducing outlier", Proceedings of the Korean Institute of Information and Communication Sciences Conference, pp.342~344 (2018)
- "MathWorks"[URL]
- Development of information analysis technology for abnormal behavior of agricultural reservoirs using big data related to weather information and water levels, "Ministry of Agriculture, Food and Rural Affairs", ~ (2016)
- Environment and maintenance and management of hydrologic research facilities and standards for quality management of hydrologic data, "Ministry of Environment", ~ (2018)
- Ministry of Land, Infrastructure and Transport, "Establishment and operation of the national hydrologic data quality management system" (2018)
- Establishment of a basic plan for hydrologic investigation (2010-2019), "Ministry of Land, Transport and Maritime Affairs", ~ (2008)
- The 4th Comprehensive Plan for Water Resources 2nd Amendment (2011-2020), "Ministry of Land, Transport and Maritime Affairs", ~ (2011)
- Ministry of the Interior and Safety, "Public Data Quality Management Manual ver 2.0", ~ (2018)
- J.E. Nash, "River flow forecasting through conceptual models part I — A discussion of principles", Journal of Hydrology, 10(3), pp.282~290 (1970), Elsevier BV[DOI]
- 오창렬, "신경망을 이용한 영산강의 수질예측", 대한토목학회 논문집B, 22(3-B), pp.371~382 (2002), 대한토목학회
- Ronald K. Pearson, "Generalized Hampel Filters", EURASIP Journal on Advances in Signal Processing, 2016(1) (2016), Springer Science and Business Media LLC[DOI]
- Press, W. H., "Numerical recipes : the art of scientific computing" (1987), Cambridge University Press
- Friedrich Pukelsheim, "The Three Sigma Rule", The American Statistician, 48(2), pp.88~ (1994), JSTOR[DOI]
- Kumar G. Ranjan, "An improved sliding window prediction-based outlier detection and correction for volatile time-series", International Journal of Numerical Modelling: Electronic Networks, Devices and Fields, 34(1), pp.e2816~ (2020), Wiley[DOI]
- Shin, H. J., "A study on the estimation of missing hydrological data using adaptive network-based fuzzy inference system (anfis)", Proceedings of the Korea Water Resources Association Conference, pp.264~ (2020)
- Computing discharge using the index velocity method, "United States Geological Survey", United States Geological Survey, 35~59 (2012)
- 양미혜, "딥러닝 기반 LSTM 모형을 이용한 저수지 수위자료 이상치 탐지", 한국방재학회논문집, 21(1), pp.71~81 (2021), 한국방재학회[DOI]