이어도 해양과학기지 수온 시계열 자료의 이상값 검출을 위한 국제 품질검사의 성능 평가
Evaluation of International Quality Control Procedures for Detecting Outliers in Water Temperature Time-series at Ieodo Ocean Research Station
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Quality control (QC) to process observed time series has become more critical as the types and amount of observed data have increased along with the development of ocean observing sensors and communication technology. International ocean observing institutions have developed and operated automatic QC procedures for these observed time series. In this study, the performance of automated QC procedures proposed by U.S. IOOS (Integrated Ocean Observing System), NDBC (National Data Buy Center), and OOI (Ocean Observatory Initiative) were evaluated for observed time-series particularly from the Yellow and East China Seas by taking advantage of a confusion matrix. We focused on detecting additive outliers (AO) and temporary change outliers (TCO) based on ocean temperature observation from the Ieodo Ocean Research Station (I-ORS) in 2013. Our results present that the IOOS variability check procedure tends to classify normal data as AO or TCO. The NDBC variability check tracks outliers well but also tends to classify a lot of normal data as abnormal, particularly in the case of rapidly fluctuating time-series. The OOI procedure seems to detect the AO and TCO most effectively and the rate of classifying normal data as abnormal is also the lowest among the international checks. However, all three checks need additional scrutiny because they often fail to classify outliers when intermittent observations are performed or as a result of systematic errors, as well as tending to classify normal data as outliers in the case where there is abrupt change in the observed data due to a sensor being located within a sharp boundary between two water masses, which is a common feature in shallow water observations. Therefore, this study underlines the necessity of developing a new QC algorithm for time-series occurring in a shallow sea.
Abstract
Quality control (QC) to process observed time series has become more critical as the types and amount of observed data have increased along with the development of ocean observing sensors and communication technology. International ocean observing institutions have developed and operated automatic QC procedures for these observed time series. In this study, the performance of automated QC procedures proposed by U.S. IOOS (Integrated Ocean Observing System), NDBC (National Data Buy Center), and OOI (Ocean Observatory Initiative) were evaluated for observed time-series particularly from the Yellow and East China Seas by taking advantage of a confusion matrix. We focused on detecting additive outliers (AO) and temporary change outliers (TCO) based on ocean temperature observation from the Ieodo Ocean Research Station (I-ORS) in 2013. Our results present that the IOOS variability check procedure tends to classify normal data as AO or TCO. The NDBC variability check tracks outliers well but also tends to classify a lot of normal data as abnormal, particularly in the case of rapidly fluctuating time-series. The OOI procedure seems to detect the AO and TCO most effectively and the rate of classifying normal data as abnormal is also the lowest among the international checks. However, all three checks need additional scrutiny because they often fail to classify outliers when intermittent observations are performed or as a result of systematic errors, as well as tending to classify normal data as outliers in the case where there is abrupt change in the observed data due to a sensor being located within a sharp boundary between two water masses, which is a common feature in shallow water observations. Therefore, this study underlines the necessity of developing a new QC algorithm for time-series occurring in a shallow sea.
참고문헌 (22)
- 해양예보 의사결정 체계의 고도화를위한 국가해양관측망개선연구 결과보고서, "국립해양조사원", 국립해양조사원, 522~ (2019)
- 기상청, "기상관측데이터 품질검사 프로세스", 기상청 국가기후데이터센터, 62~ (2019)
- 민용침, "해양과학기지 시계열 관측 자료 품질관리 시스템 구축: 국제 관측자료 품질관리 방안 수온 관측 자료 시범적용과 문제점", Ocean and Polar Research, 42(3), pp.195~210 (2020), 한국해양과학기술원[DOI]
- 유재성, "Unit Root Test를 기반으로 한 장기 시계열 데이터의 Non-Stationary 발생에 따른 구조 변화 검정 및 시각화 연구", 정보처리학회 논문지, 8(7), pp.289~302 (2019), 한국정보처리학회[DOI]
- 이상호, "R을 활용한 이상치 검출과 처리", pp.383~ (2015), 자유아카데미
- 이재익, "해양과학기지에서 수행한 염분 표류 방지 실험 결과 및 염분 고품질 시계열 자료 생산을 위한 제언", 한국연안방재학회지, 8(4), pp.323~333 (2021), (사)한국연안방재학회[DOI]
- 조홍연, "이상자료가 연안 환경자료의 통계 척도에 미치는 영향", Ocean and Polar Research, 38(2), pp.149~159 (2016), 한국해양과학기술원[DOI]
- 채정훈, "WISE 복합기상센서 관측 자료 품질관리시스템", 대기, 24(3), pp.445~456 (2014), 한국기상학회[DOI]
- Kyung-Ja Ha, "Observations Utilizing Korea Ocean Research Stations and their Applications for Process Studies", Bulletin of the American Meteorological Society, 100(10), pp.2061~2075 (2019), American Meteorological Society[DOI]
- Hawkins DM, "Identification of outliers", pp.188~ (1980), Springer
- IOOS, "Manual for real-time quality control of in-situ temperature and salinity data Version 1.0. Integrated Ocean Observing System", pp.55~ (2014), Silver Spring
- "IOOS QC" (2021)[URL]
- Kim YS, "A Yellow Sea monitoring system and its scientific applications", Front Mar Sci, 6, pp.601~ (2019)[DOI]
- NDBC, "Handbook of automated data quality control checks and procedures", National Data Buoy Center, 78~ (2009)
- "Data product specification for gradient test, version 1-01. Consortium for Ocean Leadership" (2012)[URL]
- "Data product specification for spike test, version 1-01. Consortium for Ocean Leadership" (2012)[URL]
- "Protocols and procedures for OOI data products:QA, QC, calibration, physical samples, version 1-22. Consortium for Ocean Leadership, 1102-00300" (2013)[URL]
- "Automated QC Algorithms" (2021)[URL]
- UNESCO, "Manual of quality control procedures for validation of oceanographic data", IOC, 25~ (1993)
- Weller RA, "The challenge of sustaining ocean observations", Front Mar Sci, 6, pp.105~ (2019)[DOI]
- "Guide to the global observing system, WMONO" (2010)[URL]
- Zhou Y, "A data quality control method for seafloor observatories: the application of observed time series data in the East China Sea", Sensors, 18(8), pp.2628~ (2018)[DOI]