TadGAN 기반 시계열 이상 탐지를 활용한 전처리 프로세스 연구
A Pre-processing Process Using TadGAN-based Time-series Anomaly Detection
1
조회
0
다운로드
10
인용
0
참고문헌
키워드
초록
Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.
Abstract
Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.
참고문헌 (32)
- Breiman, L, "Random forests", Machine learning, 45(1), pp.5~32 (2001)
- Carletti, M., "A deep learning approach for anomaly detection with industrial time series data : a refrigerators manufacturing case study", Procedia Manufacturing, 38, pp.233~240 (2019)
- Chang, K. B. G. N. A., "Learning Anomaly Detection for Generating Predictive Maintenance Models from LBS-AUV Mission Data", ~
- Chen, T., "Xgboost: A scalable tree boosting system", Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp.785~794 (2016)
- Cho, K., "arXiv preprint arXiv:1406.1078", arXiv preprint arXiv : 1406, ~ (2014)
- 최성현, "최적화 하이퍼 파라미터의 학습자 기반 배깅 모델을 활용한 태양광 출력 예측", 전기학회논문지, 69(7), pp.978~984 (2020), 대한전기학회
- Cook, A. A., "Anomaly detection for IoT time-series data : A survey", IEEE Internet of Things Journal, 7(7), pp.6481~6494 (2019)
- Geiger, A., "TadGAN : Time series anomaly detection using generative adversarial networks", 2020 IEEE International Conference on Big Data (Big Data), pp.33~43 (2020), IEEE
- Goodfellow, I., "Generative adversarial nets. Advances in neural information processing systems", Advances in neural information processing systems, 27 (2014)
- Hochreiter, S., "Long short-term memory", Neural computation, 9(8), pp.1735~1780 (1997)
- Hwang, J. H., "Anomaly Detection and Performance Analysis using Deep Learning", Proceedings of the Korean Institute of Information and Commucation Sciences Conference, pp.78~81 (2021), The Korea Institute of Information and Commucation Engineering
- Jiang, W., "A GAN-based anomaly detection approach for imbalanced industrial time series", IEEE Access, 7, pp.143608~143619 (2019)
- Jiao, Y., "TimeAutoAD : Autonomous Anomaly Detection With Self-Supervised Contrastive Loss for Multivariate Time Series", IEEE Transactions on Network Science and Engineering, 9(3), pp.1604~1619 (2022)
- 김현정, "준지도학습 기반의 P2P 대출 부도 위험 예측에 대한 연구", 디지털융복합연구, 20(4), pp.185~192 (2022), 한국디지털정책학회[DOI]
- 김희성, "수명 분포를 고려한 RUL 예측 파라미터 분포 및 유효성 입증에 관한 연구", 신뢰성 응용연구, 20(2), pp.145~153 (2020), 한국신뢰성학회[DOI]
- 이건희, "SVM을 이용한 기어펌프의 고장 분류", 신뢰성 응용연구, 20(2), pp.187~196 (2020), 한국신뢰성학회[DOI]
- 이승훈, "다구찌 실험계획법을 활용한 LSTM의 하이퍼파라미터 최적화에 관한 연구", 신뢰성 응용연구, 21(3), pp.238~245 (2021), 한국신뢰성학회[DOI]
- 이승훈, "광업 데이터의 시계열 분석을 통해 실리카 농도를 예측하기 위한 머신러닝 모델", 품질경영학회지, 48(3), pp.511~520 (2020), 한국품질경영학회[DOI]
- Li, Y., "DCT-GAN: Dilated Convolutional Transformer-based GAN for Time Series Anomaly Detection", IEEE Transactions on Knowledge and Data Engineering (2021)
- Meyer, P., "Network Anomaly Detection in Cars: A Case for Time-Sensitive Stream Filtering and Policing", arXiv preprint arXiv:2112.11109, ~ (2021)
- Nguyen, H. D., "Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management", International Journal of Information Management, 57, pp.102282~ (2021)
- 오민지, "제조 설비 이상탐지를 위한 지도학습 및비지도학습 모델 설계에 관한 연구", 한국빅데이터학회 학회지, 6(1), pp.23~35 (2021), 사)한국빅데이터학회[DOI]
- Oh, S., "Application TadGAN to Detect Collective Anomaly in Power Usage Data", The Journal of Contents Computing, 3(1), pp.297~306 (2021)
- 박형준, "다중 센서 데이터를 이용한 무인기 유성감속기의 고장진단 연구", 신뢰성 응용연구, 20(4), pp.332~342 (2020), 한국신뢰성학회[DOI]
- 박형준, "인공신경망을 이용한 고장 심각도에 따른 무인기용 유성감속기의 고장진단 연구", 신뢰성 응용연구, 21(4), pp.329~340 (2021), 한국신뢰성학회[DOI]
- Preuveneers, D., "Chained anomaly detection models for federated learning : An intrusion detection case study", Applied Sciences, 8(12), pp.2663~ (2018)
- Ramotsoela, D., "A survey of anomaly detection in industrial wireless sensor networks with critical water system infrastructure as a case study", Sensors, 18(8), pp.2491~ (2018)
- Rezapour, M, "Anomaly detection using unsupervised methods : credit card fraud case study", International Journal of Advanced Computer Science and Applications, 10(11) (2019)
- Song, B., "Narrative texts-based anomaly detection using accident report documents : The case of chemical process safety", Journal of Loss Prevention in the Process Industries, 57, pp.47~54 (2019)
- Srivastava, N., "Unsupervised learning of video representations using lstms", International conference on machine learning, pp.843~852 (2015)
- PMLR. TIPIRNENI, S., "Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series", ACM Trans. Knowl. Discov. Data, 1(1) (2022)
- Xu, J., "Anomaly transformer : Time series anomaly detection with association discrepancy", arXiv preprint arXiv:2110.02642, ~ (2021)