ProphetNet 모델을 활용한 시계열 데이터의 열화 패턴 기반 Health Index 연구
A Study on the Health Index Based on Degradation Patterns in Time Series Data Using ProphetNet Model
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The Fourth Industrial Revolution and sensor technology have led to increased utilization of sensor data. In our modern society, data complexity is rising, and the extraction of valuable information has become crucial with the rapid changes in information technology (IT). Recurrent neural networks (RNN) and long short-term memory (LSTM) models have shown remarkable perform- ance in natural language processing (NLP) and time series prediction. Consequently, there is a strong expectation that models excelling in NLP will also excel in time series prediction. However, current research on Transformer models for time series prediction remains limited. Traditional RNN and LSTM models have demonstrated superior performance compared to Transformers in big data analysis. Nevertheless, with continuous advancements in Transformer models, such as GPT-2 (Generative Pre-trained Transformer 2) and ProphetNet, they have gained attention in the field of time series prediction. This study aims to evaluate the classification performance and interval prediction of remaining useful life (RUL) using an advanced Transformer model. The performance of each model will be utilized to establish a health index (HI) for cutting blades, enabling real-time monitoring of machine health. The results are expected to provide valuable insights for machine monitoring, evaluation, and management, confirming the effectiveness of advanced Transformer models in time series analysis when applied in industrial settings.
Abstract
The Fourth Industrial Revolution and sensor technology have led to increased utilization of sensor data. In our modern society, data complexity is rising, and the extraction of valuable information has become crucial with the rapid changes in information technology (IT). Recurrent neural networks (RNN) and long short-term memory (LSTM) models have shown remarkable perform- ance in natural language processing (NLP) and time series prediction. Consequently, there is a strong expectation that models excelling in NLP will also excel in time series prediction. However, current research on Transformer models for time series prediction remains limited. Traditional RNN and LSTM models have demonstrated superior performance compared to Transformers in big data analysis. Nevertheless, with continuous advancements in Transformer models, such as GPT-2 (Generative Pre-trained Transformer 2) and ProphetNet, they have gained attention in the field of time series prediction. This study aims to evaluate the classification performance and interval prediction of remaining useful life (RUL) using an advanced Transformer model. The performance of each model will be utilized to establish a health index (HI) for cutting blades, enabling real-time monitoring of machine health. The results are expected to provide valuable insights for machine monitoring, evaluation, and management, confirming the effectiveness of advanced Transformer models in time series analysis when applied in industrial settings.
참고문헌 (29)
- Abiodun, O. I., "Comprehensive review of artificial neural network applications to pattern recog-nition", IEEE Access, 7, pp.158820~158846 (2019)
- Ahn, H. J., "Development of Artificial Intelligence-Based Anomaly Detection Method for Aerospace Systems", Korea Aerospace Research Institute (2019)
- Alqudsi, A., "Application of machine learning in transformer health index prediction", Energies, 12(14), pp.2694~ (2019)
- Bohatyrewicz, P., "The analysis of power transformer population working in different operating conditions with the use of health index", Energies, 14(16), pp.5213~ (2021)
- Chandar, S., "Towards non-saturating recurrent units for modelling long-term dependencies", Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), pp.3280~3287 (2019)
- Chen, G., "A gentle tutorial of recurrent neural network with error backpropagation", arXiv preprint arXiv, 2016, 1610.02583, ~ (2016)
- Fang, W., "Survey on research of RNN-based spatio-temporal sequence prediction algo-rithms", Journal on Big Data, 3(3), pp.97~ (2021)
- Jahromi, A., "An approach to power transformer asset management using health index", IEEE Electrical Insulation Magazine, 25(2), pp.20~34 (2009)
- Jana, R. K., "Taming energy and electronic waste generation in bitcoin mining: Insights from Facebook prophet and deep neural network", Technological Forecasting and Social Change, 178, pp.121584~ (2022)
- Jang, S. M., "Comparison of Transformer and LSTM for threat detection and traffic prediction on long time-series data", Journal of Korean Institute of Next Generation Computing, pp.18~21 (2021)
- 진영훈, "주성분 분석 기법을 활용한 시계열 데이터 분석 및 예측 시스템", 한국융합학회논문지, 12(11), pp.99~107 (2021), 한국융합학회[DOI]
- 정인곤, "구조적 변화 감지 과정이 포함된 페어트레이딩 알고리즘의 성과분석", 한국경영과학회지, 42(3), pp.13~24 (2017), 한국경영과학회[DOI]
- 김유선, "시계열 이미징 기법을 이용한 다변량 센서 데이터 분류", 정보과학회논문지, 49(8), pp.593~600 (2022), 한국정보과학회[DOI]
- Lim, J., "Performance comparison of network anomaly detection using BERT, LSTM and GRU,", Journal of Korean Institute of Communications and Information Sciences, pp.1268~1269 (2022)
- Mizuno, T., "Generation of individual daily trajectories by GPT-2", Frontiers in Physics, pp.1118~ (2022)
- Murugan, R., "Understanding the pow-er transformer component failures for health index-based maintenance planning in electric utilities", Engineering Failure Analysis, 96, pp.274~288 (2019)
- Naderian, A., "An approach to determine the health index of power transformers", Conference Record of the 2008 IEEE International Symposium on Electrical Insulation, pp.192~196 (2008)
- Oh, H-W., "A Study on the Small Motion Classification Model based on Time Serial Data", Journal of Korean Institute of Communications and Information Sciences, pp.949~950 (2022)
- Qi, W., "Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training", arXiv pre-print arXiv, 2020, 2001.04063, ~ (2001)
- Rediansyah, D., "Artificial intelligence-based power transformer health in-dex for handling data uncertainty", IEEE Access, 9, pp.150637~150648 (2021)
- Sandoval, R. M., "Optimal policy derivation for trans-mission duty-cycle constrained LPWAN", IEEE Internet of Things Journal, 5(4), pp.3114~3125 (2018)
- Saqlain, M., "A voting ensemble classifier for wafer map defect patterns identi-fication in semiconductor manufacturing", IEEE Transac-tions on Semiconductor Manufacturing, 32(2), pp.171~182 (2019)
- Sherstinsky, A., "Fundamentals of recurrent neural net-work (RNN) and long short-term memory (LSTM) net-work", Physica D: Nonlinear Phenomena, 404, pp.132306~ (2020)
- Tan, C., "Deep Reinforcement Learning with Copy-ori-ented Context Awareness and Weighted Rewards for Abstractive Summarizatio", Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning, pp.84~89 (2023)
- Xiao, J., "Research progress of RNN language model", 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), pp.1285~1288 (2020)
- Yi, D-H., "Classification of Building Energy Pattern based on Robust Regression and Time series clustering", Journal of the Architectural Institute of Korea, 40(2), pp.392~393 (2020)
- 유인진, "시계열 군집분석을 통한 디지털 음원의 순위 변화 패턴 분류", 지능정보연구, 26(3), pp.171~191 (2020), 한국지능정보시스템학회
- Yook, H. N., "Comprehensive Studies on Reliability of Cutting Blade According to the Material of Knife Mill Used in Biomass Pretreatment Process", Transactions of the Korean Society of Mechanical Engineers, pp.171~ (2022)
- Zhang, M., "A commentary of GPT-3 in MIT Technology Review 2021", Fundamental Research, 1(6), pp.831~833 (2021)