시계열 예측을 위한 스타일 기반 트랜스포머
Style-Based Transformer for Time Series Forecasting
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
시계열 예측은 과거 시점의 정보를 토대로 미래 시점의 정보를 예측하는 것을 말한다. 향후 시점의 정보를 정확하게 예측하는 것은 다양한분야 전략 수립, 정책 결정 등을 위해 활용되기 때문에 매우 중요하다. 최근에는 트랜스포머 모델이 시계열 예측 모델로서 주로 연구되고 있다. 그러나 기존의 트랜스포머의 모델은 예측 순차를 출력할 때 출력 결과를 다시 입력하는 자가회귀 구조로 되어 있다는 한계점이 있다. 이 한계점은멀리 떨어진 시점을 예측할 때 정확도가 떨어진다는 문제점을 초래한다. 본 논문에서는 이러한 문제점을 개선하고 더 정확한 시계열 예측을 위해스타일 변환 기법에 착안한 순차 디코딩 모델을 제안한다. 제안하는 모델은 트랜스포머-인코더에서 과거 정보의 특성을 추출하고, 이를 스타일-기반디코더에 반영하여 예측 시계열을 생성하는 구조로 되어 있다. 이 구조는 자가회귀 방식의 기존의 트랜스포머의 디코더 구조와 다르게, 예측 순차를한꺼번에 출력하기 때문에 더 먼 시점의 정보를 좀 더 정확히 예측할 수 있다는 장점이 있다. 서로 다른 데이터 특성을 가지는 다양한 시계열데이터셋으로 예측 실험을 진행한 결과, 본 논문에서 제시한 모델이 기존의 다른 시계열 예측 모델보다 예측 정확도가 우수하다는 것을 보인다.
Abstract
Time series forecasting refers to predicting future time information based on past time information. Accurately predicting futureinformation is crucial because it is used for establishing strategies or making policy decisions in various fields. Recently, a transformermodel has been mainly studied for a time series prediction model. However, the existing transformer model has a limitation in that ithas an auto-regressive structure in which the output result is input again when the prediction sequence is output. This limitation causesa problem in that accuracy is lowered when predicting a distant time point. This paper proposes a sequential decoding model focusingon the style transformation technique to handle these problems and make more precise time series forecasting. The proposed modelhas a structure in which the contents of past data are extracted from the transformer-encoder and reflected in the style-based decoderto generate the predictive sequence. Unlike the decoder structure of the conventional auto-regressive transformer, this structure has theadvantage of being able to more accurately predict information from a distant view because the prediction sequence is output all atonce. As a result of conducting a prediction experiment with various time series datasets with different data characteristics, it was shownthat the model presented in this paper has better prediction accuracy than other existing time series prediction models.
참고문헌 (25)
- A. Vaswani, "Attention is all you need", Advances in Neural Information Processing Systems (2017)
- S. Li, "Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting", Advances in Neural Information Processing Systems, 32, pp.5243~5253 (2019)
- L. Cai, "Traffic transformer : Capturing the continuity and periodicity of time series for traffic forecasting", Transactions in GIS, 24(3), pp.736~755 (2020)
- F. Giuliari, "Transformer networks for trajectory forecasting", 2020 25th International Conference on Pattern Recognition (ICPR). IEEE (2020)
- L. A. Gatys, "Image style transfer using convolutional neural networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
- B. L. Bowerman, "Time series and forecasting" (1979), Duxbury Press
- J. Contreras, "ARIMA models to predict next-day electricity prices", IEEE Transactions on Power Systems, 18(3), pp.1014~1020 (2003)
- A. I. McLeod, "Diagnostic checking of periodic autoregression models with application", Journal of Time Series Analysis, 15(2), pp.221~233 (1994)
- R. B. Darlington, "Regression analysis and linear models" (2017), Guilford
- J. T. Connor, "Recurrent neural networks and robust time series prediction", IEEE Transactions on Neural Networks, 5(2), pp.240~254 (1994)
- R. J. Frank, "Time series prediction and neural networks", Journal of Intelligent and Robotic Systems, 31(1), pp.91~103 (2001)
- A. B. Geva, "ScaleNet-multiscale neural-network architecture for time series prediction", IEEE Transactions on Neural Networks, 9(6), pp.1471~1482 (1998)
- A. Tokgöz, "A RNN based time series approach for forecasting turkish electricity load", 2018 26th Signal Processing and Communications Applications Conference (SIU), IEEE (2018)
- I. Koprinska, "Convolutional neural networks for energy time series forecasting", 2018 International Joint Conference on Neural Networks (IJCNN), IEEE (2018)
- C. -J. Huang, "A deep cnn-lstm model for particulate matter (PM2. 5) forecasting in smart cities", Sensors, 18(7), pp.2220~ (2018)
- J. Pamina, "Survey on deep learning algorithms", International Journal of Emerging Technology and Innovative Engineering, 5(1) (2019)
- J. J. Dabrowski, "ForecastNet: A time-variant deep feed-forward neural network architecture for multi-step-ahead time-series forecasting", International Conference on Neural Information Processing (2020), Springer
- Y. G. Cinar, "Position-based content attention for time series forecasting with sequence-to-sequence rnns", International Conference on neural Information Processing (2017), Springer
- Y. Lin, "SpringNet: Transformer and spring DTW for time series forecasting", International Conference on Neural Information Processing (2020), Springer
- N. Chen, "Non-autoregressive transformer for speech recognition", IEEE Signal Processing Letters, 28, pp.121~125 (2020)
- J. L. Ba, "Layer normalization", arXiv preprint arXiv:1607.06450, ~ (2016)
- I. Goodfellow, "Generative adversarial nets", Advances in Neural Information Processing Systems, 27 (2014)
- T. Karras, "A style-based generator architecture for generative adversarial networks", Proceedings of the IEEE, 2019, ~
- G. Lai, "Modeling long-and short-term temporal patterns with deep neural networks", The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (2018)
- D. P. Kingma, "Adam : A method for stochastic optimization", arXiv preprint arXiv:1412.6980, ~ (2014)