TRANSFORMER WITH LAGGED FEATURES FOR HANDLING LONG-TERM DATA DEPENDENCY IN TIME SERIES FORECASTING

Abstract

Data with long-term dependencies plays an important role in time series forecasting. However, studying data with long-term dependencies in time series data presents challenges for most algorithms. While some algorithms can forecast time series data, not all can model data with long-term dependencies effectively. The algorithm typically used for forecasting data with long-term dependencies is Long Short-Term Memory (LSTM), but LSTM can still face vanishing gradient issues, making it difficult to identify long-term dependencies in very long datasets. Another algorithm used for forecasting long-term time series data is the transformer. However, this algorithm has not yet shown better performance compared to simple linear models. The goal of this research is to develop an effective solution for forecasting time series data with long-term dependencies. The approach proposed in this research is the transformer with lagged features and also using time series cross-validation techniques. The results of this study show the performance of the transformer model in MAPE per fold on the BBCA stock dataset with a lag=5 and fold=5 configuration as follows: 0.0390, 0.0329, 0.0207, 0.0554, 0.0423. On the USD/IDR exchange rate dataset, the results are 0.0273, 0.0431, 0.0498, 0.0236, 0.237. The results of each fold are inconsistent and show unstable performance, indicating that the transformer with lagged features and using time series cross-validation techniques has not yet been able to provide its best performance in long-term time series forecasting.