Built independently by an author, for readers. Read the story and support ChapterPal

keyword

Autoformer model

The Autoformer model is a deep learning architecture designed for long-term time series forecasting that enhances standard Transformer frameworks to better capture complex temporal patterns. Rather than relying on traditional point-wise self-attention mechanisms, it utilizes an auto-correlation mechanism that identifies dependencies and aggregates representations at the sub-series level based on inherent periodicity. To handle intricate temporal variations over extended prediction horizons, the architecture integrates progressive series decomposition blocks directly into its deep layers, continuously separating sequential data into distinct trend-cyclical and seasonal components for more accurate and computationally efficient multi-step predictions.

1 item

Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long

OrganizationsSchool of SoftwareTsinghua University

Why you should read this

Presents Autoformer, a deep architecture that embeds progressive series decomposition and a periodicity-based Auto-Correlation mechanism to overcome the efficiency and accuracy bottlenecks of self-attention in long-term time series forecasting.

Extending the forecasting time is a critical demand for real applications, such as extreme weather early warning and long-term energy consumption planning. This paper studies the long-term forecasting problem of time series. Prior Transformer-based models adopt various self-attention mechanisms to discover the long-range dependencies. However, intricate temporal patterns of the long-term future prohibit the model from finding reliable dependencies. Also, Transformers have to adopt the sparse versions of point-wise self-attentions for long series efficiency, resulting in the information utilization bottleneck. Going beyond Transformers, we design Autoformer as a novel decomposition architecture with an Auto-Correlation mechanism. We break with the pre-processing convention of series decomposition and renovate it as a basic inner block of deep models. This design empowers Autoformer with progressive decomposition capacities for complex time series. Further, inspired by the stochastic process theory, we design the Auto-Correlation mechanism based on the series periodicity, which conducts the dependencies discovery and representation aggregation at the sub-series level. Auto-Correlation outperforms self-attention in both efficiency and accuracy. In long-term forecasting, Autoformer yields state-of-the-art accuracy, with a 38% relative improvement on six benchmarks, covering five practical applications: energy, traffic, economics, weather and disease. Code is available at this repository: \url{this https URL}.

Added

2026-09-09