Built independently by an author, for readers. Read the story and support ChapterPal

keyword

Progressive decomposition architecture

A progressive decomposition architecture is a neural network design for time series forecasting that integrates series decomposition directly into the internal layers of a deep learning model rather than applying it solely as an initial preprocessing step. Within this framework, complex temporal signals and their intermediate feature representations are iteratively separated across successive layers into distinct underlying components, typically isolating long-term trend-cyclical movements from periodic seasonal fluctuations. By repeatedly decoupling and refining these patterns throughout multiple stages of the network, the architecture reduces interference between conflicting temporal scales, simplifies the discovery of long-range dependencies, and allows specialized model components to handle different temporal characteristics more effectively.

1 item

Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long

OrganizationsSchool of SoftwareTsinghua University

Why you should read this

Presents Autoformer, a deep architecture that embeds progressive series decomposition and a periodicity-based Auto-Correlation mechanism to overcome the efficiency and accuracy bottlenecks of self-attention in long-term time series forecasting.

Extending the forecasting time is a critical demand for real applications, such as extreme weather early warning and long-term energy consumption planning. This paper studies the long-term forecasting problem of time series. Prior Transformer-based models adopt various self-attention mechanisms to discover the long-range dependencies. However, intricate temporal patterns of the long-term future prohibit the model from finding reliable dependencies. Also, Transformers have to adopt the sparse versions of point-wise self-attentions for long series efficiency, resulting in the information utilization bottleneck. Going beyond Transformers, we design Autoformer as a novel decomposition architecture with an Auto-Correlation mechanism. We break with the pre-processing convention of series decomposition and renovate it as a basic inner block of deep models. This design empowers Autoformer with progressive decomposition capacities for complex time series. Further, inspired by the stochastic process theory, we design the Auto-Correlation mechanism based on the series periodicity, which conducts the dependencies discovery and representation aggregation at the sub-series level. Auto-Correlation outperforms self-attention in both efficiency and accuracy. In long-term forecasting, Autoformer yields state-of-the-art accuracy, with a 38% relative improvement on six benchmarks, covering five practical applications: energy, traffic, economics, weather and disease. Code is available at this repository: \url{this https URL}.

Added

2026-09-09