Back

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

Guokun LaiWei-Cheng ChangYiming YangHanxiao Liu

article2017SIGIR2,965 citations

Introduces LSTNet, a deep learning architecture that integrates convolutional and recurrent neural networks with an autoregressive model to capture both short-term local dependencies and long-term trends while addressing scale insensitivity in multivariate time series forecasting.

Listen

The paper introduces a deep neural network called LSTNet to improve multivariate time series forecasting when data contain mixtures of short-term local dependencies and long-term repeating patterns. Such data appear in practical settings such as hourly freeway occupancy, solar power output, electricity demand, and daily exchange rates, where existing linear models (autoregressive integrated moving average, vector autoregression, Gaussian processes) and basic recurrent networks often miss one or both pattern types and produce large errors at longer forecast horizons.

The authors built LSTNet by stacking a convolutional layer to extract local variable interactions, a gated recurrent unit layer augmented with either fixed skip connections or a learned temporal attention mechanism to capture periodic structure over many time steps, and a parallel linear autoregressive component to correct for abrupt scale shifts that pure neural networks handle poorly. They trained the model with standard squared or absolute loss on four public benchmark series, using held-out validation data to select hyperparameters, and compared it against eight strong baselines across forecast horizons of 3 to 24 steps.

On the three series that exhibit clear daily or weekly repetition, both LSTNet variants reduced root relative squared error by 9–22 percent relative to the strongest recurrent baseline at the longest horizon and produced the best or second-best result in 24 of 36 metric–dataset–horizon combinations. The autoregressive bypass proved essential for robustness when input magnitude changed suddenly. On the exchange-rate series, which lacks strong periodicity, LSTNet performed comparably to the best linear models rather than degrading.

These gains translate directly into more reliable advance warnings for traffic congestion, solar plant scheduling, and grid load balancing, especially when decisions must be made many hours or days ahead. The linear bypass also reduces the risk that sudden scale shifts will render neural forecasts unusable in production.

The main limitations are that the skip length must still be chosen or tuned from data and that the model offers no automatic way to exploit additional metadata attached to the series. Future work could therefore focus on learning the skip length and on incorporating side information. Overall the empirical evidence is consistent and substantial on periodic data, giving decision makers a practical reason to adopt or pilot LSTNet where forecast accuracy at extended horizons materially affects operations or cost.

Carnegie Mellon Universityconvolutional neural networksdeep learningelectricity-consumptionLong‑and‑Short‑Term Temporal PatternMultivariate Time SeriesLSTNet
arXiv: 1703.07015laiguokun/LSTNet
Cover for Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

Abstract

Multivariate time series forecasting is an important machine learning problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation. Temporal data arise in these real-world applications often involves a mixture of long-term and short-term patterns, for which traditional approaches such as Autoregressive models and Gaussian Process may fail. In this paper, we proposed a novel deep learning framework, namely Long- and Short-term Time-series network (LSTNet), to address this open challenge. LSTNet uses the Convolution Neural Network (CNN) and the Recurrent Neural Network (RNN) to extract short-term local dependency patterns among variables and to discover long-term patterns for time series trends. Furthermore, we leverage traditional autoregressive model to tackle the scale insensitive problem of the neural network model. In our evaluation on real-world data with complex mixtures of repetitive patterns, LSTNet achieved significant performance improvements over that of several state-of-the-art baseline methods. All the data and experiment codes are available online.

Topics

convolutional neural networksdeep learningelectricity-consumptiongaussian processesgaussiansrecurrent neural networksfeedforward neural networksnetwork architecturelong short term memory neural networks

Keywords

Long‑and‑Short‑Term Temporal PatternMultivariate Time SeriesLSTNetConvolutional Neural NetworkRecurrent Neural NetworkAutoregressive ModelRecurrent‑Skip ComponentTemporal Attention LayerGaussian ProcessVector Autoregression

Table of Contents

  • 1 Introduction
  • 2 Related Background
  • 3 Framework
  • 3.1 Problem Formulation
  • 3.2 Convolutional Component
  • 3.3 Recurrent Component
  • 3.4 Recurrent-skip Component
  • 3.5 Temporal Attention Layer
  • 3.6 Autoregressive Component
  • 3.7 Objective function
  • 3.8 Optimization Strategy
  • 4 Evaluation
  • 4.1 Methods for Comparison
  • 4.2 Metrics
  • 4.3 Data
  • 4.4 Experimental Details
  • 4.5 Main Results
  • 4.6 Ablation Study
  • 4.7 Mixture of long- and short-term patterns
  • 5 Conclusion
  • References

Knowls

  1. Knowl 1 — Long- and Short-term Time-series Network Architecture

    model/method

    The Long- and Short-term Time-series Network (LSTNet) is a deep learning architecture designed for multivariate time series forecasting that simultaneously models short-term local dependencies, long-term periodic patterns, and non-periodic scale variations. Given historical multivariate signals XT={y1,…,yT}∈Rn×TX_T = \{y_1, \dots, y_T\} \in \mathbb{R}^{n \times T} with nn variables over TT time stamps, LSTNet combines two parallel components:

    1. Non-linear Neural Network Component: An input matrix passes through a Convolutional Neural Network (CNN) without pooling to extract local temporal patterns and cross-variable dependencies. The resulting feature representations are fed simultaneously into a standard Recurrent Neural Network (RNN) with Gated Recurrent Units (GRU) to capture general temporal dynamics, and either a Recurrent-skip layer or a Temporal Attention layer to capture repeating periodic patterns. The outputs of these recurrent pathways are combined via a dense layer to produce a non-linear hidden prediction vector htD∈Rnh_t^D \in \mathbb{R}^n.

    2. Linear Autoregressive (AR) Component: In parallel to the neural component, a classical linear AR model operates on an input window qarq_{ar} to output a baseline linear prediction htL∈Rnh_t^L \in \mathbb{R}^n, which directly addresses the scale-insensitivity of neural activation functions during non-periodic scale changes.

    The final forecasting output Y^t∈Rn\hat{Y}_t \in \mathbb{R}^n at forecast step tt is the element-wise sum of the neural and linear components: Y^t=htD+htL\hat{Y}_t = h_t^D + h_t^L

  2. Knowl 2 — Recurrent-Skip Component and Dense Layer Fusion

    model/method

    To alleviate gradient vanishing and capture very long-term periodic patterns (such as 24-hour daily cycles) that standard GRU and LSTM units fail to optimize over long sequences, LSTNet introduces a Recurrent-skip layer. Skip-links connect the current recurrent cell to hidden states in the same periodic phase at distance pp steps back.

    The update equations for the skip-GRU at time step tt given convolutional feature input xtx_t, reset gate rtr_t, update gate utu_t, candidate state ctc_t, and sigmoid function σ\sigma are: rt=σ(xtWxr+ht−pWhr+br)r_t = \sigma(x_t W_{xr} + h_{t-p} W_{hr} + b_r) ut=σ(xtWxu+ht−pWhu+bu)u_t = \sigma(x_t W_{xu} + h_{t-p} W_{hu} + b_u) ct=RELU(xtWxc+rt⊙(ht−pWhc)+bc)c_t = \text{RELU}(x_t W_{xc} + r_t \odot (h_{t-p} W_{hc}) + b_c) ht=(1−ut)⊙ht−p+ut⊙cth_t = (1 - u_t) \odot h_{t-p} + u_t \odot c_t where ⊙\odot denotes element-wise multiplication and RELU(z)=max⁡(0,z)\text{RELU}(z) = \max(0, z) is used as the activation function for hidden updates to improve gradient flow.

    A dense layer fuses the hidden state of the standard recurrent component htRh_t^R with the pp most recent hidden states of the recurrent-skip component {ht−p+1S,…,htS}\{h_{t-p+1}^S, \dots, h_t^S\}: htD=WRhtR+∑i=0p−1WiSht−iS+bh_t^D = W^R h_t^R + \sum_{i=0}^{p-1} W_i^S h_{t-i}^S + b where WRW^R, WiSW_i^S, and bb are learnable weight matrices and bias parameters, and htDh_t^D is the resulting neural prediction vector.

  3. Knowl 3 — Autoregressive Linear Bypass for Scale Insensitivity

    model/method

    A structural weakness of nonlinear deep neural networks (using activations like ReLU or sigmoid) in time series forecasting is scale insensitivity: predictions cannot easily scale proportionally when the input signal exhibits sudden, non-periodic scale changes or level shifts.

    To overcome this, LSTNet incorporates a linear Autoregressive (AR) bypass component in parallel to the neural network. For an input window of length qarq_{ar} over multivariate series yy, the linear forecast for variable dimension i∈{1,…,n}i \in \{1, \dots, n\} at time tt is: ht,iL=∑k=0qar−1Wkaryt−k,i+barh_{t,i}^L = \sum_{k=0}^{q_{ar}-1} W_k^{ar} y_{t-k, i} + b^{ar} where War∈RqarW^{ar} \in \mathbb{R}^{q_{ar}} and bar∈Rb^{ar} \in \mathbb{R} are autoregressive coefficients shared across all nn variable dimensions. Integrating htLh_t^L with the neural network prediction htDh_t^D via Y^t=htD+htL\hat{Y}_t = h_t^D + h_t^L allows the linear path to track baseline signal amplitude while the deep network models complex nonlinear oscillations.

  4. Knowl 4 — Convolutional Component for Local Temporal and Variable Dependencies

    model/method

    The first processing layer of LSTNet is a Convolutional Neural Network (CNN) without pooling designed to extract short-term temporal features and cross-variable correlations.

    Given an input window X∈Rn×TX \in \mathbb{R}^{n \times T} where nn is the number of time series variables and TT is the window length, the convolutional layer applies dcd_c filters. Each filter k∈{1,…,dc}k \in \{1, \dots, d_c\} has height nn (spanning all variables) and width ω\omega (spanning a short temporal window), producing a feature vector hkh_k: hk=RELU(Wk∗X+bk)h_k = \text{RELU}(W_k * X + b_k) where ∗* is the convolution operation, WkW_k and bkb_k are filter weights and bias, and RELU(x)=max⁡(0,x)\text{RELU}(x) = \max(0, x). Left zero-padding is applied to the input matrix XX so that each output vector hkh_k maintains length TT. The layer outputs a representation of shape dc×Td_c \times T that is passed directly to the recurrent layers without downsampling.

  5. Knowl 5 — Temporal Attention Layer for Dynamic and Non-Seasonal Periodicities

    model/method

    For time series where the periodicity pp is unknown, dynamic, or absent, the fixed skip length pp of the Recurrent-skip layer is suboptimal. As an alternative, LSTNet provides a temporal attention layer that computes a dynamic weighted combination of historical hidden representations.

    Given the matrix of column-stacked RNN hidden states over an input window of length qq, HtR=[ht−qR,…,ht−1R]∈Rd×qH_t^R = [h_{t-q}^R, \dots, h_{t-1}^R] \in \mathbb{R}^{d \times q}, attention weights αt∈Rq\alpha_t \in \mathbb{R}^q are computed based on the similarity between historical states and the last hidden representation ht−1Rh_{t-1}^R: αt=AttnScore(HtR,ht−1R)\alpha_t = \text{AttnScore}(H_t^R, h_{t-1}^R) where AttnScore\text{AttnScore} is a similarity function (such as dot product, cosine similarity, or a multilayer perceptron). The context vector ct=HtRαtc_t = H_t^R \alpha_t is concatenated with ht−1Rh_{t-1}^R and linearly projected to form the final neural prediction: htD=W[ct;ht−1R]+bh_t^D = W [c_t; h_{t-1}^R] + b

  6. Knowl 6 — Multivariate Time Series Forecasting Problem Formulation and Objectives

    definition

    Given a multivariate time series Y={y1,y2,…,yT}Y = \{y_1, y_2, \dots, y_T\} where each observation yt∈Rny_t \in \mathbb{R}^n has variable dimension nn, rolling forecasting aims to predict future signals yT+hy_{T+h} at a look-ahead horizon h≥1h \ge 1 using historical observations XT={yT−q+1,…,yT}∈Rn×qX_T = \{y_{T-q+1}, \dots, y_T\} \in \mathbb{R}^{n \times q} across an input window qq.

    LSTNet supports two alternative training loss objectives over the training time stamps ΩTrain\Omega_{\text{Train}}:

    1. Squared Loss (L2L_2-loss): min⁡Θ∑t∈ΩTrain∥Yt−Y^t−h∥F2\min_{\Theta} \sum_{t \in \Omega_{\text{Train}}} \|Y_t - \hat{Y}_{t-h}\|_F^2 where ∥⋅∥F\|\cdot\|_F is the Frobenius norm and Θ\Theta represents model parameters.

    2. Absolute Loss (L1L_1-loss) (derived from Support Vector Regression with ϵ=0\epsilon = 0 for robustness to anomalies and outliers): min⁡Θ∑t∈ΩTrain∑i=0n−1∣Yt,i−Y^t−h,i∣\min_{\Theta} \sum_{t \in \Omega_{\text{Train}}} \sum_{i=0}^{n-1} |Y_{t,i} - \hat{Y}_{t-h, i}|

    The optimal objective function is selected based on validation set performance.

  7. Knowl 7 — Time Series Forecasting Evaluation Metrics: RSE and CORR

    definition

    The performance of multivariate time series forecasting models is evaluated using two scale-independent metrics:

    1. Root Relative Squared Error (RSE): A normalized root mean squared error scaled by the variance of the true target data, making it invariant to scale (lower values indicate superior accuracy): RSE=∑(i,t)∈ΩTest(Yit−Y^it)2∑(i,t)∈ΩTest(Yit−mean(Y))2\text{RSE} = \frac{\sqrt{\sum_{(i,t) \in \Omega_{\text{Test}}} (Y_{it} - \hat{Y}_{it})^2}}{\sqrt{\sum_{(i,t) \in \Omega_{\text{Test}}} (Y_{it} - \text{mean}(Y))^2}}

    2. Empirical Correlation Coefficient (CORR): Measures the average Pearson correlation coefficient between predicted and true signals across all nn variable dimensions (higher values indicate superior accuracy): CORR=1n∑i=1n∑t(Yit−mean(Yi))(Y^it−mean(Y^i))∑t(Yit−mean(Yi))2∑t(Y^it−mean(Y^i))2\text{CORR} = \frac{1}{n} \sum_{i=1}^n \frac{\sum_t (Y_{it} - \text{mean}(Y_i))(\hat{Y}_{it} - \text{mean}(\hat{Y}_i))}{\sqrt{\sum_t (Y_{it} - \text{mean}(Y_i))^2 \sum_t (\hat{Y}_{it} - \text{mean}(\hat{Y}_i))^2}} where Y,Y^∈Rn×TY, \hat{Y} \in \mathbb{R}^{n \times T} are the ground truth and forecasted signal matrices, respectively.

  8. Knowl 8 — Evaluation of LSTNet and Baselines Across Benchmark Datasets

    data/table

    LSTNet variants (LST-Skip and LST-Attn) were evaluated against seven baselines: Autoregressive (AR), Vector AR with L2 regularization (LRidge), Vector AR with SVR objective (LSVR), Temporal Regularized Matrix Factorization (TRMF), Gaussian Process (GP), Vector AR combined with MLP (VAR-MLP), and RNN with GRU (RNN-GRU). Datasets were chronologically split into 60% training, 20% validation, and 20% testing across forecasting horizons h∈{3,6,12,24}h \in \{3, 6, 12, 24\}.

    Dataset Solar-Energy Traffic Electricity Exchange-Rate
    Horizon 3 6 12 24 3 6 12 24 3 6 12 24 3 6 12 24
    AR RSE 0.2435 0.3790 0.5911 0.8699 0.5991 0.6218 0.6252 0.6293 0.0995 0.1035 0.1050 0.1054 0.0228 0.0279 0.0353 0.0445
    CORR 0.9710 0.9263 0.8107 0.5314 0.7752 0.7568 0.7544 0.7519 0.8845 0.8632 0.8591 0.8595 0.9734 0.9656 0.9526 0.9357
    LRidge RSE 0.2019 0.2954 0.4832 0.7287 0.5833 0.5920 0.6148 0.6025 0.1467 0.1419 0.2129 0.1280 0.0184 0.0274 0.0419 0.0675
    CORR 0.9807 0.9568 0.8765 0.6803 0.8038 0.8051 0.7879 0.7862 0.8890 0.8594 0.8003 0.8806 0.9788 0.9722 0.9543 0.9305
    LSVR RSE 0.2021 0.2999 0.4846 0.7300 0.5740 0.6580 0.7714 0.5909 0.1523 0.1372 0.1333 0.1180 0.0189 0.0284 0.0425 0.0662
    CORR 0.9807 0.9562 0.8764 0.6789 0.7993 0.7267 0.6711 0.7850 0.8888 0.8861 0.8961 0.8891 0.9782 0.9697 0.9546 0.9370
    TRMF RSE 0.2473 0.3470 0.5597 0.9005 0.6708 0.6261 0.5956 0.6442 0.1802 0.2039 0.2186 0.3656 0.0351 0.0875 0.0494 0.0563
    CORR 0.9703 0.9418 0.8475 0.5598 0.6964 0.7430 0.7748 0.7278 0.8538 0.8424 0.8304 0.7471 0.9142 0.8123 0.8993 0.8678
    GP RSE 0.2259 0.3286 0.5200 0.7973 0.6082 0.6772 0.6406 0.5995 0.1500 0.1907 0.1621 0.1273 0.0239 0.0272 0.0394 0.0580
    CORR 0.9751 0.9448 0.8518 0.5971 0.7831 0.7406 0.7671 0.7909 0.8670 0.8334 0.8394 0.8818 0.8713 0.8193 0.8484 0.8278
    VAR-MLP RSE 0.1922 0.2679 0.4244 0.6841 0.5582 0.6579 0.6023 0.6146 0.1393 0.1620 0.1557 0.1274 0.0265 0.0304 0.0407 0.0578
    CORR 0.9829 0.9655 0.9058 0.7149 0.8245 0.7695 0.7929 0.7891 0.8708 0.8389 0.8192 0.8679 0.8609 0.8725 0.8280 0.7675
    RNN-GRU RSE 0.1932 0.2628 0.4163 0.4852 0.5358 0.5522 0.5562 0.5633 0.1102 0.1144 0.1183 0.1295 0.0192 0.0264 0.0408 0.0626
    CORR 0.9823 0.9675 0.9150 0.8823 0.8511 0.8405 0.8345 0.8300 0.8597 0.8623 0.8472 0.8651 0.9786 0.9712 0.9531 0.9223
    LST-Skip RSE 0.1843 0.2559 0.3254 0.4643 0.4777 0.4893 0.4950 0.4973 0.0864 0.0931 0.1007 0.1007 0.0226 0.0280 0.0356 0.0449
    CORR 0.9843 0.9690 0.9467 0.8870 0.8721 0.8690 0.8614 0.8588 0.9283 0.9135 0.9077 0.9119 0.9735 0.9658 0.9511 0.9354
    LST-Attn RSE 0.1816 0.2538 0.3466 0.4403 0.4897 0.4973 0.5173 0.5300 0.0868 0.0953 0.0984 0.1059 0.0276 0.0321 0.0448 0.0590
    CORR 0.9848 0.9696 0.9397 0.8995 0.8704 0.8669 0.8540 0.8429 0.9243 0.9095 0.9030 0.9025 0.9717 0.9656 0.9499 0.9339

    On datasets with distinct periodicities (Solar-Energy, Traffic, Electricity), LSTNet-Skip achieved 17 best results and LSTNet-Attn achieved 7 best results across metrics. When h=24h=24, LSTNet reduced RSE compared to the strongest neural baseline (RNN-GRU) by 9.2% on Solar-Energy, 11.7% on Traffic, and 22.2% on Electricity. On the non-periodic Exchange-Rate dataset, linear models (AR, LRidge) performed best because no recurring patterns exist to leverage.

  9. Knowl 9 — Ablation Analysis of LSTNet Components and Sensitivity to Scale Shifts

    empirical result

    Ablation experiments evaluating LSTNet variants with equalized parameter counts establish the following findings:

    1. Autoregressive Component (LSTw/oARLSTw/oAR): Removing the AR linear bypass caused the largest performance degradation across Solar-Energy, Traffic, and Electricity datasets. A controlled simulation where an autoregressive process undergoes scale shifts via noise mean increments ϵ∼N(⌊t/T⌋μ0,1)\epsilon \sim \mathcal{N}(\lfloor t/T \rfloor \mu_0, 1) demonstrated that standard RNN models fail on test sets with shifting scale, whereas the linear AR component accurately adjusts to scale changes.

    2. Convolutional Component (LSTw/oCNNLSTw/oCNN): Removing CNN filters resulted in marked performance drops, confirming the benefit of capturing local cross-variable feature interactions prior to recurrent modeling.

    3. Recurrent-Skip / Attention Components (LSTw/oskipLSTw/oskip): Eliminating skip connections caused substantial degradation on datasets containing long-range periodicity (such as daily 24-hour cycles in Traffic and Electricity), where standard RNN units suffer from vanishing gradients over long horizons.

  10. Knowl 10 — Hyperparameter Dependency and Variable Uniformity Limitations in LSTNet

    limitation

    LSTNet has two primary operational limitations stated by the authors:

    1. Fixed Skip-Length Parameter pp: The Recurrent-skip module relies on a predefined, fixed period hyperparameter pp. While pp is easily set for data with known sampling frequencies (e.g., p=24p=24 for hourly traffic or electricity data), it requires heuristic manual grid-search on validation sets for datasets with less obvious or multi-scale repeating cycles.

    2. Homogeneous Variable Treatment: The convolutional layer applies identical filters uniformly across all nn variable dimensions, treating each time series channel symmetrically without incorporating domain-specific metadata or heterogeneous node attribute information.

Coverage note — None was omitted; all key architectural components (CNN, RNN-GRU, Skip-RNN, Attention, Linear AR bypass), objective functions, evaluation metrics, empirical results across datasets, ablation analyses, and stated limitations are fully represented.

References

  1. 1.D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.
  2. 2.G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
  3. 3.G. E. Box and D. A. Pierce. Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. Journal of the American statistical Association, 65(332):1509–1526, 1970.
  4. 4.L.-J. Cao and F. E. H. Tay. Support vector machine with adaptive parameters in financial time series forecasting. IEEE Transactions on neural networks, 14(6):1506–1518, 2003.
  5. 5.Z. Che, S. Purushotham, K. Cho, D. Sontag, and Y. Liu. Recurrent neural networks for multivariate time series with missing values. arXiv preprint arXiv:1606.01865, 2016.
  6. 6.J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  7. 7.J. Connor, L. E. Atlas, and D. R. Martin. Recurrent networks and narma modeling. In NIPS, pages 301–308, 1991.
  8. 8.S. Dasgupta and T. Osogami. Nonlinear dynamic boltzmann machines for time-series prediction. AAAI-17. Extended research report available at goo. gl/Vd0wna, 2016.
  9. 9.J. L. Elman. Finding structure in time. Cognitive science, 14(2):179–211, 1990.
  10. 10.R. Frigola, F. Lindsten, T. B. Schön, and C. E. Rasmussen. Bayesian inference and learning in gaussian process state-space models with particle mcmc. In Advances in Neural Information Processing Systems, pages 3156–3164, 2013.
  11. 11.R. Frigola-Alcade. Bayesian Time Series Learning with Gaussian Processes. PhD thesis, PhD thesis, University of Cambridge, 2015.
  12. 12.J. D. Hamilton. Time series analysis, volume 2. Princeton university press Princeton, 1994.
  13. 13.N. Y. Hammerla, S. Halloran, and T. Ploetz. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880, 2016.
  14. 14.G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82–97, 2012.
  15. 15.S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  16. 16.A. Jain and A. M. Kumar. Hybrid neural network models for hydrologic time series forecasting. Applied Soft Computing, 7(2):585–592, 2007.
  17. 17.K.-j. Kim. Financial time series forecasting using support vector machines. Neurocomputing, 55(1):307–319, 2003.
  18. 18.D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  19. 19.A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
  20. 20.C. Lea, R. Vidal, A. Reiter, and G. D. Hager. Temporal convolutional networks: A unified approach to action segmentation. In Computer Vision–ECCV 2016 Workshops, pages 47–54. Springer, 2016.
  21. 21.Y. LeCun and Y. Bengio. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10):1995, 1995.
  22. 22.J. Li and W. Chen. Forecasting macroeconomic time series: Lasso-based approaches and their forecast combinations with dynamic factor models. International Journal of Forecasting, 30(4):996–1015, 2014.
  23. 23.Z. C. Lipton, D. C. Kale, C. Elkan, and R. Wetzell. Learning to diagnose with lstm recurrent neural networks. arXiv preprint arXiv:1511.03677, 2015.
  24. 24.H. Lütkepohl. New introduction to multiple time series analysis. Springer Science & Business Media, 2005.
  25. 25.E. McKenzie. General exponential smoothing and the equivalent arma process. Journal of Forecasting, 3(3):333–344, 1984.
  26. 26.I. Melnyk and A. Banerjee. Estimating structured vector autoregressive model. arXiv preprint arXiv:1602.06606, 2016.
  27. 27.H. Qiu, S. Xu, F. Han, H. Liu, and B. Caffo. Robust estimation of transition matrices in high dimensional heavy-tailed vector autoregressive processes. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 1843–1851, 2015.
  28. 28.S. Roberts, M. Osborne, M. Ebden, S. Reece, N. Gibson, and S. Aigrain. Gaussian processes for time-series modelling. Phil. Trans. R. Soc. A, 371(1984):20110550, 2013.
  29. 29.R. K. Srivastava, K. Greff, and J. Schmidhuber. Highway networks. arXiv preprint arXiv:1505.00387, 2015.
  30. 30.V. Vapnik, S. E. Golowich, A. Smola, et al. Support vector method for function approximation, regression estimation, and signal processing. Advances in neural information processing systems, pages 281–287, 1997.
  31. 31.J. B. Yang, M. N. Nguyen, P. P. San, X. L. Li, and S. Krishnaswamy. Deep convolutional neural networks on multichannel time series for human activity recognition. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), Buenos Aires, Argentina, pages 25–31, 2015.
  32. 32.H.-F. Yu, N. Rao, and I. S. Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. In Advances in Neural Information Processing Systems, pages 847–855, 2016.
  33. 33.R. Yu, Y. Li, C. Shahabi, U. Demiryurek, and Y. Liu. Deep learning: A generic approach for extreme condition traffic forecasting. In Proceedings of the 2017 SIAM International Conference on Data Mining, pages 777–785. SIAM, 2017.
  34. 34.G. Zhang, B. E. Patuwo, and M. Y. Hu. Forecasting with artificial neural networks:: The state of the art. International journal of forecasting, 14(1):35–62, 1998.
  35. 35.G. P. Zhang. Time series forecasting using a hybrid arima and neural network model. Neurocomputing, 50:159–175, 2003.

Citation

MLA
Lai, G., et al. “Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks”. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018, pp. 95–104, https://doi.org/10.1145/3209978.3210006.
APA
Lai, G., Chang, W.-C., Yang, Y., & Liu, H. (2018). Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 95–104. https://doi.org/10.1145/3209978.3210006
Chicago
Lai, G., W.-C. Chang, Y. Yang, and H. Liu. 2018. “Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks”. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 95–104. https://doi.org/10.1145/3209978.3210006.
Harvard
Lai, G. et al. (2018) “Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks”, The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, pp. 95–104. Available at: https://doi.org/10.1145/3209978.3210006.
Vancouver
1. Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, pp 95–104

BibTeX

@inproceedings{Lai_2018, series={SIGIR ’18}, title={Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks}, url={http://dx.doi.org/10.1145/3209978.3210006}, DOI={10.1145/3209978.3210006}, booktitle={The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval}, publisher={ACM}, author={Lai, Guokun and Chang, Wei-Cheng and Yang, Yiming and Liu, Hanxiao}, year={2018}, month=June, pages={95–104}, collection={SIGIR ’18} }
Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF