Adaptive Conformal Predictions for Time Series

Margaux ZaffranOlivier FéronYannig GoudeJulie JosseAymeric Dieuleveut

article2022ICML201 citations

Develops a parameter-free online expert aggregation algorithm that adapts conformal inference to dependent time series data, providing theoretical analysis of its efficiency and validated performance on electricity price forecasting.

Listen

Modern industrial and financial operations increasingly rely on complex machine learning models to forecast dynamic variables such as energy prices and market demand. However, deploying automated forecasts in high-stakes environments requires dependable uncertainty quantification to manage operational and financial risk. Traditional conformal prediction methods provide mathematically sound, distribution-free prediction intervals, but they rely on the assumption of exchangeable data—a condition routinely violated by real-world time series due to temporal dependencies, trends, and regime shifts. The article evaluates how Adaptive Conformal Inference (ACI), a framework designed to update coverage parameters sequentially, can be adapted to provide reliable and informative prediction intervals for dependent time series data.

To address this challenge, the authors mathematically analyzed how ACI's learning rate parameter affects interval width across both independent and autoregressive noise processes. Recognizing that manually tuning this parameter in production is difficult and error-prone, the authors introduced an automated method called Aggregated Adaptive Conformal Inference (AgACI). This parameter-free approach uses online expert aggregation to dynamically weight multiple learning rates based on past performance. The authors validated the framework through extensive synthetic benchmarks across varying degrees of temporal correlation and demonstrated its practical utility on a real-world case study forecasting hourly French electricity spot prices across four years (2016–2019) using random forest regression models.

The analysis produced several key findings. First, while adaptive updates slightly widen intervals when data points are completely independent, they substantially tighten interval widths in strongly autocorrelated time series when paired with an effective learning rate. Second, selecting an inappropriate static learning rate creates severe trade-offs: learning rates that are too small result in under-coverage during persistent forecast errors, while rates that are too large generate uninformative, infinite-width intervals. Third, the proposed AgACI algorithm consistently achieved the target 90% coverage rate (maintaining above 89.8% across all synthetic benchmarks) while achieving the narrowest intervals among valid competing methods. Finally, in the electricity market application, adaptive conformal methods adapted effectively to price volatility and sudden market spikes while maintaining tighter median interval bounds than non-adaptive baselines.

These findings mean that organizations can implement rigorous, distribution-free risk boundaries around their predictive models without assuming idealized statistical conditions. Adopting adaptive conformal methods reduces exposure to unexpected prediction errors in volatile markets, supporting better capital allocation, hedging, and operational planning. Decision-makers should consider adopting online adaptive conformal algorithms like AgACI over static split conformal methods for time series pipelines, as automated aggregation eliminates the need for manual parameter tuning while preserving coverage validity.

While the article establishes strong theoretical and empirical confidence in overall marginal coverage, leaders should exercise caution regarding localized sub-group performance. The electricity market experiments revealed that coverage varied systematically across calendar regimes, achieving approximately 93% coverage on midweek days but dropping to roughly 88% on weekends and Mondays. Organizations should conduct pilot evaluations to verify conditional coverage across operational sub-periods before full-scale deployment, and future analytical efforts should focus on refining conditional calibration for recurring temporal cycles.

Électricité de France IDESP INRIA Institut polytechnique de Paris Paris Dauphine University Université Paris-Saclay Adaptive Conformal Predictions Time Series Conformal Inference

arXiv: 2202.07282

Paper: A tutorial on conformal prediction, Glenn Shafer et al. (2007). Introduces the foundational framework and sequential error guarantees of conformal prediction that the source directly adapts for non-exchangeable time series.
Paper: Distribution-Free Predictive Inference for Regression, Jing Lei et al. (2016). Establishes distribution-free split conformal inference for regression models, which serves as the core static baseline compared and modified in the source.
Paper: Introduction to Online Convex Optimization, Elad Hazan (2016). Provides the essential theory of online convex optimization and gradient descent updates that underpin Adaptive Conformal Inference and online expert aggregation.
Paper: Quantile Regression Forests, Nicolai Meinshausen (2006). Details quantile regression forests, a foundational non-parametric methodology for conditional interval estimation applied in the source's empirical benchmarks.

Paper: Conformal Inference for Online Prediction with Arbitrary Distribution Shifts, Isaac Gibbs et al. (2024). Directly builds on adaptive conformal inference for non-stationary streams by introducing online step-size tuning to minimize local interval regret under arbitrary distribution shifts.

Cover for Adaptive Conformal Predictions for Time Series

Abstract

Uncertainty quantification of predictive models is crucial in decision-making problems. Conformal prediction is a general and theoretically sound answer. However, it requires exchangeable data, excluding time series. While recent works tackled this issue, we argue that Adaptive Conformal Inference (ACI, Gibbs & Candès, 2021), developed for distribution-shift time series, is a good procedure for time series with general dependency. We theoretically analyse the impact of the learning rate on its efficiency in the exchangeable and auto-regressive case. We propose a parameter-free method, AgACI, that adaptively builds upon ACI based on online expert aggregation. We lead extensive fair simulations against competing methods that advocate for ACI’s use in time series. We conduct a real case study: electricity price forecasting. The proposed aggregation algorithm provides efficient prediction intervals for day-ahead forecasting. All the code and data to reproduce the experiments are made available on GitHub.

Keywords

Adaptive Conformal Predictions Time Series Conformal Inference prediction intervals probabilistic forecasts conformity score distribution shifts temporal dependence homogeneous Markov Chain asymptotic validity

1. Introduction
2. Setting: ACI for time series
3. Impact of γ on ACI efficiency
3.1. Exchangeable case
3.2. AR(1) case
4. Adaptive strategies based on ACI
5. Numerical evaluation on synthetic data sets
5.1. Data generation process and settings
5.2. Impact of γ, performance of AgACI
5.3. Description of baseline methods
5.4. Experimental results: impact of φ, θ
6. Forecasting French electricity spot prices
6.1. Presentation of the price data
6.2. Price prediction with predictive intervals in 2019
7. Conclusion
Acknowledgements
References
Appendices
A. Details on Split Conformal Prediction
A.1. Split Conformal Prediction Algorithm
A.2. Illustration of the SCP procedure
A.3. Theoretical guarantees of Split Conformal Prediction
A.4. Examples of dependent scores when data noise is exchangeable
A.5. How should we visualise CP predicted intervals?
B. Proof of the results presented in Section 3 and additional numerical experiments
B.1. Proof of Theorem 3.1
B.2. Proof of Lemmas B.1 and B.2
B.3. Control of the first four moments: Lemmas B.3 to B.6
B.4. Proof of Theorem 3.3
B.5. Numerical study of ACI efficiency with AR(1) residuals, with respect to the median length
C. Experimental details.
C.1. Details on the BOA procedure
C.2. Details ARMA(1,1) processes
C.3. Random forest parameters
C.4. Details about the baselines and comparison
D. Additional experiments on synthetic data sets
D.1. Additional experimental results of ACI sensitivity to γ, presented in Section 5.2
D.2. Comparison to baselines, extension of Section 5.4
D.3. Closer look at infinite intervals
D.4. Randomised, sequential and other splits.
E. Forecasting French electricity spot prices
E.1. Details about the data set
E.2. Forecasting year 2019

Knowls

Knowl 1 — Adaptive Conformal Inference for Time Series
model/method
Adaptive Conformal Inference (ACI) adapts split conformal prediction to time series settings subject to non-exchangeability or distribution shifts. Given historical observations $((x_1, y_1), \dots, (x_{T_0}, y_{T_0})) \in (\mathbb{R}^d \times \mathbb{R})^{T_0}$ , predictions and predictive intervals are constructed sequentially for $t \in \{T_0+1, \dots, T_0+T_1\}$ . At step $t$ , the historical window $((x_{t-T_0}, y_{t-T_0}), \dots, (x_{t-1}, y_{t-1}))$ is partitioned sequentially into a proper training set $\mathrm{Tr}_t$ and a calibration set $\mathrm{Cal}_t$ , where $\mathrm{Tr}_t$ contains earlier indices and $\mathrm{Cal}_t$ contains the most recent indices.

A regression model $\hat{\mu}_t$ is fitted on $\mathrm{Tr}_t$ , and conformity scores on the calibration set are computed as absolute residuals $S_{\mathrm{Cal}_t} = \{s_i = |y_i - \hat{\mu}_t(x_i)| : i \in \mathrm{Cal}_t\}$ . A target miscoverage level $\alpha \in [0, 1]$ is dynamically adjusted through an effective miscoverage parameter $\alpha_t$ , initialized at $\alpha_1 = \alpha$ . At each time step $t \ge 1$ :
$\hat{C}_{\alpha_t}(x_t) = \left[ \hat{\mu}_t(x_t) \pm \hat{Q}_{1-\alpha_t}(S_{\mathrm{Cal}_t}) \right]$ $\alpha_{t+1} = \alpha_t + \gamma \left( \alpha - \mathbf{1}\{y_t \notin \hat{C}_{\alpha_t}(x_t)\} \right)$
where $\gamma \ge 0$ is a learning rate step size, $\hat{Q}_{1-\alpha_t}(S_{\mathrm{Cal}_t})$ denotes the $(1-\alpha_t)$ -empirical quantile of the calibration conformity scores, and $\mathbf{1}\{\cdot\}$ is the indicator function.

If $\alpha_t \le 0$ , by convention $\hat{Q}_{1-\alpha_t} = +\infty$ , yielding an infinite prediction interval $\hat{C}_{\alpha_t}(x_t) = \mathbb{R}$ . If $\alpha_t \ge 1$ , by convention $\hat{Q}_{1-\alpha_t} = 0$ , collapsing the interval to the point prediction $\hat{C}_{\alpha_t}(x_t) = \{\hat{\mu}_t(x_t)\}$ . If the interval covers $y_t$ , $\alpha_{t+1}$ increases, tightening the next interval; if it fails to cover, $\alpha_{t+1}$ decreases, widening the next interval.
Knowl 2 — Asymptotic Efficiency Degradation of ACI in Exchangeable Settings
theoretical result
In the ideal setting where data errors are exchangeable and the score quantile function $Q$ is perfectly known, Adaptive Conformal Inference (ACI) strictly degrades interval efficiency (increases expected interval length) compared to standard non-adaptive Split Conformal Prediction (SCP).

Let $\alpha \in \mathbb{Q} \cap (0, 1)$ be a rational target miscoverage rate. Assume the conformity scores $S_t = |Y_t - \hat{\mu}(X_t)|$ are exchangeable with a bounded and 4-times continuously differentiable quantile function $Q \in \mathcal{C}^4([0, 1])$ , and the empirical quantile estimator is exact: $\hat{Q}_{1-\hat{\alpha}}(S_{\mathrm{Cal}_t}) = Q(1 - \hat{\alpha})$ for all $\hat{\alpha}$ . Let $L(\alpha_t) = 2 Q(1 - \alpha_t)$ denote the length of the ACI prediction interval at step $t$ , with $Q(1-\hat{\alpha})$ extended by $Q(1)$ for $\hat{\alpha} < 0$ , and let $L_0 = 2 Q(1 - \alpha)$ denote the non-adaptive conformal interval length (corresponding to $\gamma = 0$ ).

For any learning rate $\gamma > 0$ , the sequence of effective miscoverage levels $(\alpha_t)_{t > 0}$ forms an irreducible Markov chain on a finite state space:
$\mathcal{A} = \left\{ \alpha + \gamma \frac{\gcd(q-p, p)}{q} \mathbb{Z} \right\} \cap \left] \gamma(\alpha - 1), 1 + \gamma\alpha \right[$
where $\alpha = p/q$ with $(p, q) \in \mathbb{N} \times \mathbb{N}^*$ , $p < q$ , and $\gcd$ denotes the greatest common divisor. The Markov chain admits a unique stationary distribution $\pi_\gamma$ , and the time-averaged interval length converges almost surely:
$\frac{1}{T} \sum_{t=1}^T L(\alpha_t) \xrightarrow[T \to +\infty]{\text{a.s.}} \mathbb{E}_{\tilde{\alpha} \sim \pi_\gamma} [L(\tilde{\alpha})].$
Furthermore, as the learning rate $\gamma \to 0$ , the stationary expected interval length satisfies:
$\mathbb{E}_{\pi_\gamma}[L] = L_0 + \frac{Q''(1 - \alpha)}{2} \gamma \alpha (1 - \alpha) + O(\gamma^{3/2}).$
For standard error distributions where the probability density function is decreasing in the upper tail, $Q''(1 - \alpha) > 0$ . Thus, when data are exchangeable, running ACI increases the expected interval width linearly with $\gamma$ relative to standard conformal prediction.
Knowl 3 — Stationarity and Asymptotic Efficiency of ACI under AR(1) Residuals
theoretical result
When model residuals follow a stationary autoregressive process of order 1, the joint state of the Adaptive Conformal Inference (ACI) miscoverage level and the lagged residual forms an ergodic Markov chain that admits a unique stationary distribution.

Let $\alpha = p/q \in \mathbb{Q} \cap (0, 1)$ with $(p, q) \in \mathbb{N} \times \mathbb{N}^*$ . Assume the residuals $\varepsilon_t = Y_t - \hat{\mu}(X_t)$ evolve as a clipped AR(1) process:
$\varepsilon_{t+1} = -R \vee (\phi \varepsilon_t + \xi_{t+1}) \wedge R$
where $|\phi| < 1$ , $R > 0$ is a large clipping bound, and $(\xi_t)_{t \ge 1}$ are i.i.d. innovations possessing a continuous probability density with respect to the Lebesgue measure on support $\mathcal{S} \supseteq [-R, R]$ . Assume the quantile function $Q$ of the stationary distribution of $|\varepsilon_t|$ is known.

The joint process $Z_t = (\alpha_t, \varepsilon_{t-1})$ defined on the state space $\mathcal{Z} = \mathcal{A} \times [-R, R]$ (where $\mathcal{A} = \{ \alpha + \gamma \frac{\gcd(q-p, p)}{q} \mathbb{Z} \} \cap ]\gamma(\alpha-1), 1+\gamma\alpha[$ ) is a homogeneous, Harris-recurrent Markov chain. It admits a unique stationary distribution $\pi_{\gamma, \phi}$ . Consequently, the time-averaged interval length $L(\alpha_t) = 2 Q(1 - \alpha_t)$ converges almost surely:
$\frac{1}{T} \sum_{t=1}^T L(\alpha_t) \xrightarrow[T \to +\infty]{\text{a.s.}} \mathbb{E}_{(\tilde{\alpha}, \tilde{\varepsilon}) \sim \pi_{\gamma, \phi}} [L(\tilde{\alpha})].$
Knowl 4 — Optimal Learning Rate Dynamics of ACI under Autoregressive Dependencies
theoretical result
For dependent time series with autoregressive residuals $\varepsilon_{t+1} = \phi \varepsilon_t + \xi_{t+1}$ , the optimal learning rate $\gamma^*_\phi = \arg\min_{\gamma} \mathbb{E}_{\pi_{\gamma, \phi}}[L]$ that minimizes the expected interval length exhibits three key dynamics:
1. Efficiency Gain for Positive $\gamma$ : Unlike the exchangeable case ( $\phi = 0$ ) where $\gamma^* = 0$ , for correlated residuals ( $\phi > 0$ ), there exists a strictly positive $\gamma^*_\phi > 0$ such that $\mathbb{E}_{\pi_{\gamma^*_\phi, \phi}}[L] < \mathbb{E}_{\pi_{0, \phi}}[L]$ , proving that ACI strictly reduces interval length in the presence of temporal dependency.
2. Monotonicity in Correlation Strength: For any fixed learning rate $\gamma$ , the expected interval length $\phi \mapsto \mathbb{E}_{\pi_{\gamma, \phi}}[L]$ is monotonically decreasing in $\phi$ . Stronger positive autocorrelation between consecutive residuals allows ACI to construct narrower valid prediction intervals.
3. Non-Monotonicity of Optimal $\gamma^*_\phi$ : The mapping $\phi \mapsto \gamma^*_\phi$ is non-monotonic. Starting from $\gamma^*_0 = 0$ , the optimal rate first increases for moderate values of $\phi$ (e.g., $\phi \in [0.6, 0.95]$ ) to quickly adapt to short-term temporal dependency. However, as $\phi \to 1$ (e.g., $\phi \ge 0.98$ ), the optimal learning rate decreases because long-memory dynamics require a smaller, more stable step size to prevent producing excessive uninformative infinite intervals (where $\alpha_t \le 0$ ).
Knowl 5 — Online Expert Aggregation on Adaptive Conformal Inference (AgACI)
algorithm
AgACI is a parameter-free conformal prediction algorithm for time series that adaptively combines predictions from a grid of $K$ parallel ACI instances running with different learning rates $\{\gamma_1, \dots, \gamma_K\}$ . It uses online expert aggregation (Bernstein Online Aggregation, BOA) with the pinball loss to predict the lower and upper interval bounds separately, eliminating the need to tune $\gamma$ .
Input: Target miscoverage rate $\alpha \in (0, 1)$ , learning rate grid $\{\gamma_k\}_{k=1}^K$ , aggregation rule $\Phi$ (BOA), threshold values $M^{(\ell)}, M^{(u)}$ , prediction indices $t \in \{T_0+1, \dots, T_0+T_1\}$ .
Output: Prediction intervals $\tilde{C}_t(x_t) = [\tilde{b}_t^{(\ell)}(x_t), \tilde{b}_t^{(u)}(x_t)]$ .
Let $\beta^{(\ell)} = \alpha/2$ and $\beta^{(u)} = 1 - \alpha/2$
for $t = T_0+1, \dots, T_0+T_1$ do
    for $k = 1, \dots, K$ do
        Compute bounds $\hat{b}_{t,k}^{(\ell)}(x_t)$ and $\hat{b}_{t,k}^{(u)}(x_t)$ using ACI with learning rate $\gamma_k$
        if $\hat{b}_{t,k}^{(\ell)}(x_t) \notin \mathbb{R}$ then
            Set $\hat{b}_{t,k}^{(\ell)}(x_t) = M^{(\ell)}$
        end if
        if $\hat{b}_{t,k}^{(u)}(x_t) \notin \mathbb{R}$ then
            Set $\hat{b}_{t,k}^{(u)}(x_t) = M^{(u)}$
        end if
    end for
    for bound type $\cdot \in \{\ell, u\}$ do
        for $k = 1, \dots, K$ do
            Compute weight $\omega_{t,k}^{(\cdot)} = \Phi\left(\{\rho_{\beta^{(\cdot)}}(y_s, \hat{b}_{s,l}^{(\cdot)}(x_s)) : s \in \{T_0+1, \dots, t-1\}, l \in \{1, \dots, K\}\}\right)$
        end for
        Compute aggregated bound:
         $\tilde{b}_t^{(\cdot)}(x_t) = \frac{\sum_{k=1}^K \omega_{t,k}^{(\cdot)} \hat{b}_{t,k}^{(\cdot)}(x_t)}{\sum_{l=1}^K \omega_{t,l}^{(\cdot)}}$
    end for
    Set prediction interval $\tilde{C}_t(x_t) = [\tilde{b}_t^{(\ell)}(x_t), \tilde{b}_t^{(u)}(x_t)]$
    Observe true value $y_t$
end for
The pinball loss is defined as $\rho_\beta(y, \hat{y}) = (y - \hat{y})(\beta - \mathbf{1}\{y < \hat{y}\})$ . All $K$ experts share the same underlying regression model $\hat{\mu}_t$ and calibration set $\mathrm{Cal}_t$ , so running $K$ parallel experts only adds the minor computational cost of evaluating $K$ different empirical quantiles.
Knowl 6 — Online Sequential Split Conformal Prediction (OSSCP)
model/method
Online Sequential Split Conformal Prediction (OSSCP) adapts Split Conformal Prediction to streaming time series data by combining periodic model refitting with a strictly temporal, sequential train/calibration split.

At each time step $t \in \{T_0+1, \dots, T_0+T_1\}$ , given a sliding window of the most recent $T_0$ historical observations $((x_{t-T_0}, y_{t-T_0}), \dots, (x_{t-1}, y_{t-1}))$ :
1. Sequential Partitioning: The window is partitioned into a proper training set $\mathrm{Tr}_t = \{t - T_0, \dots, t - T_0 + |\mathrm{Tr}| - 1\}$ and a calibration set $\mathrm{Cal}_t = \{t - T_0 + |\mathrm{Tr}|, \dots, t - 1\}$ . All time indices in $\mathrm{Tr}_t$ strictly precede those in $\mathrm{Cal}_t$ . This deterministic ordering avoids temporal data leakage; in contrast, randomized splitting places future observations into the training set and past ones into calibration, leading to an underestimation of errors on $\mathrm{Cal}_t$ and consequent undercoverage on test data.
2. Model Refitting: A regression model $\hat{\mu}_t$ is fitted on $\mathrm{Tr}_t$ .
3. Score Calibration: Absolute residual conformity scores $S_{\mathrm{Cal}_t} = \{ |y_j - \hat{\mu}_t(x_j)| : j \in \mathrm{Cal}_t \}$ are evaluated on $\mathrm{Cal}_t$ .
4. Interval Generation: The prediction interval at $x_t$ is constructed as:
$\hat{C}_\alpha(x_t) = \left[ \hat{\mu}_t(x_t) \pm \hat{Q}_{1-\alpha_{\mathrm{SCP}}}(S_{\mathrm{Cal}_t}) \right]$
with finite-sample quantile level $1 - \alpha_{\mathrm{SCP}} = (1 - \alpha)(1 + 1/|\mathrm{Cal}_t|)$ .
Knowl 7 — EnbPI V2: Consistent Mean Aggregation for Ensemble Prediction Intervals
model/method
Ensemble Prediction Interval (EnbPI) constructs sequential prediction intervals for time series without refitting the base regression model by maintaining a rolling buffer of out-of-sample calibration residuals computed from $B$ bootstrap models.

In the original EnbPI formulation, out-of-bag ensemble predictions on training points were aggregated using the mean:
$\hat{f}_{-i}(x_i) = \frac{1}{|\{b : i \notin S_b\}|} \sum_{b : i \notin S_b} \hat{f}^b(x_i),$
whereas predictions for test points $\hat{f}_{-t}(x_t)$ were computed using the $(1-\alpha)$ -quantile of the ensemble predictors $\{\hat{f}^b(x_t)\}_{b=1}^B$ .

EnbPI V2 replaces the test-time quantile aggregation with the exact same aggregation function used during calibration (the sample mean):
$\hat{f}_{-t}(x_t) = \frac{1}{B} \sum_{b=1}^B \hat{f}^b(x_t).$
Using consistent mean aggregation across both training and test phases eliminates an architectural discrepancy that impairs coverage, bringing the empirical coverage of the ensemble prediction intervals substantially closer to the nominal $1-\alpha$ target.
Knowl 8 — Synthetic Experimental Setup for Time Series Conformal Prediction
experimental setup
Conformal prediction methods for time series are benchmarked on synthetic data generated from a non-linear multivariate regression function with autocorrelated noise:
$Y_t = 10 \sin(\pi X_{t,1} X_{t,2}) + 20 (X_{t,3} - 0.5)^2 + 10 X_{t,4} + 5 X_{t,5} + 0 X_{t,6} + \varepsilon_t$
where covariates $X_t = (X_{t,1}, \dots, X_{t,6}) \sim \mathcal{U}([0, 1]^6)$ are i.i.d., with $X_{t,6}$ being an uninformative variable. The noise $\varepsilon_t$ follows a Gaussian $\mathrm{ARMA}(1, 1)$ process:
$\varepsilon_{t+1} = \phi \varepsilon_t + \xi_{t+1} + \theta \xi_t, \quad \xi_t \sim \mathcal{N}(0, \sigma^2)$
where the dependence parameters are varied over $\phi, \theta \in \{0.1, 0.8, 0.9, 0.95, 0.99\}$ . To isolate the effect of temporal dependence structure from variance scaling, the innovation variance is set to maintain a constant asymptotic noise variance $\lim_{t\to\infty} \mathrm{Var}(\varepsilon_t) = v$ (tested at $v = 10$ and $v = 1$ ):
$\sigma^2 = v \frac{1 - \phi^2}{1 - 2\phi\theta + \theta^2}.$
The regression model $\hat{\mu}$ is a Random Forest with 1000 trees, a minimum leaf size of 1, and maximum features $d = 6$ . For each parameter configuration, $n = 500$ independent sequences of length $T = T_0 + T_1 = 300$ ( $T_0 = 200$ initial window, $T_1 = 100$ sequential evaluation steps) are evaluated at target miscoverage $\alpha = 0.1$ . The methods compared include Offline SCP, OSSCP, EnbPI, EnbPI V2, ACI (with $\gamma \in \{0.01, 0.05\}$ ), and AgACI ( $K = 30$ experts).
Knowl 9 — Comparative Validity and Efficiency Across Autoregressive Error Regimes
empirical result
Across 500 independent simulations of non-linear time series with $\mathrm{ARMA}(1, 1)$ and $\mathrm{AR}(1)$ noise at asymptotic variance 10 and nominal miscoverage $\alpha = 0.10$ (target coverage $90\%$ ):
1. Online Adaptation: Online sequential split conformal prediction (OSSCP) significantly outperforms offline SCP, which exhibits severe validity degradation over time.
2. OSSCP Coverage Loss: OSSCP achieves valid coverage ( $\ge 90\%$ ) when autocorrelation is weak ( $\phi, \theta \le 0.8$ ), but degrades as temporal dependence increases, falling below $84\%$ empirical coverage at $\phi = \theta = 0.99$ .
3. EnbPI Vulnerability to Autoregressive Noise: Both EnbPI and EnbPI V2 lose coverage as autocorrelation increases in $\mathrm{AR}(1)$ and $\mathrm{ARMA}(1,1)$ noise (dropping toward $83\%$ coverage at $\phi = 0.99$ ). EnbPI achieves nominal coverage only for $\mathrm{MA}(1)$ noise, where temporal memory is strictly 1-lag.
4. ACI Robustness: ACI with $\gamma = 0.05$ maintains empirical coverage near the nominal $90\%$ target across all dependence regimes $\phi, \theta \in [0.1, 0.99]$ , showing strong robustness to autoregressive structure.
5. AgACI Performance: AgACI achieves valid coverage ( $\ge 89.8\%$ ) across all values of $\phi$ and $\theta$ , while producing the shortest median prediction interval lengths among all methods that maintain validity.

Knowl 10 — Infinite Interval Proportions in ACI under Varying Learning Rates

data/table

In Adaptive Conformal Inference (ACI), when the effective miscoverage parameter $\alpha_t \le 0$ , the algorithm outputs an uninformative infinite prediction interval $\hat{C}_{\alpha_t}(x) = \mathbb{R}$ . The table below reports the percentage of infinite intervals produced by ACI with $\gamma = 0.01$ versus $\gamma = 0.05$ across 500 simulation runs ( $T_0 = 200, T_1 = 100$ , target $\alpha = 0.1$ , asymptotic variance 10), alongside the intersection rate (the number of points where $\gamma = 0.05$ predicted $\mathbb{R}$ and $\gamma = 0.01$ failed to cover):

Noise parameters	$\gamma = 0.01$ (%)	$\gamma = 0.05$ (%)	Intersection
$\phi = \theta = 0.1$	0.00	1.12	53 out of 562 (9.43%)
$\phi = \theta = 0.8$	0.00	2.76	263 out of 1381 (19.04%)
$\phi = \theta = 0.9$	0.00	3.72	425 out of 1862 (22.83%)
$\phi = \theta = 0.95$	0.03	4.45	514 out of 2224 (23.11%)
$\phi = \theta = 0.99$	0.04	6.22	554 out of 3109 (17.82%)
$\phi = 0.1, \theta = 0$	0.00	1.00	37 out of 500 (7.40%)
$\phi = 0.8, \theta = 0$	0.00	2.75	212 out of 1373 (15.44%)
$\phi = 0.9, \theta = 0$	0.00	3.24	359 out of 1622 (22.13%)
$\phi = 0.95, \theta = 0$	0.03	4.32	488 out of 2160 (22.59%)
$\phi = 0.99, \theta = 0$	0.06	6.15	560 out of 3073 (18.22%)
$\theta = 0.1, \phi = 0$	0.00	1.03	38 out of 516 (7.36%)
$\theta = 0.8, \phi = 0$	0.00	1.42	49 out of 710 (6.90%)
$\theta = 0.9, \phi = 0$	0.00	1.54	47 out of 772 (6.09%)
$\theta = 0.95, \phi = 0$	0.00	1.54	45 out of 770 (5.84%)
$\theta = 0.99, \phi = 0$	0.00	1.56	53 out of 781 (6.79%)

Setting $\gamma = 0.01$ generates virtually zero infinite intervals ( $\le 0.06\%$ across all settings). In contrast, $\gamma = 0.05$ produces between $1.00\%$ and $6.22\%$ infinite intervals. Less than $25\%$ of these infinite intervals occur at points where $\gamma = 0.01$ miscovered, indicating that larger learning rates frequently produce uninformative infinite intervals to compensate for easy errors.

Knowl 11 — Day-Ahead French Electricity Spot Price Forecasting Application
empirical result
Conformal prediction methods were evaluated on hourly French electricity spot prices from 2016 to 2019 (35,064 hourly observations). The objective is predicting the 24 hourly prices of day $D+1$ before 12:00 PM on day $D$ , using day-ahead forecast consumption, day of week, 24 hourly prices from day $D-1$ , and 24 hourly prices from day $D-7$ .

Individual Random Forest models (1000 trees) were trained for each of the 24 hours on a 3-year rolling window (1.5 years training, 1.5 years calibration), generating sequential prediction intervals across all 8,760 hours of 2019 at target coverage $1-\alpha = 0.90$ :
1. Offline SCP vs. OSSCP: Offline SCP over-covers ( $>93.5\%$ ) with a large median interval width ( $\approx 26.5$ €/MWh) because extreme price spikes in 2016 (reaching 800 €/MWh) permanently inflate the calibration quantile. OSSCP recalibrates over the sliding window, achieving $93\%$ coverage with a narrower median width ( $\approx 24.8$ €/MWh).
2. EnbPI V2: Achieves valid coverage ( $\approx 92.7\%$ ) with median width $\approx 24.5$ €/MWh.
3. ACI and AgACI: ACI with $\gamma = 0.01$ and $\gamma = 0.05$ yields the most efficient valid intervals, with median lengths of $\approx 21.8$ €/MWh and $\approx 23.0$ €/MWh, respectively. AgACI achieves valid empirical coverage ( $\approx 91.5\%$ ) with a median length of $\approx 23.8$ €/MWh, improving efficiency over the non-adaptive baseline $\gamma = 0$ (median length $\approx 25.0$ €/MWh) without requiring parameter tuning.
Knowl 12 — Day-of-the-Week Conditional Undercoverage in Time Series Conformal Prediction
limitation
Although online conformal methods (OSSCP, EnbPI V2, ACI, and AgACI) achieve valid marginal coverage ( $\ge 90\%$ ) over the full 2019 French electricity spot price test period, all methods suffer from conditional coverage discrepancies across days of the week:
- Tuesdays through Fridays: All algorithms systematically over-cover, attaining empirical coverage rates between $92\%$ and $95\%$ .
- Mondays and Weekends (Saturdays and Sundays): All algorithms systematically under-cover, dropping to empirical coverage rates of approximately $88\%$ (below the nominal $90\%$ target).
This discrepancy arises because electricity consumption patterns and price dynamics differ substantially between weekdays and weekends. Standard conformal inference guarantees apply only marginally over time, failing to guarantee conditional validity across cyclic calendar sub-periods.

Coverage note — Deliberately omitted the toy 1D synthetic visualization in Model A.5 and the generic mathematical descriptions of BOA/OPERA aggregation rules from prior literature, as they represent illustrative examples or standard background material rather than original contributions.

References

1.Angelopoulos, A. N. and Bates, S. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021.
2.Berrisch, J. and Ziel, F. CRPS learning. Journal of Econometrics, 2021.
3.Cai, Y. and Davies, N. A simple bootstrap method for time series. Communications in Statistics-Simulation and Computation, 41(5):621–631, 2012.
4.Cauchois, M., Gupta, S., Ali, A., and Duchi, J. C. Robust Validation: Confident Predictions Even When Distributions Shift. arXiv preprint arXiv:2008.04267, 2020.
5.Cesa-Bianchi, N. and Lugosi, G. Prediction, learning, and games. Cambridge University Press, 2006.
6.Chernozhukov, V., Wuthrich, K., and Yinchu, Z. Exact and Robust Conformal Inference Methods for Predictive Machine Learning with Dependent Data. In Conference On Learning Theory, pp. 732–749. PMLR, 2018.
7.Dashevskiy, M. and Luo, Z. Network traffic demand prediction with confidence. In IEEE Global Telecommunications Conference. IEEE, 2008.
8.Dashevskiy, M. and Luo, Z. Time series prediction with performance guarantee. IET communications, 5(8):1044–1051, 2011.
9.EUPHEMIA. Euphemia public description, single price coupling algorithm, April 2019. URL https://www.nemo-committee.eu/assets/files/190410_Euphemia%20Public%20Description%20version%20NEMO%20Committee.pdf.
10.Friedman, J. H., Grosse, E., and Stuetzle, W. Multidimensional additive spline approximation. SIAM J. Sci. Stat. Comput., 1983.
11.Gaillard, P. and Goude, Y. OPERA, 2021. URL https://cran.r-project.org/package=opera. R package version 1.2.0.
12.Gaillard, P., Stoltz, G., and Van Erven, T. A second-order bound with excess losses. In Conference on Learning Theory, pp. 176–196. PMLR, 2014.
13.Gaillard, P., Goude, Y., and Nedellec, R. Additive models and robust aggregation for GEFCom2014 probabilistic electric load and electricity price forecasting. International Journal of Forecasting, 32(3):1038–1050, 2016.
14.Gibbs, I. and Cand`es, E. Adaptive conformal inference under distribution shift. In Advances in Neural Information Processing Systems, 2021.
15.Goehry, B. Random forests for time-dependent processes. ESAIM: Probability and Statistics, 24:801–826, 2020.
16.Goehry, B., Yan, H., Goude, Y., Massart, P., and Poggi, J.-M. Random forests for time series. HAL hal-03129751, 2021.
17.H¨ardle, W., Horowitz, J., and Kreiss, J.-P. Bootstrap methods for time series. International Statistical Review, 71(2):435–459, 2003.
18.Hazan, E. Introduction to online convex optimization. arXiv preprint arXiv:1909.05207, 2019.
19.Kath, C. and Ziel, F. Conformal prediction interval estimation and applications to day-ahead and intraday power markets. International Journal of Forecasting, 37(2):777–799, 2021.
20.Kreiss, J.-P. and Paparoditis, E. The hybrid wild bootstrap for time series. Journal of the American Statistical Association, 107(499):1073–1084, 2012.
21.Lago, J., De Ridder, F., and De Schutter, B. Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms. Applied Energy, 221:386–405, 2018.
22.Lago, J., Marcjasz, G., De Schutter, B., and Weron, R. Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an openaccess benchmark. Applied Energy, 293:116983, 2021.
23.Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., and Wasserman, L. Distribution-Free Predictive Inference for Regression. Journal of the American Statistical Association, 113(523):1094–1111, 2018.
24.Maciejowska, K., Nowotarski, J., and Weron, R. Probabilistic forecasting of electricity spot prices using Factor Quantile Regression Averaging. International Journal of Forecasting, 32(3):957–965, 2016.
25.Meyn, S. P. and Tweedie, R. L. Markov chains and stochastic stability. Springer Science & Business Media, 2012.
26.Nowotarski, J. and Weron, R. Recent advances in electricity price forecasting: A review of probabilistic forecasting. Renewable and Sustainable Energy Reviews, 81:1548–1568, 2018.
27.Papadopoulos, H., Proedrou, K., Vovk, V., and Gammerman, A. Inductive Confidence Machines for Regression. In Machine Learning: ECML 2002, pp. 345–356. Springer, 2002.
28.Remlinger, C., Bri`ere, M., Alasseur, C., and Mikael, J. Expert aggregation for financial forecasting. arXiv preprint arXiv:2111.15365, 2021.
29.Romano, Y., Patterson, E., and Cand`es, E. Conformalized Quantile Regression. Advances in Neural Information Processing Systems, 32, 2019.
30.Saha, A., Basu, S., and Datta, A. Random forests for spatially dependent data. Journal of the American Statistical Association, 0(0):1–19, 2021.
31.Shafer, G. and Vovk, V. A Tutorial on Conformal Prediction. JMLR, 9:51, 2008.
32.Shalev-Shwartz, S. and Singer, Y. A primal-dual perspective of online learning algorithms. Machine Learning, 69(2-3):115–142, 2007.
33.Tibshirani, R. J., Barber, R. F., Cand`es, E., and Ramdas, A. Conformal Prediction Under Covariate Shift. Advances in Neural Information Processing Systems, 32:11, 2019.
34.Uniejewski, B. and Weron, R. Regularized quantile regression averaging for probabilistic electricity price forecasting. Energy Economics, pp. 105121, 2021.
35.Vovk, V., Gammerman, A., and Saunders, C. Machine-Learning Applications of Algorithmic Randomness. In Proceedings of the Sixteenth International Conference on Machine Learning, pp. 444–453. Morgan Kaufmann Publishers Inc., 1999.
36.Vovk, V., Gammerman, A., and Shafer, G. Algorithmic Learning in a Random World. Springer US, 2005.
37.Vovk, V. G. Aggregating strategies. Proc. of Computational Learning Theory, 1990.
38.Weron, R. Electricity price forecasting: A review of the state-of-the-art with a look into the future. International Journal of Forecasting, 30(4):1030–1081, 2014.
39.Wintenberger, O. Optimal learning with bernstein online aggregation. Machine Learning, 106(1):119–141, 2017.
40.Wisniewski, W., Lindsay, D., and Lindsay, S. Application of conformal prediction interval estimations to market makers’ net positions. In Proceedings of the Ninth Symposium on Conformal and Probabilistic Prediction and Applications, volume 128 of Proceedings of Machine Learning Research, pp. 285–301. PMLR, 2020.
41.Xu, C. and Xie, Y. Conformal prediction interval for dynamic time-series. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 11559–11569. PMLR, 2021a.
42.Xu, C. and Xie, Y. Conformal prediction for dynamic timeseries. arXiv preprint arXiv:2010.09107, 2021b.

Citation

MLA

Zaffran, M., et al. “Adaptive Conformal Predictions for Time Series”. International Conference on Machine Learning, vol. 162, 2022, pp. 25834–66, https://proceedings.mlr.press/v162/zaffran22a.html.

APA

Zaffran, M., Feron, O., Goude, Y., Josse, J., & Dieuleveut, A. (2022). Adaptive Conformal Predictions for Time Series. International Conference on Machine Learning, 162, 25834–25866. https://proceedings.mlr.press/v162/zaffran22a.html

Chicago

Zaffran, M., O. Feron, Y. Goude, J. Josse, and A. Dieuleveut. 2022. “Adaptive Conformal Predictions for Time Series”. International Conference on Machine Learning 162: 25834–66. https://proceedings.mlr.press/v162/zaffran22a.html.

Harvard

Zaffran, M. et al. (2022) “Adaptive Conformal Predictions for Time Series”, International Conference on Machine Learning. PMLR, pp. 25834–25866. Available at: https://proceedings.mlr.press/v162/zaffran22a.html.

Vancouver

1. Zaffran M, Feron O, Goude Y, Josse J, Dieuleveut A (2022) Adaptive Conformal Predictions for Time Series. In: International Conference on Machine Learning. PMLR, pp 25834–25866

BibTeX

@InProceedings{pmlr-v162-zaffran22a,
  title = 	 {Adaptive Conformal Predictions for Time Series},
  author =       {Zaffran, Margaux and Feron, Olivier and Goude, Yannig and Josse, Julie and Dieuleveut, Aymeric},
  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},
  pages = 	 {25834--25866},
  year = 	 {2022},
  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume = 	 {162},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--23 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v162/zaffran22a/zaffran22a.pdf},
  url = 	 {https://proceedings.mlr.press/v162/zaffran22a.html},
  abstract = 	 {Uncertainty quantification of predictive models is crucial in decision-making problems. Conformal prediction is a general and theoretically sound answer. However, it requires exchangeable data, excluding time series. While recent works tackled this issue, we argue that Adaptive Conformal Inference (ACI, Gibbs & Cand{è}s, 2021), developed for distribution-shift time series, is a good procedure for time series with general dependency. We theoretically analyse the impact of the learning rate on its efficiency in the exchangeable and auto-regressive case. We propose a parameter-free method, AgACI, that adaptively builds upon ACI based on online expert aggregation. We lead extensive fair simulations against competing methods that advocate for ACI’s use in time series. We conduct a real case study: electricity price forecasting. The proposed aggregation algorithm provides efficient prediction intervals for day-ahead forecasting. All the code and data to reproduce the experiments are made available on GitHub.}
}

Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

Adaptive Conformal Predictions for Time Series

Abstract

Keywords

Table of Contents

Knowls

Knowl 1 — Adaptive Conformal Inference for Time Series

Knowl 2 — Asymptotic Efficiency Degradation of ACI in Exchangeable Settings

Knowl 3 — Stationarity and Asymptotic Efficiency of ACI under AR(1) Residuals

Knowl 4 — Optimal Learning Rate Dynamics of ACI under Autoregressive Dependencies

Knowl 5 — Online Expert Aggregation on Adaptive Conformal Inference (AgACI)

Knowl 6 — Online Sequential Split Conformal Prediction (OSSCP)

Knowl 7 — EnbPI V2: Consistent Mean Aggregation for Ensemble Prediction Intervals

Knowl 8 — Synthetic Experimental Setup for Time Series Conformal Prediction

Knowl 9 — Comparative Validity and Efficiency Across Autoregressive Error Regimes

Knowl 10 — Infinite Interval Proportions in ACI under Varying Learning Rates

Knowl 11 — Day-Ahead French Electricity Spot Price Forecasting Application

Knowl 12 — Day-of-the-Week Conditional Undercoverage in Time Series Conformal Prediction

References

Citation

BibTeX

Access the Paper