Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics

Cristopher SalviMaud LemercierAndris Gerasimovics

article2022NeurIPS70 citations

Introduces a neural architecture based on mild solutions of stochastic PDEs that learns solution operators from partially observed, continuously arriving data across arbitrary spatiotemporal resolutions up to three orders of magnitude faster than traditional numerical solvers.

Listen

Many complex natural and engineered systems—ranging from fluid turbulence and wave propagation to magnetic phase transitions—are subject to environmental fluctuations and are mathematically modeled using stochastic partial differential equations (SPDEs). Traditional numerical solvers for these equations require fine discretization grids to maintain accuracy and stability, which makes high-resolution simulations computationally intractable for time-sensitive decision-making and large-scale modeling. While recent physics-informed machine learning architectures attempt to approximate dynamical systems, existing methods either process temporal sequences without spatial resolution invariance or learn spatial mappings without accounting for continuous, external stochastic disturbances.

The article demonstrates the Neural Stochastic Partial Differential Equation (Neural SPDE) model, a novel deep learning architecture designed to learn the solution operators of continuous spatiotemporal systems driven by random noise from partially observed data. The authors evaluate whether this architecture can process inputs arriving at arbitrary spatial and temporal resolutions and generalize across different grid scales while maintaining high physical fidelity.

The approach models the mild solutions of randomly forced differential equations by operating in a latent functional space through frequency-domain kernel parameterizations. The model is evaluated via two distinct methods: a system of ordinary differential equations in Fourier space solved with standard integration tools, and a space-time fixed-point problem solved via root-finding iterations. The authors validated the framework on three benchmark physical systems: the 1D Ginzburg-Landau equation, the 1D Korteweg-De Vries equation, and the 2D stochastic Navier-Stokes equations under varying data volumes and subsampling conditions.

The findings show that the proposed model substantially outperforms existing baseline architectures across all physical benchmarks. When predicting dynamics conditioned on both initial states and random noise, the model achieved test errors of approximately 0.6% to 1.2% on Ginzburg-Landau dynamics and under 1% on the Korteweg-De Vries equation, outperforming alternatives by one to two orders of magnitude. For 2D Navier-Stokes turbulence, the model achieved a 4.9% relative error compared to 17.8% for the next best baseline, while demonstrating zero-shot super-resolution by training on a coarse 16x16 grid and accurately predicting on a fine 64x64 grid. Furthermore, the Neural SPDE executed inference up to 300 times faster than classical numerical solvers for Navier-Stokes equations and up to 80 times faster for wave equations.

These results indicate that data-driven surrogate models can drastically reduce computational overhead and simulation cycle times for complex, uncertain physical systems without sacrificing accuracy. For operational environments in engineering, meteorology, and fluid dynamics, this enables real-time forecasting, efficient sensitivity analysis, and rapid risk assessment using significantly smaller training datasets. Unlike prior neural operator methods, the framework successfully incorporates the impacts of external random signals on spatial fields.

Organizations developing surrogate physical models should consider adopting Neural SPDE architectures for spatiotemporal problems involving stochastic forcing. Implementation teams are advised to prioritize the fixed-point evaluation approach, which demonstrated approximately tenfold faster runtimes than the differential equation solver approach. Future technical development should explore non-Fourier spatial parameterizations to better handle complex, non-periodic geometries and investigate generative modeling frameworks for video and physical field generation.

While the empirical results demonstrate robust performance, users should exercise caution regarding temporal data resolution; the model's accuracy degrades when inputs are sparsely sampled in time due to the mathematical roughness of temporal driving noise. Additionally, current theoretical convergence guarantees are established for regularized noise under specific differential operators, meaning broader mathematical guarantees for arbitrary operators remain an open research problem.

Imperial College London The Alan Turing Institute University of Bath University of Warwick stochastic stochastic dynamics partial differential equations Neural Stochastic PDEs resolution-invariant learning continuous spatiotemporal dynamics

arXiv: 2110.10249 crispitagorico/torchspde

Paper: Fourier Neural Operator for Parametric Partial Differential Equations, Zongyi Li et al. (2020). Introduces Fourier Neural Operators, the foundational spectral operator-learning framework that Neural SPDE extends to stochastic and continuous-time spatiotemporal dynamics.
Paper: Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Lu Lu et al. (2021). Presents DeepONet and establishes the universal operator approximation theorem for learning infinite-dimensional mappings, which underlies neural PDE and SPDE solution operators.
Paper: Neural Ordinary Differential Equations, Ricky T. Q. Chen et al. (2018). Establishes Neural Ordinary Differential Equations and continuous-depth architectures, which provide the theoretical basis for Neural CDEs and continuous-time ODE solver evaluations in Neural SPDEs.
Paper: DGM: A deep learning algorithm for solving partial differential equations, Justin Sirignano et al. (2017). Formulates the Deep Galerkin Method for learning solutions to differential equations without spatial meshes, motivating the spectral Galerkin emulation used in Neural SPDEs.
Paper: DeepXDE: A Deep Learning Library for Solving Differential Equations, Lu Lu et al. (2019). Provides a comprehensive foundation for solving forward and inverse differential equations with neural networks.

Paper: Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs, Nikola Kovachki et al. (2023). Provides a comprehensive mathematical framework and benchmark analysis for neural operators that map between infinite-dimensional function spaces.
Paper: Score-Based Diffusion Models in Function Space, Jae Hyun Lim 0001 et al. (2025). Generalizes score-based diffusion models to infinite-dimensional Hilbert spaces using neural operators, continuing the study of continuous functional dynamics.
Paper: Neural Operators with Localized Integral and Differential Kernels, Miguel Liu-Schiaffini et al. (2024). Enhances neural operator learning by combining differential and integral kernels to capture localized high-frequency spatiotemporal features that global spectral methods smooth out.
Paper: Convolutional Neural Operators for robust and accurate learning of PDEs, Bogdan Raonic et al. (2023). Introduces continuous-discrete equivalent convolutional neural operators as an alternative discretization-invariant architecture for learning PDE solution operators.
Paper: Learning Operators with Coupled Attention, Georgios Kissas et al. (2022). Explores operator learning with coupled attention and kernel transformations to improve data efficiency and handle sparse, noisy spatial measurements.
Paper: PDEBench: An Extensive Benchmark for Scientific Machine Learning, Makoto Takamoto et al. (2022). Offers a standardized, comprehensive benchmark suite of time-dependent PDE tasks to evaluate the accuracy and generalization of scientific machine learning models.

Cover for Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics

Abstract

Stochastic partial differential equations (SPDEs) are the mathematical tool of choice for modelling spatiotemporal PDE-dynamics under the influence of randomness. Based on the notion of mild solution of an SPDE, we introduce a novel neural architecture to learn solution operators of PDEs with (possibly stochastic) forcing from partially observed data. The proposed Neural SPDE model provides an extension to two popular classes of physics-inspired architectures. On the one hand, it extends Neural CDEs and variants – continuous-time analogues of RNNs – in that it is capable of processing incoming sequential information arriving at arbitrary spatial resolutions. On the other hand, it extends Neural Operators – generalizations of neural networks to model mappings between spaces of functions – in that it can parameterize solution operators of SPDEs depending simultaneously on the initial condition and a realization of the driving noise. By performing operations in the spectral domain, we show how a Neural SPDE can be evaluated in two ways, either by calling an ODE solver (emulating a spectral Galerkin scheme), or by solving a fixed point problem. Experiments on various semilinear SPDEs, including the stochastic Navier-Stokes equations, demonstrate how the Neural SPDE model is capable of learning complex spatiotemporal dynamics in a resolution-invariant way, with better accuracy and lighter training data requirements compared to alternative models, and up to 3 orders of magnitude faster than traditional solvers.

Topics

stochastic stochastic dynamics partial differential equations

Keywords

Neural Stochastic PDEs resolution-invariant learning continuous spatiotemporal dynamics stochastic partial differential equations stochastic Navier-Stokes equations stochastic Ginzburg-Landau neural operators adjoint-based backpropagation Stratonovich solution spectral Galerkin

1 Introduction
2 Background on SPDEs
3 Neural SPDEs
3.1 The model
3.2 Evaluating the model by solving a system of ODEs
3.3 Evaluating the model by solving a fixed point problem
3.4 Space-time resolution-invariance
3.5 Comparison of the two evaluation methods
3.6 Considerations about convergence
4 Experiments
4.1 Stochastic Ginzburg-Landau equation
4.2 Stochastic Korteweg-De Vries equation
4.3 Stochastic Navier-Stokes equations in 2D
5 Conclusion
Acknowledgments and Disclosure of Funding
References

Knowls

Knowl 1 — Neural SPDE solution-operator architecture
model/method
A Neural SPDE learns a map from an initial field and an observed, continuously interpolated forcing signal to a spatiotemporal solution field. Let $D\subset\mathbb{R}^d$ be the spatial domain, $u_0:D\to\mathbb{R}^{d_u}$ the initial condition, and $\xi:[0,T]\to H_\xi=L^2(D,\mathbb{R}^{d_\xi})$ the forcing signal. The latent field belongs to $H_h=L^2(D,\mathbb{R}^{d_h})$ , where $d_h>d_u$ . Four pointwise feedforward neural networks are learned: $L_\theta:\mathbb{R}^{d_u}\to\mathbb{R}^{d_h}$ , $F_\theta:\mathbb{R}^{d_h}\to\mathbb{R}^{d_h}$ , $G_\theta:\mathbb{R}^{d_h}\to\mathbb{R}^{d_h\times d_\xi}$ , and $\Pi_\theta:\mathbb{R}^{d_h}\to\mathbb{R}^{d_u}$ . A stationary matrix-valued kernel $K_t$ represents the latent linear evolution, and $*$ denotes spatial convolution.

The Neural SPDE is
$z_0(x)=L_\theta(u_0(x)),\qquad z_t=K_t*z_0+\int_0^tK_{t-s}*\left[F_\theta(z_s)+G_\theta(z_s)\xi_s\right]ds,\qquad u_t(x)=\Pi_\theta(z_t(x)).$
Here $z_t:D\to\mathbb{R}^{d_h}$ is the latent state, $u_t:D\to\mathbb{R}^{d_u}$ is the predicted physical field, and $G_\theta(z_s)\xi_s$ is matrix-vector multiplication at each spatial point. The kernel $K_t$ parameterizes the action of the unknown or partially known linear differential operator rather than requiring that operator to be supplied. Lipschitz-compatible activations such as ReLU or tanh can be used in $F_\theta$ and $G_\theta$ .
Knowl 2 — Fourier-space ODE evaluation
algorithm
When the latent linear operator is a polynomial differential operator, the Neural SPDE can be evaluated by solving an ordinary differential equation in Fourier space. Let $\mathcal{F}_d$ and $\mathcal{F}_d^{-1}$ be the spatial Fourier transform and its inverse. Assume that the Fourier transform of the kernel satisfies
$\mathcal{F}_d(K_t)(y)=e^{tA(y)},$
where $y\in\mathbb{R}^d$ is a spatial frequency and $A(y)\in\mathbb{C}^{d_h\times d_h}$ is the Fourier symbol of the latent linear operator. Define $v_t=\mathcal{F}_d(z_t)$ and $v_0=\mathcal{F}_d(z_0)$ . Then $v_t$ solves
$\partial_t v_t(y)=A(y)v_t(y)+\mathcal{F}_d\!\left(F_\theta(z_t)+G_\theta(z_t)\xi_t\right)(y), \qquad z_t=\mathcal{F}_d^{-1}(v_t).$
Equivalently, the vector field is $\Psi_{\theta,\xi}=A+\mathcal{F}_d\circ H_{\theta,\xi}\circ\mathcal{F}_d^{-1}$ , where $H_{\theta,\xi}(h)=F_\theta(h)+G_\theta(h)\xi$ . The model prediction is obtained by applying any numerical ODE solver to the initial Fourier state $\mathcal{F}_d(z_0)$ and transforming the resulting trajectory back with $\mathcal{F}_d^{-1}$ . This construction is a neural analogue of a spectral Galerkin solver and permits memory-efficient adjoint-based differentiation. In implementation, Fourier transforms are replaced by discrete Fourier transforms and truncated to prescribed maximum frequency modes.
Knowl 3 — Fourier fixed-point evaluation
algorithm
A second evaluation method represents the space-time kernel directly in Fourier space and solves the Neural SPDE as a fixed-point problem. Let $\mathcal{F}_1$ be the time-only Fourier transform, $\mathcal{F}_d$ the space-only Fourier transform, and $\mathcal{F}_{d+1}$ the joint space-time transform. Let $B$ be a learned complex tensor representing $\mathcal{F}_{d+1}(K)$ , let $\mathbf{1}_{\geq 0}$ be the indicator of nonnegative time, and let $[\cdot]_t$ denote evaluation at time $t$ . Define
$\Phi_{\theta,\xi}(z)_t= \mathcal{F}_d^{-1}\!\left(\left[\mathcal{F}_1^{-1}(B)\right]_t\,\mathcal{F}_d(z_0)\right) +\left[\mathcal{F}_{d+1}^{-1}\!\left(B\,\mathcal{F}_{d+1}\!\left(\mathbf{1}_{\geq 0}H_{\theta,\xi}(z_{\cdot})\right)\right)\right]_t,$
where all products in Fourier space are matrix-vector products over the latent channels. The latent trajectory is defined by the fixed-point equation
$z=\Phi_{\theta,\xi}(z).$
Classical root-finding methods, including Picard iteration, approximate the solution. The method supports memory-efficient differentiation through implicit differentiation rather than storing every fixed-point iteration. The transforms are implemented with discrete Fourier transforms and a selected finite set of spatial and temporal frequency modes; the mode counts are hyperparameters.
Knowl 4 — Space-time resolution invariance and partial-observation processing
model/method
A Neural SPDE first interpolates possibly irregular and partially observed space-time measurements into a continuous forcing signal $\xi$ and initial field $u_0$ . Because its latent evolution is defined over continuous space and time rather than tied to the training mesh, the same learned operator can be evaluated at an arbitrary spatial and temporal resolution. In particular, a model trained on a coarse grid can be evaluated directly on a finer grid without retraining, providing zero-shot space-time super-resolution. The interpolation must produce a sufficiently regular signal and, for the Fourier implementation, must map irregular observations to a regular grid so that the discrete Fourier transform remains a good approximation.
Knowl 5 — Computational complexity and evaluation trade-offs
model/method
For latent dimension $d_h$ , spatial frequency counts $k_1^{\max},\ldots,k_d^{\max}$ , temporal frequency count $k_{d+1}^{\max}$ , spatial-grid size $N_x$ , temporal-grid size $N_t$ , ODE-solver step count $N$ , and fixed-point iteration count $I$ , the dominant Fourier parameters are the complex tensor $A$ for the ODE method and $B$ for the fixed-point method. Their sizes are respectively
$\left(\prod_{i=1}^d k_i^{\max}\right)d_h^2 \qquad\text{and}\qquad \left(\prod_{i=1}^{d+1} k_i^{\max}\right)d_h^2.$
The ODE evaluation has computational complexity $O(NN_x\log N_x)$ , whereas the fixed-point evaluation has complexity
$O\!\left(IN_xN_t\bigl(\log N_x+\log N_t\bigr)\right).$
In the experiments, $N_t\approx N_x$ and $IN_t\approx N$ , making the asymptotic costs comparable. The authors found the ODE implementation approximately $10$ times slower than the fixed-point implementation, which they attribute largely to the ODE library implementation and the highly optimized Fourier transforms. Although the fixed-point parameterization has an additional temporal-frequency dimension, comparable accuracy typically required a latent dimension roughly $20$ times larger than for the ODE parameterization.
Knowl 6 — Scope and convergence limitation of mollified infinite-dimensional noise
limitation
The Neural SPDE formulation regularizes a Wiener driving signal $W$ by convolution with a smooth compactly supported mollifier, $W^\varepsilon=\varphi^\varepsilon*W$ , and treats the resulting derivative as a random PDE forcing. For finite-dimensional noise, Wong–Zakai theory gives convergence of the mollified random PDEs to a Stratonovich solution and continuity of the solution map in an appropriate rough-path topology. The paper emphasizes that its driving noise is infinite-dimensional, for which the Itô–Stratonovich correction can be infinite and the usual Stratonovich interpretation is not available. For the heat operator, existing results establish convergence to an Itô solution under suitable renormalization and drift correction, with continuity expressed in a regularity-structures topology. A corresponding convergence theorem for the generic differential operators used by the Neural SPDE architecture is not established and is left as future work.
Knowl 7 — Supervised learning tasks and benchmark protocol
experimental setup
The experiments evaluate three operator-learning tasks for stochastic PDEs: $u_0\mapsto u$ when the driving noise is unobserved; $\xi\mapsto u$ when the initial condition $u_0$ is fixed across samples and the noise is observed; and $(u_0,\xi)\mapsto u$ when both the initial condition and noise vary and are observed. The loss is the relative pathwise $L^2$ error. Neural SPDEs are compared with Neural CDEs, Neural RDEs, Fourier Neural Operators, DeepONets, and a hybrid Neural CDE-FNO whose drift is represented by an FNO and whose diffusion is a feedforward network. Hyperparameters are selected by grid search, and experiments use a Tesla P100 GPU.

The unobserved-noise task is intentionally difficult: since a stochastic solution depends on information absent from $u_0$ , all models are expected to perform poorly relative to tasks that observe $\xi$ . The experiments nevertheless use it as a sanity check and also test that Neural SPDEs can represent deterministic PDE solution operators when the noise input is absent.
Knowl 8 — Ginzburg–Landau accuracy across data regimes
data/table
For the one-dimensional stochastic Ginzburg–Landau equation
$\partial_tu-\Delta u=3u-u^3+\xi, \qquad u(t,0)=u(t,1), \qquad u(0,x)=u_0(x),$
with space-time white noise $\xi$ , trajectories are generated on $128$ evenly spaced spatial and temporal points using time step $\Delta t=10^{-3}$ until $T=0.05$ . The initial condition is $u_0(x)=x(1-x)+\kappa\eta(x)$ , where $\eta(x)=a_0+\sum_{k=-10}^{10}a_k(1+|k|^2)^{-1}\sin(k\pi x)$ and $a_k\sim\mathcal{N}(0,1)$ . The value $\kappa=0$ gives fixed initial data, while $\kappa=0.1$ gives varying initial data. The table reports relative pathwise $L^2$ test error for Neural CDE (NCDE), Neural RDE (NRDE), the hybrid NCDE-FNO, DeepONet, Fourier Neural Operator (FNO), and Neural SPDE (NSPDE); $x$ means not applicable.
Could not parse LaTeX table
Neural SPDE has the lowest error in every applicable task. On the main task $(u_0,\xi)\mapsto u$ , it reaches $0.012$ with $1{,}000$ training observations and $0.006$ with $10{,}000$ , approximately one order of magnitude better than the applicable baselines in the large-data regime. It maintains roughly $1\%$ error even in the low-data regime.
Knowl 9 — Stochastic Korteweg–De Vries accuracy and robustness to subsampling
data/table
The stochastic Korteweg–De Vries experiment uses
$\partial_tu+\gamma\partial_x^3u=6u\partial_xu+\xi, \qquad u(t,0)=u(t,1), \qquad u(0,x)=u_0(x),$
with $\gamma=0.1$ . The forcing is $\xi=\dot W$ , where $W$ is a truncated $Q$ -Wiener process with spatial basis $\phi_j(x)=\sin(j\pi x)$ and eigenvalues $\lambda_j\sim j^{-5+\varepsilon}$ for small positive $\varepsilon$ . Data are generated on $128$ spatial points with forcing time step $10^{-3}$ and solution time step $10^{-2}$ . The initial field is $u_0(x)=\sin(2\pi x)+\kappa\eta(x)$ , with $\kappa=0$ for fixed and $\kappa=1$ for varying initial conditions. All datasets contain $N=1{,}000$ training observations. The entries are relative pathwise $L^2$ test errors; $x$ means not applicable.
Could not parse LaTeX table
Could not parse LaTeX table
Could not parse LaTeX table
At $T=0.5$ , Neural SPDE improves over the second-best FNO by about one order of magnitude for $\xi\mapsto u$ and over NCDE-FNO by almost two orders of magnitude for $(u_0,\xi)\mapsto u$ . Its performance is nearly unchanged when $50\%$ of spatial observations are removed, but degrades when $10\%$ of temporal observations are removed, consistent with the forcing being smoother in space than in time.
Knowl 10 — Stochastic Navier–Stokes performance and zero-shot super-resolution
data/table
The two-dimensional stochastic Navier–Stokes experiment uses the vorticity equation
$\partial_tw-\nu\Delta w=-u\cdot\nabla w+f+\sigma\xi, \qquad w(0,x)=w_0(x),$
on the periodic domain $[0,1]^2$ . Here $u$ is the unique divergence-free velocity satisfying $w=\nabla\times u$ , $f$ is a deterministic spatial forcing, $\xi=\dot W$ is a spatially colored $Q$ -Wiener forcing, $\sigma=0.05$ , and $\nu=10^{-4}$ . The initial vorticity is sampled as $w_0\sim\mathcal{N}\!\left(0,3^{3/2}(-\Delta+49I)^{-3}\right)$ , where $I$ is the identity operator. Reference trajectories are generated with a pseudo-spectral solver on a $64\times64$ spatial grid and time step $10^{-3}$ . For the $u_0\mapsto u$ and $\xi\mapsto u$ tasks, training uses $16\times16$ spatial fields obtained by downsampling and $N=1{,}000$ samples. For $(u_0,\xi)\mapsto u$ , $2{,}000$ rolling-window input-output pairs of length $500$ are obtained from ten long trajectories.
Could not parse LaTeX table
Neural SPDE achieves the best error in all applicable tasks, including $0.049$ for the joint initial-condition-and-noise task and $0.034$ for the noise-only task. It is trained on a $16\times16$ mesh and evaluated on a $64\times64$ mesh, while also being evaluated over a longer time horizon; the predicted fields remain qualitatively accurate, demonstrating zero-shot space-time super-resolution.

Using identical spatiotemporal discretizations for the learned model and the numerical solver, the inference-time ratios are
Could not parse LaTeX table
Thus, in these experiments, Neural SPDE inference is between $59\times$ and $300\times$ faster than the corresponding traditional numerical solvers.

Coverage note — No substantial contributed method or result was omitted; ancillary Fourier/sampling derivations, additional appendix experiments, and proposed future applications were excluded because they support or extend the main contribution rather than constitute separate load-bearing knowls.

References

1.Sh A Alimov, RR Ashurov, and AK Pulatov. Multiple fourier series and fourier integrals. In Commutative Harmonic Analysis IV, pages 1–95. Springer, 1992.
2.Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Deep equilibrium models. Advances in Neural Information Processing Systems, 32:690–701, 2019.
3.Alexis Bellot and Mihaela Van Der Schaar. Policy analysis using synthetic controls in continuous-time. In International Conference on Machine Learning, pages 759–768. PMLR, 2021.
4.William L Briggs and Van Emden Henson. The DFT: an owner’s manual for the discrete Fourier transform. SIAM, 1995.
5.Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 6572–6583, 2018.
6.Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, 6(4):911–917, 1995.
7.Ilya Chevyrev, Andris Gerasimovics, and Hendrik Weber. Feature engineering with regularity structures. arXiv preprint arXiv:2108.05879, 2021.
8.James W Cooley and John W Tukey. An algorithm for the machine calculation of complex fourier series. Mathematics of computation, 19(90):297–301, 1965.
9.Peter K Friz and Martin Hairer. A course on rough paths. Springer, 2020.
10.Massimiliano Gubinelli. Controlling rough paths. Journal of Functional Analysis, 216(1):86–140, 2004.
11.Martin Hairer. An introduction to stochastic pdes. arXiv preprint arXiv:0907.4178, 2009.
12.Martin Hairer. Solving the kpz equation. Annals of mathematics, pages 559–664, 2013.
13.Martin Hairer. A theory of regularity structures. Inventiones mathematicae, 198(2):269–504, 2014.
14.Martin Hairer and Étienne Pardoux. A wong-zakai theorem for stochastic pdes. Journal of the Mathematical Society of Japan, 67(4):1551–1604, 2015.
15.Helge Holden, Bernt Øksendal, Jan Ubøe, and Tusheng Zhang. Stochastic partial differential equations. In Stochastic partial differential equations, pages 141–191. Springer, 1996.
16.Peiyan Hu, Qi Meng, Bingguang Chen, Shiqi Gong, Yue Wang, Wei Chen, Rongchan Zhu, Zhi-Ming Ma, and Tie-Yan Liu. Neural operator with regularity structure for modeling dynamics driven by spdes. arXiv preprint arXiv:2204.06255, 2022.
17.Patrick Kidger. On neural differential equations. arXiv preprint arXiv:2202.02435, 2022.
18.Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. arXiv preprint arXiv:2005.08926, 2020.
19.Patrick Kidger, James Foster, Xuechen Li, and Terry Lyons. Efficient and accurate gradients for neural sdes. arXiv preprint arXiv:2105.13493, 2021.
20.Patrick Kidger, James Foster, Xuechen Li, Harald Oberhauser, and Terry Lyons. Neural sdes as infinite-dimensional gans. arXiv preprint arXiv:2102.03657, 2021.
21.Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.08481, 2021.
22.Xuechen Li, Ting-Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, pages 3870–3882. PMLR, 2020.
23.Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020.
24.Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhattacharya, and Anima Anandkumar. Multipole graph neural operator for parametric partial differential equations. Advances in Neural Information Processing Systems, 33, 2020.
25.Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar, et al. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2020.
26.Xuanqing Liu, Tesi Xiao, Si Si, Qin Cao, Sanjiv Kumar, and Cho-Jui Hsieh. Neural sde: Stabilizing neural ode networks with stochastic noise. arXiv preprint arXiv:1906.02355, 2019.
27.Gabriel J Lord, Catherine E Powell, and Tony Shardlow. An introduction to computational stochastic PDEs, volume 50. Cambridge University Press, 2014.
28.Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021.
29.Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 393:114778, 2022.
30.Terry J Lyons. Differential equations driven by rough signals. Revista Matemática Iberoamericana, 14(2):215–310, 1998.
31.Remigijus Mikulevicius and Boris L Rozovskii. Stochastic navier–stokes equations for turbulent flows. SIAM Journal on Mathematical Analysis, 35(5):1250–1310, 2004.
32.James Morrill, Cristopher Salvi, Patrick Kidger, and James Foster. Neural rough differential equations for long time series. In International Conference on Machine Learning, pages 7829–7838. PMLR, 2021.
33.Cristopher Salvi, Thomas Cass, James Foster, Terry Lyons, and Weixin Yang. The signature kernel is the solution of a goursat pde. SIAM Journal on Mathematics of Data Science, 3(3):873–899, 2021.
34.Roger Temam. Infinite-dimensional dynamical systems in mechanics and physics, volume 68. Springer Science & Business Media, 2012.
35.Krystyna Twardowska. Wong-zakai approximations for stochastic differential equations. Acta Applicandae Mathematica, 43(3):317–359, 1996.
36.Abdul-Majid Wazwaz. Solitary waves theory. In Partial Differential Equations and Solitary Waves Theory, pages 479–502. Springer, 2009.
37.E Weinan. A proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics, 1(5):1–11, 2017.
38.William Henry Young. Vi. on the general theory integration. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 204(372-386):221–252, 1905.

Citation

MLA

Salvi, C., et al. “Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics”. Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 1333–44, https://proceedings.neurips.cc/paper_files/paper/2022/file/091166620a04a289c555f411d8899049-Paper-Conference.pdf.

APA

Salvi, C., Lemercier, M., & Gerasimovics, A. (2022). Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics. Advances in Neural Information Processing Systems, 35, 1333–1344. https://proceedings.neurips.cc/paper_files/paper/2022/file/091166620a04a289c555f411d8899049-Paper-Conference.pdf

Chicago

Salvi, C., M. Lemercier, and A. Gerasimovics. 2022. “Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics”. Advances in Neural Information Processing Systems 35: 1333–44. https://proceedings.neurips.cc/paper_files/paper/2022/file/091166620a04a289c555f411d8899049-Paper-Conference.pdf.

Harvard

Salvi, C., Lemercier, M. and Gerasimovics, A. (2022) “Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics”, Advances in Neural Information Processing Systems. Curran Associates, Inc., pp. 1333–1344. Available at: https://proceedings.neurips.cc/paper_files/paper/2022/file/091166620a04a289c555f411d8899049-Paper-Conference.pdf.

Vancouver

1. Salvi C, Lemercier M, Gerasimovics A (2022) Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics. In: Advances in Neural Information Processing Systems. Curran Associates, Inc., pp 1333–1344

BibTeX

@inproceedings{salvi2022neural,
  title = {Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics},
  author = {Salvi, Cristopher and Lemercier, Maud and Gerasimovics, Andris},
  year = {2022},
  booktitle = {Advances in Neural Information Processing Systems},
  publisher = {Curran Associates, Inc.},
  volume = {35},
  pages = {1333-1344},
  url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/091166620a04a289c555f411d8899049-Paper-Conference.pdf}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Neural SPDE solution-operator architecture

Knowl 2 — Fourier-space ODE evaluation

Knowl 3 — Fourier fixed-point evaluation

Knowl 4 — Space-time resolution invariance and partial-observation processing

Knowl 5 — Computational complexity and evaluation trade-offs

Knowl 6 — Scope and convergence limitation of mollified infinite-dimensional noise

Knowl 7 — Supervised learning tasks and benchmark protocol

Knowl 8 — Ginzburg–Landau accuracy across data regimes

Knowl 9 — Stochastic Korteweg–De Vries accuracy and robustness to subsampling

Knowl 10 — Stochastic Navier–Stokes performance and zero-shot super-resolution

References

Citation

BibTeX

Source Code

Access the Paper