The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems

Weinan EBing Yu

article2017Communications in Mathematics and Statistics1,955 citations

Proposes the Deep Ritz method, a numerical framework that solves variational problems and high-dimensional partial differential equations by training neural networks to minimize energy formulations using stochastic gradient descent.

Listen

Traditional numerical methods for solving partial differential equations often struggle with complex geometries, singularities, and high-dimensional spaces. Because conventional grid-based techniques suffer from severe computational scaling issues when problem dimensions increase, finding flexible, scalable alternatives is critical for computational science and engineering applications.

The article introduces and evaluates the Deep Ritz Method, a deep learning-based framework designed to solve variational problems and partial differential equations. The objective is to demonstrate how deep neural networks can approximate trial functions within a Ritz formulation and scale effectively across different dimensions.

The approach integrates deep residual networks to represent trial functions and uses stochastic gradient descent with random mini-batch spatial sampling for numerical integration. The authors evaluate this technique across several benchmark problems, including two-dimensional Poisson equations with corner singularities, high-dimensional Poisson problems up to 100 dimensions, Neumann boundary condition setups, and quantum eigenvalue problems such as the infinite potential well and harmonic oscillator.

The experiments yield several key findings. First, in low-dimensional singular problems, the Deep Ritz Method achieved higher accuracy than the standard finite difference method while requiring significantly fewer parameters (for example, achieving a 0.0072 relative error with 811 parameters compared to 0.0125 error with 625 parameters in finite difference). Second, the method successfully scaled to high dimensions, solving a 10-dimensional Poisson problem to roughly 0.4% relative error and a 100-dimensional problem to about 2.2% error. Third, transfer learning accelerated early-stage training when system forcing terms changed. Finally, for eigenvalue estimation, the framework maintained accuracy in low to moderate dimensions (0.11% to 1.6% error in 5 dimensions), though errors degraded in 10-dimensional eigenvalue benchmarks (up to 12.6% error).

These findings indicate that neural network representations provide a naturally adaptive, nonlinear framework capable of circumventing the curse of dimensionality for many partial differential equations. This has significant implications for reducing modeling complexity and expanding computational feasibility in fields like quantum mechanics and high-dimensional physics without requiring tailored spatial meshing.

Organizations evaluating this approach should consider pilot implementations on high-dimensional or singularity-prone problems where standard finite element or finite difference methods are intractable. For operational deployment, teams should combine the method with transfer learning pipelines to minimize initial training time when evaluating recurring problem variants.

Despite these strengths, confidence in the method should be balanced against key limitations. Transforming the variational formulation into a neural network parameter optimization introduces non-convex loss landscapes with local minima, and strict mathematical convergence rates remain unproven. Furthermore, handling essential boundary conditions requires penalty terms, and performance degrades in high-dimensional eigenvalue settings, indicating that further refinements to architectures and optimization strategies are necessary before broad production use.

Beijing Institute of Big Data Research Peking University Princeton University deep learning eigenvalue problem eigenvalues Deep Ritz method variational problems deep learning

arXiv: 1710.00211

Book: Convex Optimization: Algorithms and Complexity, Sébastien Bubeck (2015). Provides foundational principles and complexity guarantees for continuous optimization algorithms, which underpin the minimization of variational energies in deep learning solvers.
Paper: An overview of gradient descent optimization algorithms, Sebastian Ruder (2016). Provides a comprehensive foundation on stochastic gradient descent and adaptive optimization methods used to minimize the Deep Ritz variational loss.
Paper: Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming, Saeed Ghadimi et al. (2013). Establishes convergence properties of stochastic first-order optimization methods for nonconvex objectives that are essential for training neural networks on variational problem formulations.

Paper: An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications, Esteban Samaniego et al. (2019). Builds directly on the Deep Ritz variational philosophy to formulate the Deep Energy Method for structural and continuum mechanics PDEs.
Paper: DGM: A deep learning algorithm for solving partial differential equations, Justin Sirignano et al. (2017). Explores the complementary strong-form collocation approach for solving high-dimensional partial differential equations via deep neural networks.
Paper: DeepXDE: A Deep Learning Library for Solving Differential Equations, Lu Lu et al. (2019). Implements an open-source software platform uniting physics-informed and variational deep learning solvers across forward and inverse differential equation benchmarks.
Paper: Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next, Salvatore Cuomo et al. (2022). Synthesizes the development and applications of deep learning methods for solving PDEs, surveying the landscape established by foundational techniques like the Deep Ritz method.
Paper: Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Lu Lu et al. (2021). Generalizes deep learning solvers from finding individual PDE solutions to learning nonlinear solution operators across infinite-dimensional function spaces.
Paper: Fourier Neural Operator for Parametric Partial Differential Equations, Zongyi Li et al. (2020). Advances deep scientific computing by learning mesh-independent operator mappings for parametric PDEs using Fourier neural representations.

Cover for The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems

Abstract

We propose a deep learning based method, the Deep Ritz Method, for numerically solving variational problems, particularly the ones that arise from partial differential equations. The Deep Ritz method is naturally nonlinear, naturally adaptive and has the potential to work in rather high dimensions. The framework is quite simple and fits well with the stochastic gradient descent method used in deep learning. We illustrate the method on several problems including some eigenvalue problems.

1 Introduction
2 The Deep Ritz Method
2.1 Building trial functions
2.2 The stochastic gradient descent algorithm and the quadrature rule
3 Numerical Results
3.1 The Poisson equation in two dimension
3.2 Poisson equation in high dimension
3.3 An example with the Neumann boundary condition
3.4 Transfer learning
3.5 Eigenvalue problems
4 Discussion
References

Knowls

Knowl 1 — The Deep Ritz Method for Variational Problems
model/method
The Deep Ritz method numerically solves partial differential equations by reformulating them as variational energy minimization problems and parameterizing the trial function using a deep neural network $u(x; \theta)$ with parameter vector $heta$ .

For a domain $\Omega \subset \mathbb{R}^d$ , a source term $f(x)$ , and Dirichlet boundary conditions $u(x) = g(x)$ on $\partial\Omega$ , the continuous variational problem is formulated with a boundary penalty term:

$I(u) = \int_\Omega \left( \frac{1}{2} |\nabla_x u(x)|^2 - f(x)u(x) \right) dx + \beta \int_{\partial\Omega} |u(x) - g(x)|^2 ds$

where $\beta > 0$ is a penalty parameter enforcing the essential boundary condition.

Rather than discretizing the domain with a fixed grid or mesh, the spatial integrals are approximated via Monte Carlo numerical quadrature using randomly sampled mini-batches of points in $\Omega$ and on $\partial\Omega$ at each iteration of stochastic gradient descent (SGD or Adam). This allows the method to remain mesh-free and naturally adaptive across different spatial regions.
Knowl 2 — Residual Network Architecture with Smooth Activation for Deep Ritz
model/method
The trial function $u(x; \theta)$ in the Deep Ritz method is constructed using a deep residual neural network. Given an input coordinate $x \in \mathbb{R}^d$ , it is mapped to a hidden representation $s \in \mathbb{R}^m$ (using zero-padding if $d < m$ or a linear projection if $d > m$ ).

The network comprises a stack of residual blocks $f_i: \mathbb{R}^m \to \mathbb{R}^m$ , where each block consists of two linear transformations, two activations, and a skip connection:

$f_i(s) = \phi\left(W_{i,2} \cdot \phi(W_{i,1} s + b_{i,1}) + b_{i,2}\right) + s$

with weight matrices $W_{i,1}, W_{i,2} \in \mathbb{R}^{m \times m}$ , bias vectors $b_{i,1}, b_{i,2} \in \mathbb{R}^m$ , and element-wise activation function $\phi$ .

To ensure $C^2$ smoothness necessary for computing spatial derivatives $\nabla_x u(x; \theta)$ while maintaining gradient flow, the smooth cubic rectified activation function is used:

$\phi(x) = \max\{x^3, 0\}$

After passing through $n$ sequential blocks $z_\theta(x) = (f_n \circ \dots \circ f_1)(x)$ , the scalar trial solution is computed by a linear output layer:

$u(x; \theta) = a \cdot z_\theta(x) + b$

where $a \in \mathbb{R}^m$ and $b \in \mathbb{R}$ , and $\theta$ denotes the complete set of parameters $\{W_{i,j}, b_{i,j}, a, b\}$ .
Knowl 3 — Deep Ritz Optimization Algorithm
algorithm
The Deep Ritz algorithm trains a neural network parameterizing a trial function $u(x; \theta)$ by minimizing the variational loss using stochastic quadrature.
Input: Domain $\Omega \subset \mathbb{R}^d$ , boundary $\partial\Omega$ , source function $f(x)$ , boundary function $g(x)$ , penalty parameter $\beta$ , batch sizes $N_\Omega$ and $N_{\partial\Omega}$ , learning rate $\eta$ , total steps $K$
Output: Optimized network parameters $\theta$
Initialize network parameters $\theta^{(0)}$
for $k = 0, 1, \dots, K-1$ do
    Sample $N_\Omega$ points $\{x_{j}\}_{j=1}^{N_\Omega}$ uniformly and independently at random from $\Omega$
    Sample $N_{\partial\Omega}$ points $\{y_{j}\}_{j=1}^{N_{\partial\Omega}}$ uniformly and independently at random from $\partial\Omega$
    Compute domain loss approximation:
         $L_\Omega(\theta^{(k)}) = \frac{\mathrm{Vol}(\Omega)}{N_\Omega} \sum_{j=1}^{N_\Omega} \left( \frac{1}{2} |\nabla_x u(x_j; \theta^{(k)})|^2 - f(x_j)u(x_j; \theta^{(k)}) \right)$
    Compute boundary loss approximation:
         $L_{\partial\Omega}(\theta^{(k)}) = \frac{\mathrm{Area}(\partial\Omega)}{N_{\partial\Omega}} \sum_{j=1}^{N_{\partial\Omega}} |u(y_j; \theta^{(k)}) - g(y_j)|^2$
    Evaluate total loss: $L(\theta^{(k)}) = L_\Omega(\theta^{(k)}) + \beta L_{\partial\Omega}(\theta^{(k)})$
    Compute parameter gradient: $G^{(k)} = \nabla_\theta L(\theta^{(k)})$
    Update parameters: $\theta^{(k+1)} = \mathrm{OptimizerUpdate}(\theta^{(k)}, G^{(k)}, \eta)$
end for
return $\theta^{(K)}$
In standard implementations, $\mathrm{OptimizerUpdate}$ is performed using the Adam optimizer.
Knowl 4 — Deep Ritz Method for Quantum Eigenvalue Problems
model/method
To compute the ground-state eigenvalue $\lambda_0$ of the Schrödinger-type eigenvalue problem $-\Delta u(x) + v(x)u(x) = \lambda u(x)$ on $\Omega$ with $u|_{\partial\Omega} = 0$ (where $v(x)$ is a potential function), the Deep Ritz method minimizes the Rayleigh quotient subject to normalization and boundary constraints.

The unconstrained objective functional is formulated with penalty terms:

$L(u(\cdot; \theta)) = \frac{\int_\Omega |\nabla u|^2 dx + \int_\Omega v u^2 dx}{\int_\Omega u^2 dx} + \beta \int_{\partial\Omega} u(x)^2 ds + \gamma \left( \int_\Omega u(x)^2 dx - 1 \right)^2$

where $\beta > 0$ penalizes boundary non-zero values, and $\gamma > 0$ stabilizes the $L_2$ -norm normalization. Keeping the Rayleigh quotient denominator $\int_\Omega u^2 dx$ alongside the penalty parameter $\gamma$ accelerates training and avoids requiring excessively large penalty weights.

For eigenvalue problems, a DenseNet-style architecture is used with pairwise cross-layer skip connections:

$y_i = \phi(W_{i-1}x_{i-1} + b_{i-1}), \quad x_i = [x_{i-1}; y_i]$

with quadratic rectified activation $\phi(x) = \max(0, x)^2$ ( $\mathrm{ReLU}^2$ ), which prevents gradient explosion during optimization.

Knowl 5 — Accuracy of Deep Ritz vs Finite Difference on 2D Corner Singularity Poisson Problem

data/table

The Deep Ritz Method (DRM) was compared with the standard Finite Difference Method (FDM) on the 2D domain $\Omega = (-1, 1)^2 \setminus [0, 1) \times \{0\}$ for the Poisson equation $\Delta u(x) = 0$ in $\Omega$ with exact solution $u(r, \theta) = r^{1/2} \sin(\theta/2)$ on $\partial\Omega$ . The solution exhibits a corner singularity at the origin where derivative magnitudes diverge.

Method	Number of Blocks	Parameters	Relative $L_2$ Error
DRM	3	591	0.0079
DRM	4	811	0.0072
DRM	5	1031	0.00647
DRM	6	1251	0.0057
FDM	-	625	0.0125
FDM	-	2401	0.0063

The results demonstrate that DRM achieves a smaller relative $L_2$ error than FDM with fewer parameters (e.g., DRM with 591 parameters yields 0.0079 error vs FDM with 625 parameters yielding 0.0125). This improvement in accuracy with fewer degrees of freedom is attributed to the natural nonlinear adaptivity of the neural network representation near the singularity.

Knowl 6 — High-Dimensional Poisson Equation Solving via Deep Ritz Method
empirical result
The Deep Ritz method was evaluated on high-dimensional Poisson problems on unit hypercubes $\Omega = (0, 1)^d$ :
1. Ten-dimensional Dirichlet problem ( $d=10$ ): $-\Delta u = 0 \quad \text{in } (0, 1)^{10}, \quad u(x) = \sum_{k=1}^5 x_{2k-1} x_{2k} \quad \text{on } \partial(0, 1)^{10}$ Using a network with 6 fully-connected layers, 3 skip connections (671 total parameters), $\beta = 10^3$ , and sampling 1,000 domain points plus 100 points per bounding hyperplane per step, the relative $L_2$ error was reduced to approximately $0.4\%$ after 50,000 SGD/Adam iterations.
2. One-hundred-dimensional Poisson problem ( $d=100$ ): $-\Delta u = -200 \quad \text{in } (0, 1)^{100}, \quad u(x) = \sum_{k=1}^{100} x_k^2 \quad \text{on } \partial(0, 1)^{100}$ Using 3 residual blocks of width $m=100$ , the relative $L_2$ error was reduced to approximately $2.2\%$ after 50,000 iterations.
These results confirm that the Deep Ritz method can solve high-dimensional PDEs without suffering from the grid-based curse of dimensionality.
Knowl 7 — Deep Ritz Method with Natural Neumann Boundary Conditions
empirical result
The Deep Ritz method naturally handles homogeneous Neumann boundary conditions without requiring penalty terms on the boundary.

For the $d$ -dimensional PDE system:

$-\Delta u(x) + \pi^2 u(x) = 2\pi^2 \sum_{k=1}^d \cos(\pi x_k) \quad \text{for } x \in [0, 1]^d$ $\frac{\partial u}{\partial n}\Big|_{\partial[0, 1]^d} = 0$

with exact solution $u(x) = \sum_{k=1}^d \cos(\pi x_k)$ , the energy functional is minimized directly over the domain:

$I(u) = \int_\Omega \left( \frac{1}{2} \left( |\nabla u(x)|^2 + \pi^2 u(x)^2 \right) - f(x)u(x) \right) dx$

Because the zero-flux condition is a natural boundary condition for this variational form, no boundary penalty integral is added. Using a residual neural network architecture, the trained model achieved a relative $L_2$ error of $1.3\%$ for $d = 5$ and $1.9\%$ for $d = 10$ after 50,000 iterations.
Knowl 8 — Transfer Learning in the Deep Ritz Method Across Forcing Functions
empirical result
Transferring pre-trained weights significantly accelerates the convergence of the Deep Ritz method when solving a PDE with a modified forcing function on the same domain.

When solving $-\Delta u(x) = 6(1+x_1)(1-x_1)x_2 + 2(1+x_2)(1-x_2)x_2$ on the slit domain $\Omega = (-1, 1)^2 \setminus [0, 1) \times \{0\}$ , network parameters were initialized using the pre-trained weights from the homogeneous Laplace equation $-\Delta u(x) = 0$ on the same domain.

Compared to random weight initialization, weight transfer resulted in:
1. A much steeper decrease in solution error during early training iterations.
2. Substantially smaller parameter change norms $\ln \|\Delta W\|_2^2$ across training iterations.
This indicates that transferring weights provides an effective warm-start strategy for parametric or sequentially varying PDEs.

Knowl 9 — Ground-State Eigenvalue Estimation Across Dimensions

data/table

The Deep Ritz method was tested for ground-state eigenvalue computation across dimensions $d \in \{1, 5, 10\}$ for an infinite potential well ( $v(x) = 0$ on $[0,1]^d$ , with exact eigenvalue $\lambda_0 = d\pi^2$ ) and a truncated harmonic oscillator ( $v(x) = |x|^2$ on $[-3, 3]^d$ , with exact eigenvalue $\lambda_0 = d$ ).

Problem	Dimension $d$	Exact $\lambda_0$	Approximate $\lambda_0$	Relative Error
Infinite Potential Well	1	9.87	9.85	0.20%
Infinite Potential Well	5	49.35	49.29	0.11%
Infinite Potential Well	10	98.70	92.35	6.43%
Harmonic Oscillator	1	1.00	1.0016	0.16%
Harmonic Oscillator	5	5.00	5.0814	1.60%
Harmonic Oscillator	10	10.00	11.26	12.60%

For lower dimensions ( $d=1, 5$ ), the relative errors remain below $2\%$ . However, as dimension increases to $d=10$ , accuracy degrades substantially (e.g., reaching $6.43\%$ error for the potential well and $12.6\%$ for the harmonic oscillator), highlighting optimization and domain truncation challenges in higher-dimensional eigenvalue problems.

Knowl 10 — Limitations of the Deep Ritz Method
limitation
The Deep Ritz method exhibits several core limitations:
1. Non-convex parameter optimization: Even when the underlying continuous variational problem $I(u)$ is convex, composing the objective with a nonlinear deep neural network parameterization yields a non-convex optimization objective $L(\theta)$ , exposing training to local minima and saddle points.
2. Absence of convergence rates: There is no established theoretical convergence rate for the combination of neural network variational approximation and stochastic gradient optimization.
3. Essential boundary enforcement: Essential (Dirichlet) boundary conditions must be enforced via penalty terms, introducing sensitive penalty hyperparameters (e.g., $\beta$ ) and potential optimization stiffness.
4. Scalability degradation on eigenvalue problems: In higher dimensions (e.g., $d=10$ ), eigenvalue estimation accuracy degrades significantly compared to low-dimensional cases.

Coverage note — No substantial contributed material was omitted. All formulations, network architectures, algorithms, benchmark experiments, transfer learning investigations, eigenvalue results, and stated limitations are covered.

References

1.I. Goodfellow, Y. Bengio and A. Courville, Deep Learning. MIT Press, 2016.
2.W. E, “A proposal for machine learning via dynamical systems”, Communications in Mathematics and Statistics, March 2017, Volume 5, Issue 1, pp 1-11.
3.J. Q. Han, A. Jentzen and W. E, “Overcoming the curse of dimensionality: Solving high-dimensional partial differential equations using deep learning”, submitted, arXiv:1707.02568.
4.W. E, J. Q. Han and A. Jentzen, “Deep learning-based numerical methods for highdimensional parabolic partial differential equations and backward stochastic differential equations”, submitted, arXiv:1706.04702.
5.C. Beck, W. E and Arnulf Jentzen, “Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations”, submitted. arXiv:1709.05963.
6.J. Q. Han, L. Zhang, R. Car and W. E, “Deep Potential: A general and “first-principle” representation of the potential energy”, submitted, arXiv:1707.01478.
7.L. Zhang, J.Q. Han, H. Wang, R. Car and W.E, “Deep Potential Molecular Dynamics: A scalable model with the accuracy of quantum mechanics”, submitted, arXiv:1707.09571.
8.L. C. Evans, Partial Differential Equations, 2nd ed. American Mathematical Society, 2010.
9.K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun, “Deep residual learning for image recognition”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 00, no. , pp. 770-778, 2016, doi:10.1109/CVPR.2016.90
10.D. P. Kingma, and J. Ba. “Adam: A method for stochastic optimization.” arXiv preprint arXiv:1412.6980, 2014.
11.G. Strang and G. Fix, An Analysis of the Finite Element Method. Prentice-Hall, 1973.
12.G. Huang, Z. Liu, K. Q. Weinberger, V. D. M. Laurens, “Densely connected convolutional networks.”, arXiv preprint arXiv:1608.06993, 2016.

Citation

MLA

E, W., and B. Yu. “The Deep Ritz Method: A Deep Learning-based Numerical Algorithm for Solving Variational Problems”. arXiv, 2017, http://arxiv.org/abs/1710.00211v1.

APA

E, W., & Yu, B. (2017). The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. arXiv. http://arxiv.org/abs/1710.00211v1

Chicago

E, W., and B. Yu. 2017. “The Deep Ritz Method: A Deep Learning-based Numerical Algorithm for Solving Variational Problems”. arXiv. http://arxiv.org/abs/1710.00211v1.

Harvard

E, W. and Yu, B. (2017) “The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1710.00211v1.

Vancouver

1. E W, Yu B (2017) The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. arXiv

BibTeX

@article{e2017the,
  title = {The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems},
  author = {E, Weinan and Yu, Bing},
  year = {2017},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1710.00211v1},
  eprint = {1710.00211}
}

Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — The Deep Ritz Method for Variational Problems

Knowl 2 — Residual Network Architecture with Smooth Activation for Deep Ritz

Knowl 3 — Deep Ritz Optimization Algorithm

Knowl 4 — Deep Ritz Method for Quantum Eigenvalue Problems

Knowl 5 — Accuracy of Deep Ritz vs Finite Difference on 2D Corner Singularity Poisson Problem

Knowl 6 — High-Dimensional Poisson Equation Solving via Deep Ritz Method

Knowl 7 — Deep Ritz Method with Natural Neumann Boundary Conditions

Knowl 8 — Transfer Learning in the Deep Ritz Method Across Forcing Functions

Knowl 9 — Ground-State Eigenvalue Estimation Across Dimensions

Knowl 10 — Limitations of the Deep Ritz Method

References

Citation

BibTeX

Access the Paper