Understanding the Pillars of Numerical Schemes: Consistency, Stability, and Convergence

Author: Nishant Soni

In the realm of numerical methods, solving Partial Differential Equations (PDEs) plays a fundamental role in various scientific and engineering applications. However, directly solving PDEs analytically can be a daunting task for many complex problems. Hence, numerical schemes come to the rescue, offering a way to approximate solutions to these intricate equations. But how do we ensure that these numerical approximations are reliable and accurate? This blog delves into three essential concepts – Consistency, Stability, and convergence, which form the foundation for validating numerical schemes. The concepts will be explained in the context of 1-dimensional diffusion equation for simplicity.

Consistency and its significance in numerical solutions
Consistency is the key to establishing the reliability of numerical schemes. A numerical scheme is considered consistent with the original PDE if, as the grid spacing approaches zero, the numerical solution approaches the exact solution of the PDE. In simpler terms, the numerical scheme should mimic the behavior of the original PDE as we refine the grid.
Consistency pertains to the behavior of discretized equations as the grid spacing ( $\Delta t$ and $\Delta x$ ) approaches zero. In simpler terms, it addresses how closely the numerical approximations align with the original differential equations they are based on. Though seemingly straightforward, this concept leads us to two crucial properties of numerical discretizations: the truncation error and the equivalent differential equation (EDE).
The truncation error of a numerical scheme represents the discrepancy between the exact solution of the original differential equation and the numerical approximation. Simultaneously, the associated equivalent differential equation (EDE) emerges as a vital element in understanding the relationship between the numerical scheme and the true solution. It offers a fresh perspective on how these numerical and exact solutions are connected through the truncation error.
Upon careful analysis, it becomes evident that the numerical solution does not exactly satisfy the mathematical model equations due to discretization errors. Instead, it serves as a solution to the equivalent differential equation. This crucial revelation implies that gaining insights into the properties of the solution to the EDE will provide us valuable information about the behavior and errors associated with the numerical solution.
In essence, consistency underscores the convergence of the numerical scheme to the original differential equation and emphasizes the importance of understanding the EDE to better grasp the behavior of the numerical solution and its associated errors.

Consider the 1-dimensional diffusion equation:

$\frac{\partial u}{\partial t} = D \frac{\partial^2 u}{\partial x^2} \hspace{60pt} (1)$

We will discretize it using the Forward in Time and Central in Space (FTCS) method. The discretized equation is:

$\frac{u_i^{n+1} - u_i^n}{\Delta t} = D \frac{u_{i+1}^n -2u_i^n + u_{i-1}^n}{\Delta x^2} \hspace{60pt} (2)$

which yields,

$u_i^{n+1} = u_i^n + \frac{D {\Delta t}}{\Delta x^2} \left( u_{i+1}^n -2u_i^n + u_{i-1}^n \right) \hspace{60pt} (3)$

In order to obtain the equivalent differential equation (EDE) we need to follow the steps outlined below.
Step 1: We write the following Taylor expansions,

$u_i^{n+1}=u_i^n+\Delta t\left(u_t\right)i^n+\frac{\Delta t^2}{2}\left(u{t t}\right)i^n+\frac{\Delta t^3}{6}\left(u{t t t}\right)_i^n+\cdots$

$u_{i+1}^n=u_i^n+\Delta x\left(u_x\right)i^n+\frac{\Delta x^2}{2}\left(u{x x}\right)i^n+\frac{\Delta x^3}{6}\left(u{x x x}\right)_i^n+\cdots$

$u_{i-1}^n=u_i^n-\Delta x\left(u_x\right)i^n+\frac{\Delta x^2}{2}\left(u{x x}\right)i^n-\frac{\Delta x^3}{6}\left(u{x x x}\right)_i^n+\cdots$

where the $x$ and $t$ subscripts indicate partial derivatives.

Step 2: We substitute the above expansions into the equation (3), obtaining

$\frac{u_i^{n+1}-u_i^n}{\Delta t}-D \frac{u_{i+1}^n-2 u_i^n+u_{i-1}^n}{\Delta x^2}-\left(u_t+D u_{x x}\right)_i^n=\frac{\Delta t}{2}\left(u_{t t}\right)_i^n-D \Delta x^2\left(u_{x x x x}\right)_i^n+O\left(\Delta t^2, \Delta x^4\right) \hspace{60pt} (4)$

The right-hand side of this consistency relation represents the truncation error $\epsilon_T$ , equal to

$\epsilon_T=\frac{\Delta t}{2}\left(u_{t t}\right)i^n-D \Delta x^2\left(u{x x x x}\right)_i^n+O\left(\Delta t^2, \Delta x^4\right) \hspace{60pt} (5)$

The truncation error (TE) is therefore defined as the difference between the numerical scheme and the differential equation.
The associated equivalent differential equation (EDE) is,

$u_t=D u_{x x}+D \Delta x^2\left(1-\frac{1}{2} \frac{D \Delta t}{\Delta x^2}\right) u_{x x x x}+O\left(\Delta t^2, \Delta x^4\right) \hspace{60pt} (6)$

The observation from the equation 5 reveals that as the grid spacing ( $\Delta t$ and $\Delta x$ ) approaches zero, the right-hand side of the FTCS scheme for the diffusion equation becomes negligible. This behavior indicates that the FTCS scheme demonstrates consistency with the original differential equation, as it converges to the correct solution in the limit of infinitesimally small grid spacings. As a result, the scheme exhibits first-order accuracy in time and second-order accuracy in space, signifying that the error decreases linearly with $\Delta t$ and quadratically with $\Delta x$ .

Stability and Von Neumann Stability Analysis
Stability is another critical aspect of numerical schemes, ensuring that small errors in the initial conditions or numerical approximations do not lead to excessive growth in the solution. The stability condition can be defined by considering the error, denoted as $\bar{\epsilon_i^n}$ , which represents the difference between the computed solution $u_i^n$ and the exact solution of the discretized equation $\bar{u_i^n}$ :

$\bar{\epsilon_i^n} = u_i^n - \bar{u_i^n} \hspace{60pt} (7)$

To meet the stability requirement, it is essential that any such error $\bar{\epsilon_i^n}$ between the computed solution and the exact solution should remain uniformly bounded as the time step $n$ approaches infinity, while keeping the time-step size $\Delta t$ fixed. In other words, the errors should not grow uncontrollably with increasing time steps, ensuring the numerical solution remains reliable and well-behaved throughout the computation.
The Von Neumann stability analysis is a powerful technique used to determine the stability of a numerical scheme. Applying the Von Neumann stability analysis to the 1-D FTCS scheme for the diffusion equation, we can derive stability criteria. These criteria provide constraints on the time step and spatial grid spacing to ensure a stable and well-behaved numerical solution.
It can be shown that the evolution of error ( $\epsilon$ ) in the solution also follows the same difference equation i.e.,

$\epsilon_i^{n+1} = \epsilon_i^n + \frac{D\Delta t}{\Delta x^2} (\epsilon_{i+1}^n -2\epsilon_i^n + \epsilon_{i-1}^n) \hspace{60pt} (8)$

Based on the Von-Neumann stability analysis, the amplification factor of error in each time step at any point `i’ can be given by the following formula,

$G = 1 - 4\frac{D\Delta t}{\Delta x^2} \sin^2\theta \hspace{60pt} (9)$

where $\sin^2 \theta < 1$ . Thus for the scheme to remain numerically stable, we must have $|G| \leq 1$ which leads to the following condition that

$\frac{D\Delta t}{\Delta x^2} \leq \frac{1}{2} \hspace{60pt} (10)$

The quantity $\frac{D\Delta t}{\Delta x^2}$ is commonly known as the Fourier number or CFL number for diffusion equation. Since the choice of $\Delta x$ is usually decided based on the accuracy considerations, the maximum allowable $\Delta t$ is decided by the CFL limit once $\Delta x$ is fixed. Any choice of $\Delta t$ above this prescribed limit leads to divergence of the solution.

Hence, It can be concluded that the FTCS scheme for diffusion equation is conditionally stable wherein the CFL number always has to be less than a certain value prescribed by the stability criterion.

Convergence and ensuring accurate results
Convergence is the final piece of the puzzle, ensuring that the numerical solution approaches the exact solution of the PDE as the grid is refined. When a numerical scheme is both consistent and stable, it has the potential for convergence.
Convergence is a condition on the numerical solution: we have to be sure that the output of the simulation is a correct representation of the model we solve, i.e. the numerical solution must tend to the exact solution of the mathematical model, when time and space steps tend to zero.
By establishing that the FTCS scheme for the 1-D diffusion equation meets the conditions of consistency and stability, we can demonstrate that the numerical solution obtained through this scheme converges to the true solution as the grid is refined, as shown in the figures below.

It is evident that the computed solution matches well with the exact solution and the numerical error approaches zero at a rate proportional to $\Delta x^{2}$ as the grid spacing $\Delta x$ is refined.
The conditions of consistency, stability and convergence are related to each other and the precise relation is contained in the fundamental Equivalence Theorem of Lax.

Equivalence Theorem of Lax
For a well-posed initial value problem and a consistent discretization scheme, stability is the necessary and sufficient condition for convergence.
This fundamental theorem highlights the essential steps required to analyze a time-dependent or initial value problem using numerical schemes. These steps are as follows:

First, it is necessary to examine the consistency condition, which allows us to determine the accuracy order of the numerical scheme and its associated truncation error. By scrutinizing the consistency, we gain insights into how well the numerical approximation reflects the original problem.
Secondly, the stability properties of the numerical scheme need to be assessed. Analyzing stability ensures that the errors in the numerical solution do not grow uncontrollably over time, guaranteeing a well-behaved and reliable solution.

By successfully completing these two steps, convergence can be established without requiring additional analysis. Convergence is crucial as it confirms that as we refine the grid and reduce the step sizes, the numerical solution approaches the true solution of the problem we aim to simulate.
The most significant advantage of this theorem lies in its ability to simplify the analysis process. It asserts that testing the stability of a consistent numerical scheme is adequate to ensure that the resulting numerical solution is a valid representation of the reality we are trying to simulate numerically. This property saves us from the need to conduct further analyses, streamlining the process of validating the numerical solution’s accuracy and reliability.

Conclusion
In summary, the successful numerical solution of PDEs hinges on three fundamental concepts – Consistency, Stability, and Convergence. By understanding and validating these pillars, we can confidently use numerical schemes to solve complex problems in various fields. The 1-dimensional diffusion equation and its FTCS discretization serve as a concrete example to illustrate the significance of these concepts in practical applications.
As you continue your exploration of numerical methods, always remember that ensuring consistency, stability, and convergence is crucial for obtaining reliable and accurate results, thus unlocking the full potential of numerical simulations in tackling real-world challenges.

References
1. “Numerical Computation of Internal and External Flows”, 2nd Edition by Charles Hirsch
2. Computational Methods for Fluid Dynamics, Ferziger and Peric
3. Computational Gas Dynamics, Culbert Laney, Cambridge Press

Convergence in CFD refers to the process where the solution of the simulation steadily approaches a consistent value as the number of iterations increases. A converged solution indicates that the simulation has stabilized and is no longer changing significantly with further iterations, signalling that the results are reliable. Achieving convergence is often linked to factors like mesh quality, boundary conditions, time step size, and numerical methods.

One of the key factors contributing to the stability of CFD simulations is the Courant-Friedrichs-Lewy (CFL condition), a crucial condition that governs the simulation. The CFL condition ensures that the solution updates are synchronized with the physical constraints of the flow, which is critical for maintaining stability and accuracy in the simulation. By adhering to the CFL condition, CFD experts can produce reliable results for a variety of engineering applications. In this discussion, we will explore the importance of the CFL condition and its role in CFD simulations.

Understanding the Pillars of Numerical Schemes: Consistency, Stability, and Convergence

Responses

India Office

UK Office (Main)

Support Center

Related Articles

Responses