Exercise 3.5: Examine stability of a diffusion model with a source term Consider a diffusion equation with a linear u term
3.6 Diffusion in 2D
3.6.15 Direct Versus Iterative Methods
# Red points ic = slice(1,-1,2) im1 = slice(0,-2,2) ip1 = slice(2,None,2) jc = slice(1,-1,2) jm1 = slice(0,-2,2) jp1 = slice(2,None,2) u_new[ic,jc] = update(
u_new, u_n, ic, im1, ip1, jc, jm1, jp1) ic = slice(2,-1,2)
im1 = slice(1,-2,2) ip1 = slice(3,None,2) jc = slice(2,-1,2) jm1 = slice(1,-2,2) jp1 = slice(3,None,2) u_new[ic,jc] = update(
u_new, u_n, ic, im1, ip1, jc, jm1, jp1)
# Black points ic = slice(2,-1,2) im1 = slice(1,-2,2) ip1 = slice(3,None,2) jc = slice(1,-1,2) jm1 = slice(0,-2,2) jp1 = slice(2,None,2) u_new[ic,jc] = update(
u_new, u_n, ic, im1, ip1, jc, jm1, jp1) ic = slice(1,-1,2)
im1 = slice(0,-2,2) ip1 = slice(2,None,2) jc = slice(2,-1,2) jm1 = slice(1,-2,2) jp1 = slice(3,None,2) u_new[ic,jc] = update(
u_new, u_n, ic, im1, ip1, jc, jm1, jp1)
# Relax
c = slice(1,-1)
u[c,c] = omega*u_new[c,c] + (1-omega)*u_[c,c]
The functionsolver_classic_iterativeindiffu2D_u0.pycontains a uni- fied implementation of the relaxed Jacobi and SOR methods in scalar and vectorized versions using the techniques explained above.
The most common direct method today is to use theLU factorizationprocedure to factor the coefficient matrixAas the product of a lower-triangular matrixL(with unit diagonal terms) and an upper-triangular matrixU: ADLU. As soon as we haveLandU, a system of equationsLUc D bis easy to solve because of the triangular nature ofLandU. We first solveLy Dbfory (forward substitution), and thereafter we findcfrom solvingUcDy(backward substitution). WhenAis a denseNN matrix, the LU factorization costs13N3arithmetic operations, while the forward and backward substitution steps each require of the orderN2arithmetic operations. That is, factorization dominates the costs, while the substitution steps are cheap.
Symmetric, positive definite coefficient matrices often arise when discretizing PDEs. In this case, the LU factorization becomesA D LLT, and the associated algorithm is known asCholesky factorization. Most linear algebra software offers highly optimized implementations of LU and Cholesky factorization as well as for- ward and backward substitution (scipy.linalgis the relevant Python package).
Finite difference discretizations lead to sparse coefficient matrices. An extreme case arose in Sect.3.2.1 whereAwas tridiagonal. For a tridiagonal matrix, the amount of arithmetic operations in the LU and Cholesky factorization algorithms is just of the orderN, notN3. Tridiagonal matrices are special cases ofbanded matrices, where the matrices contain just a set of diagonal bands. Finite difference methods on regularly numbered rectangular and box-shaped meshes give rise to such banded matrices, with 5 bands in 2D and 7 in 3D for diffusion problems.
Gaussian elimination only needs to work within the bands, leading to much more efficient algorithms.
IfAi;j D 0forj > i Cp andj < i p, p is thehalf-bandwidth of the matrix. We have in our 2D problempDNxC2, while in 3D,pD.NxC1/.NyC 1/C2. The cost of Gaussian elimination is then O.Np2/, so with p N, we see that banded matrices are much more efficient to compute with. By reordering the unknowns in clever ways, one can reduce the work of Gaussian elimination further. Fortunately, the Python programmer has access to such algorithms through thescipy.sparse.linalgpackage.
Although a direct method is an exact algorithm, rounding errors may in practice accumulate and pollute the solution. The effect grows with the size of the linear system, so both for accuracy and efficiency, iterative methods are better suited than direct methods for solving really large linear systems.
Iterative methods The Jacobi and SOR iterative methods belong to a class of it- erative methods where the idea is to solveAu Dbby splitting A into two parts, ADM N, such that solving systemsM uD cis easy and efficient. With the splitting, we get a system
M uDN uCb;
which suggests an iterative method
M urC1DN ur Cb; r D0; 1; 2; : : : ;
whereurC1 is a new approximation touin therC1-th iteration. To initiate the iteration, we need a start vectoru0.
The Jacobi and SOR methods are based on splittingAinto a lower tridiagonal partL, the diagonalD, and an upper tridiagonal partU, such thatADLCDCU. The Jacobi method corresponds toM DDandN D LU. The Gauss-Seidel method employsM DLCDandN D U, while the SOR method corresponds to
M D 1
!DCL; N D1!
! DU : The relaxed Jacobi method has similar expressions:
M D 1
!D; N D 1!
! DLU :
With the matrix forms of the Jacobi and SOR methods as written above, we could in an implementation alternatively fill the matrixAwith entries and call general implementations of the Jacobi or SOR methods that work on a systemAu D b.
However, this is almost never done since forming the matrixArequires quite some code and storingAin the computer’s memory is unnecessary. It is much easier to just apply the Jacobi and SOR ideas to the finite difference stencils directly in an implementation, as we have shown in detail.
Nevertheless, the matrix formulation of the Jacobi and SOR methods have been important for analyzing their convergence behavior. One can show that the error ur ufulfills ur u D Gr.u0 u/, where G D M1N andGk is a matrix exponential. For the method to converge, limr!1jjGrjj D 0is a necessary and sufficient condition. This implies that thespectral radiusofG must be less than one. SinceG is directly related to the finite difference scheme for the underlying PDE problem, one can in principle compute the spectral radius. For a given PDE problem, however, this is not a practical strategy, since it is very difficult to de- velop useful formulas. Analysis of model problems, usually related to the Poisson equation, reveals some trends of interest: the convergence rate of the Jacobi method goes likeh2, while that of SOR with an optimal!goes likeh, wherehis the spa- tial spacing: h Dx Dy. That is, the efficiency of the Jacobi method quickly deteriorates with the increasing mesh resolution, and SOR is much to be preferred (even if the optimal!remains an open question). We refer to Chapter 4 of [16] for more information on the convergence theory. One important result is that ifAis symmetric and positive definite, then SOR will converge for any0 < ! < 2.
The optimal!parameter can be theoretically established for a Poisson problem as
!oD 2
1Cp
1%2; %D cos.=Nx/C.x=y/2cos.=Ny/
1C.x=y/2 : (3.112) This formula can be used as a guide also in other problems.
The Jacobi and the SOR methods have their great advantage of being trivial to implement, so they are obviously popular of this reason. However, the slow convergence of these methods limits the popularity to fairly small linear systems (i.e., coarse meshes). As soon as the matrix size grows, one is better off with more sophisticated iterative methods like the preconditioned Conjugate gradient method, which we now turn to.
Finally, we mention that there is a variant of the SOR method, called theSymmet- ric Successive Over-relaxationmethod, known as SSOR, where one runs a standard SOR sweep through the mesh points and then a new sweep while visiting the points in reverse order.