Depending on the chosen time discretization of (5.1), the mathematical problem to be solved at each time level will be either a linear algebraic equation or a nonlinear algebraic equation. Despite the simplicity of (5.1), the conclusions reveal typical features of different methods in much more complicated nonlinear PDE problems. In this case, the nonlinearity in the original equation does not present difficulties in the discrete algebraic equation.
The unknown in the algebraic equation is denoted by u, while u.1/ is the value of the unknown at the previous time level (in general, u.`/ is the value of the unknown levels back in time). In this simple case, however, we can mathematically analyze the roots and give the answer. The Picard iteration for solving the nonlinear equation resulting from the inverse Euler discretization of the logistic equation can be written as
The repetition starts with the value of the unknown at the previous time level: uDu.1/. Newton's method, expressed by the recurrence index in the unknown, takes a more familiar mathematical form. The reason Newton's method is much better than Picard's iteration in the top left graph is that Newton's one-step method gets far below the r tolerance, while Picard's iteration takes an average of 7 iterations to bring the residual down to D101, which does not give enough accuracy when solving a nonlinear equation.
Ideally, one should estimate the error in the time discretization as the solution progresses and tune accordingly.
Generalization to a General Nonlinear ODE
Note The simple Crank-Nicolson geometric mean method for the quadratic nonlinearity gives visually more accurate solutions than the inverse Euler discretization. Even with a tolerance of D 103, all methods for dealing with nonlinearities in the backward Euler discretization give indistinguishable plots. Explicit treatment off usually implies stricter conditions for achieving stability of time discretization schemes.
The same applies to iteration techniques for nonlinear algebraic equations: the "less" we linearize (ie, the more we keep in the original formula), the faster convergence can be. We can say, daf .u; t /Du3 is handled explicitly if we evaluate f as.u/3, partially implicitly if we linearize as.u/2u, and fully implicitly if we representf byu3. Of course, a fully implicit representation will require further linearization, but with .u; t /Du2a completely implicit treatment is possible if the resulting quadratic equation is solved with the formula.).
For ODEu0 D u3 with f .u; t / D u3 and coarse time resolution t D0:4, Picard iteration with .u/2 requires 8 iterations with D103 for the first time step, while .u/3 leads to 22 iterations. After approx. 10 time steps, both approaches are down to approx. 2 iterations per time step, but this example shows a potential to treatf more implicitly. Starting with the solution at the previous time level, uDu.1/, we can just use the standard formula.
1t@[email protected]; tn/ : (5.11) Crank-Nicolson discretization The standard Crank-Nicolson scheme with arithmetic mean approximation off takes the form.
Systems of ODEs We may write a system of ODEs
Using the notation0andu1 for the unknown unC10 andunC11 in this system, writing u.1/0 and u.1/1 for the previous values un0 and un1, multiplying by t and moving the terms to the left-hand sides, gives. 2t .u0Cu.1/0 /D0 : (5.19) It is clear that we have a need to solve systems of nonlinear algebraic equations, which is the subject of the next section.
Systems of Nonlinear Algebraic Equations
- Picard Iteration
- Newton’s Method
- Stopping Criteria
- Example: A Nonlinear ODE Model from Epidemiology
Next, we will explain how Picard iteration and Newton's method can be applied to systems such as F .u/ D 0 and A.u/u D b.u/. More theoretical considerations, including quite general results on convergence properties of these methods, can be found in Kelley [8]. We cannot apply Picard iteration to nonlinear equations unless there is a special structure.
In other words, we solve a system of nonlinear algebraic equations as a series of linear systems. Until convergence” means that the iteration is stopped when the change in the unknown, jjuujj, or the residual jjA.u/ubjj, is sufficiently small, see Section 5.2.3 for more details. We realize that the Jacobian needed in Newton's method consists of A.u/as in the Picard iteration plus two additional terms arising from the differentiation.
Here we have inserted a parameter such that D0 gives the Picard system and D 1 gives the Newton system. Relative change in solution: jjuujj ujju0jj, whereu0 denotes the starting value of u in the iteration. To prevent divergent iterations from running forever, one terminates the iterations when the current number of iterations exceeds a maximum value kmax.
The relative criteria are most widely used, as they are not sensitive to the characteristic size of u. Nevertheless, the relative criteria can be misleading when the starting value of the iteration is very close to the solution, as an unnecessary reduction of the error measure is imposed. It is common to combine the absolute and relative measures of residual size, as in
With a very good initial guess for the iteration (typically the solution of a differential equation at the previous time level), the termjjF .u0/jjis small andra is the dominant tolerance. A very simple model of the spread of a disease, such as a flu, takes the form of a 22ODE system.
Linearization at the Differential Equation Level
- Explicit Time Integration
- Backward Euler Scheme and Picard Iteration
- Backward Euler Scheme and Newton’s Method
- Crank-Nicolson Discretization
- Finite Difference Discretization
- Solution of Algebraic Equations
A typical linearization of the .˛.un/run/term in iterationkC1 is to use the previously calculated approximation of the diffusion coefficient: ˛.un;k/. At the beginning of the iteration we start with the value of the previous time level: uDu.1/, and after each iteration u is updated. Normally Newton's method is defined for systems of algebraic equations, but the idea of the method can also be applied at the PDE level.
The term˛0.un;k/ıurıuis of orderıu2en therefore omitted as we expect the correctionıu to be small (ıuıu2). C r .˛0.un;k/ıurun;k/Cf0.un;k/ıu : (5.40) Note thatıF is a linear function ofıu, and F only contains terms that are known, so that the PDE forıu is indeed linear is. Similarity to Picard iteration We can also rewrite the PDE in a slightly different way if we define;kCıuasun;kC1.
C r .˛0.un;k/ıurun;k/Cf0.un;k/ıu : (5.41) Note that the first line is the same PDE as generated by the Picard iteration, while the remaining terms come from differentiations that are an integral component of Newton's method. The form that classifies PDEs as Picard iteration terms and derivative terms of Newton's method becomes. Alternatively, we can perform a space discretization of the time-discrete nonlinear PDE problem and obtain a system of nonlinear algebraic equations that can be solved by Picard iteration or Newton's method, as presented in Section 5.2.
The non-linearity in the differential equation (5.50) poses no more problems than a variable coefficient, as in the term.˛.x/u0/0. Alternatively, one can use a ghost cellŒx; 0 and update dieu1 value in the ghost cell according to (5.57) after each Picard or Newton iteration. As for the last equation, its form depends on whether we include the Dirichlet conditionu.L/ D D, which meansuNx D D, in the nonlinear algebraic system of equations or not.
Picard Iteration The obvious Picard iteration scheme is to use the previously calculated values of ui in A.u/ and b.u/, as described in more detail in Sect. Here this means that we have to differentiate F .u/DA.u/ub.u/ with respect to the unknown parametersu0; u1; : : : ; um (m DNx or DNx1, depending on whether the Dirichlet condition is included in the nonlinear systemF .u/ D0 or not). A similar substitution of u1anduNx must be made in the Jacobian for the first and last rows.
KuruNx is included as an unknown, the last row in the Jacobian should help enforce the uNx D0 condition, since we assume it contains the appropriate Dirichlet value at the start of the iteration (uNx D D), and then the Newton update should be zero foriD0, i.e. , uNxD0. We have seen and can see from the present example that the linear system in Newton's method contains all the terms present in the system that arises in the Picard iteration method.
Multi-Dimensional Nonlinear PDE Problems
Finite Difference Discretization
We first define the nonlinear algebraic equations to be solved, drop the superscriptn(useuforun) and introduceu.1/forun1:. It is convenient to work with two indices in 2D discretizations of finite differences, but it complicates the derivation of the Jacobian, which then gets four indices. Inserting the latest approximationu foru into the J and F formulas and then forming J ıuD F produces the linear system to be solved in each Newton iteration.
Boundary conditions will affect the formulas when any of the indices coincide with a boundary value of an index.
Continuation Methods
Operator Splitting Methods
Ordinary Operator Splitting for ODEs
Strang Splitting for ODEs
Example: Logistic Growth Let us split the (scaled) logistic equation
