About Extrapolation Methods

(3.4.6)

T_ik=T_i,k−1+ T_i,k−1−T_i−1,k−1 h_i−k

h_{i 2}

1−T_i,k−1−T_i−1,k−1 T_i,k−1−T_i−1,k−2

−1

, 1≤k≤i≤m.

The same triangular tableau arrangement is used as for polynomial ex- trapolation: k is the column index, and the recursion (3.4.6) relates each tableau element to its left-hand neighbors. The meaning ofT_ik, however, is now as follows: The functionsT_ik(h) are rational functions inh²,

T_ik(h) :=p₀+p₁h₂+· · ·+p_µh^2µ q₀+q₁h²+· · ·+q_νh^2ν, µ+ν =k, µ=ν or µ=ν−1, with the interpolation property

T_ik(h_j) =T(h_j), j =i−k, i−k+ 1, . . . , i.

We then define

T_ik:=T_ik(0),

and initiate the recursion (3.4.6) by putting T_i0 := T(h_i) for i = 0, 1, . . ., m, andT_i,−1 := 0 fori = 0, 1, . . ., m−1. The observed superiority of rational extrapolation methods reflects the more flexible approximation properties of rational functions [see Section 2.2.4].

In Section 3.5, we will illustrate how error estimates for extrapolation methods can be obtained from asymptotic expansions like (3.4.1). Under mild restrictions on the sequence of step lengths, it will follow that, for polynomial extrapolation methods based on even asymptotic expansions, the errors of T_i1 behave like h²_i, those of T_i1 like h²_i−1h²_i and, in general, those ofT_iklikeh²_i−kh²_i−k+1· · ·h²_i asi→ ∞. For fixedk, consequently, the sequenceT_ik i=k,k+ 1,. . ., approximates the integral like a method of order 2k+ 2. For the sequence (3.4.5a) a stronger result has been found:

(3.4.7) T_ik−

# _b

a

f(x)dx= (b−a)h²_i−kh²_i−k+1. . . h²_i(−1)^kB_2k+2

(2k+ 2)! f^(2k+2)(ξ), for a suitable ξ ∈ (a, b) and f ∈ C^2k+2[a, b] [see Bauer, Rutishauser and Stiefel (1963), Bulirsch (1964)].

of points whose distance — and consequently the coarseness of the sample

— was governed by a “step length”. To each such steph= 0 corresponded an approximate result T(h), which furthermore admitted an asymptotic expansion in powers of h. Analogous discretization methods are available for many other problems, of which the numerical integration of functions is but one instance. In all these cases, the asymptotic expansion of the result T(h) is of the form

(3.5.1) T(h) =τ₀+τ₁h^γ1+τ₂h^γ2+· · ·+τ_mh^γm+α_m+1(h)h^γm+1, 0< γ₁< γ₂<· · ·< γ_m+1,

where the exponents γ_i need not to be integers. The coefficients τ_i are independent ofh, the function α_m+1(h) is bounded for h→0, and τ₀ = lim_h→0T(h) is the exact solution of the problem at hand.

Consider, for example, numerical differentiation. Forh= 0, thecentral difference quotient

T(h) =f(x+h)−f(x−h) 2h

is an approximation to f(x). For functionsf ∈C^2m+3[x−a, x+a] and

|h| ≤ |a|, Taylors’s theorem gives T(h) =1

2h

(f(x) +hf(x) +h²

2!f(x) +· · ·+ h^2m+3

(2m+ 3)![f^(2m+3)(x) +o(1)]

−f(x) +hf(x)−h²

2!f(x) +· · ·+ h^2m+3

(2m+ 3)![f^(2m+3)(x) +o(1)])

=τ₀+τ₁h²+· · ·+τ_mh^2m+h^2m+2α_m+1(h)

where τ₀ = f(x), τ_k = f^(2k+1)(x)/(2k+ 1)!, k = 1, 2, . . ., m+ 1, and α_m+1(h) =τ_m+1+o(1).

Using the one-sided difference quotient T(h) := f(x+h)−f(x)

h , h= 0,

leads to the asymptotic expansion

T(h) =τ₀+τ₁h+τ₂h²+· · ·+τ_mh^m+h^m+1(τ_m+1+o(1)) with

τ_k= f^(k+1)(x)

(k+ 1)! , k= 0,1,2, . . . , m+ 1.

We will see later that the central difference quotient is a better ap- proximation to base an extrapolation method on, as far as convergence is concerned, because its asymptotic expansion contains only even powers of the step lengthh. Other important examples of discretization methods

which lead to such asymptotic expansions are those for the solution of or- dinary differential equations [see Sections 7.2.3 and 7.2.12]. A systematic treatment of extrapolation methods is found in Brezinski and Zaglia (1991).

In order to derive an extrapolation method for a given discretization method with (3.5.1), we select a sequence of step lengths

F ={h₀, h₁, h₂, . . .}, h₀> h₁> h₂>· · ·>0,

and calculate the corresponding approximate solutionsT(h_i),i= 0, 1, 2, . . .. Fori≥k, we introduce the “polynomials”

T_ik(h) =b₀+b₁h^γ1+· · ·+b_kh^γk, for which

T_ik(h_j) =T(h_j), j=i−k, i−k+ 1, . . . , i, and we consider the values

T_ik:=T_ik(0)

as approximations to the desired value τ₀. Rational functions T_ik(h) are frequently preferred over polynomials. Also the exponentsγ_k need not be integer [see Bulirsch and Stoer (1964)].

For the following discussion of the discretization errors, we will assume thatT_ik(h) are polynomials with exponents of the formγ_k=γ k. Romberg integration [see Section 3.4] is a special case withγ= 2.

First, we consider the casei=k. In the sequel, we will use the abbre- viations

z:=h^γ, z_j:=h^γ_j, j= 0,1, . . . , m.

Applying Lagrange’s interpolation formula (2.1.1.4) to the polynomial T_kk(h) =:P_k(z) =b₀+b₁z+b₂z²+· · ·+b_kz^k

yields forz= 0

T_kk=P_k(0) = k j=0

c_jP_k(z_j) = k j=0

c_jT(h_j) with

c_j :=

!k

σ=jσ=0

z_σ z_σ−z_j. Then

(3.5.2)

k j=0

c_jz^τ_j =





1 ifτ = 0,

0 ifτ = 1, 2,. . ., k, (−1)^kz₀z₁· · ·z_k ifτ =k+ 1.

Proof.By Lagrange’s interpolation formula (2.1.1.4) and the uniqueness of polynomial interpolation,

p(0) = k j=0

c_jp(z_j)

holds for all polynomialsp(z) of degree≤k. One obtains (3.5.2) by choosing p(z) as the polynomials

z^τ, τ= 0,1, . . . , k, and

z^k+1−(z−z₀)(z−z₁)· · ·(z−z_k).

(3.5.2) can be sharpened for sequencesh_j for which there exists a con- stantb >0 such that

(3.5.3) h_j+1

h_j ≤b <1 for allj.

In this case, there exists a constant C_k which depends only on b and for which

(3.5.4)

k j=0

|c_j|z^k+1_j ≤C_kz₀z₁· · ·z_k.

We prove (3.5.4) only for the special case of geometric sequences{h_j}with h_j=h₀b^j, 0< b <1, j = 0,1, . . . .

For the general case see Bulirsch and Stoer (1964). With the abbreviation θ:=b^γ we have

z^τ_j = (h₀b^j)^γτ =z^τ₀θ^jτ. In view of (3.5.2), the polynomial

P_k(z) :=

k j=0

c_jz^j

satisfies P_k(θ^τ) =

k j=0

c_jθ^jτ =z₀^−τ k j=0

c_jz_j^τ=

1 forτ= 0,

0 forτ= 1, 2,. . .,k, so that P_k(z) has the k different rootsθ^τ, τ = 1, . . ., k. Since P_k(1) = 1 the polynomialP_k must have the form

P_k(z) =

!k l=1

z−θ^l 1−θ^l. The coefficients ofP_k alternate in sign, so that

k j=0

|c_j|z_j^k+1=z₀^k+1 k j=0

|c_j|(θ^(k+1))^j=z^k+1₀ |P_k(−θ^k+1)|

=z₀^k+1

!k l=1

θ^k+1+θ^l 1−θ^l

=z₀^k+1θ^1+2+···+k

!k l=1

1 +θ^l 1−θ^l

=C_k(θ)z₀z₁· · ·z_k with

(3.5.5) C_k=C_k(θ) :=

!k l=1

1 +θ^l 1−θ^l.

This proves (3.5.4) for the special case of geometrically decreasing step lengthsh_j.

We are now able to make use of the asymptotic expansion (3.5.1) which gives fork≤m

T_kk= k j=0

c_jT(h_j)

= k j=0

c_j[τ₀+τ₁z_j+τ₂z_j²+· · ·+τ_kz^k_j +α_k+1(h_j)z_j^k+1], where of course, fork < m

α_k+1(h_j)z_j^k+1=τ_k+1h^γ_j^k+1+· · ·+τ_mh^γ_j^m+α_m+1(h_j)h^γ_j^m+1. By (3.5.2)

(3.5.6) T_kk=τ₀+

k j=0

c_jα_k+1(h_j)z^k+1_j .

Using (3.5.4) and|α_m+1(h_j)| ≤M_m+1 for allj≥0 [see (3.4.2)] we find for k=m

(3.5.7) |T_mm−τ₀| ≤M_m+1C_mz₀z₁· · ·z_m,

and fork < m,

(3.5.8) T_kk−τ₀= (−1)^kz₀z₁· · ·z_k(τ_k+1+ 0(h^γ₀)), because ofα_k+1(h_j) =τ_k+1+O(h^γ_j), (3.5.2), and (3.5.6).

The corresponding estimates for the error of T_ik for the general case i≥kare obtained by just replacing in (3.5.7) and (3.5.8)z₀,z₁,. . .,z_k by z_i−k,z_i−k+1, . . .,z_i, andh₀ byh_i−k respectively, because T_ikis obtained by extrapolating fromT(h_j),j=i−k,i−k+ 1,. . .,i. Sincez_j=h^γ_j this leads to

(3.5.9) |T_im−τ₀| ≤M_m+1C_mz_i−mz_i−m+1· · ·z_i fori≥m, and fork < m,i≥k to

(3.5.10) T_ik−τ₀= (−1)^kh^γ_i−kh^γ_i−k+1· · ·h^γ_i(τ_k+1+O(h^γ_i−k)).

Consequently, for fixedk andi→ ∞, T_ik−τ₀=O(h^(k+1)γ_i−k ).

In other words, the elements T_ik of the (k+ 1)st column of the tableau (3.4.4) converge to τ₀ like a method of order (k+ 1)γ. Note that the in- crease of the order of convergence from column to column which can be achieved by extrapolation methods is equal to γ : γ= 2 is twice as good as γ = 1. This explains the preference for discretization methods whose corresponding asymptotic expansions contain only even powers ofh, e.g., the asymptotic expansion of the trapezoidal sum (3.4.1) or the central dif- ference quotient discussed in this section.

The formula (3.5.10) shows furthermore that the sign of the error re- mains constant for fixedk < mand sufficiently largeiprovidedτ_k+1 = 0.

Then

(3.5.11) 0<T_i+1,k−τ₀

T_ik−τ₀ ≈ h^γ_i+1

h^γ_i−k ≤b^γ(k+1)

Now in many casesb^γ(k+1)< ¹₂. Then the error of the quantity U_ik:= 2T_i+1,k−T_ik

satisfies

U_ik−τ₀= 2(T_i+1,k−τ₀)−(T_ik−τ₀).

Fors:= sign (T_i+1,k−τ₀) = sign (T_i,k−τ₀) we have

s(U_ik−τ₀) = 2|T_i,k+1−τ₀| − |T_ik−τ₀| ≈ −|T_ik−τ₀|<0.

Thus U_ik converges monotonically to τ₀, for i → ∞ at roughly the same rate asT_ik but from the opposite direction, so that eventuallyT_ik andU_ik

will include the limitτ₀between them. This observation yields a convenient stopping criterion.

Example.The exact value of the integral

# π/2 0

5(e^π−2)⁻¹e^2xcosx dx

is 1. Using the polynomial extrapolation method of Romberg, and carrying 12 digits, we obtain for Tik, Uik, 0 ≤ i ≤ 6, 0 ≤ k ≤ 3 the values given in the following table.

i Ti0 Ti1 Ti2 Ti3

0 0.185 755 068 924

1 0.724 727 335 089 0.904 384 757 145

2 0.925 565 035 158 0.992 510 935 182 0.998 386 013 717

3 0.981 021 630 069 0.999 507 161 706 0.999 973 576 808 0.999 998 776 222 4 0.995 232 017 388 0.999 968 813 161 0.999 999 589 925 1.000 000 002 83 5 0.998 806 537 974 0.999 998 044 836 0.999 999 993 614 1.000 000 000 02 6 0.999 701 542 775 0.999 999 877 709 0.999 999 999 901 1.000 000 000 00

i Ui0 Ui1 Ui2 Ui3

0 1.263 699 601 26

1 1.126 402 735 23 1.080 637 113 22

2 1.036 478 224 98 1.006 503 388 23 1.001 561 139 90

3 1.009 442 404 71 1.000 430 464 62 1.000 025 603 04 1.000 001 229 44 4 1.002 381 058 56 1.000 027 276 51 1.000 000 397 30 0.999 999 997 211 5 1.000 596 547 58 1.000 001 710 58 1.000 000 006 19 0.999 999 999 978 6 1.000 149 217 14 1.000 000 107 00 1.000 000 000 09 1.000 000 000 00