About Extrapolation Methods

Topics in Integration

3.5 About Extrapolation Methods

element to its left-hand neighbors. The meaning of 1ik' however, is now as follows: The functions 1;dh) are rational functions in h^{2 ,}

f.

(h):=Po

+

P1 h2

+ '" +

PI'h21'

lk Qo+Q1h2+"'+qvh2v' J1

+

v

=

k, J1

=

v or J1

=

v-I, with the interpolation property

j = i - k, i - k

+

1, ... , i.

We then define

and initiate the recursion (3.4.6) by putting Iio:= T(hJ for i

=

0, 1, ... , m, and Ii. -1 := 0 for i

=

0, 1, ... , m - 1. The observed superiority of rational extrapolation methods reflects the more flexible approximation properties of rational functions (see Section 2.2.4).

In Section 3.5, we will illustrate how error estimates for extrapolation methods can be obtained from asymptotic expansions like (3.4.1). Under mild restrictions on the sequence of step lengths, it will follow that, for polynomial extrapolation methods based on even asymptotic expansions, the errors of Iio behave like h?, those of Ii1 like h?-1 h?, and, in general, those of Iik like h?-k h?_k+ ^{1 ...} h? as i -+ 00. For fixed k, consequently, the sequence 1ik' i = k, k

+

1, ... , approximates the integral like a method of order 2k

+

2. For the sequences (3.4.5) a stronger result has ~een found:

(3.4.7) 1ik - (f(X) dx = (b - a)h?-kh?_k+ 1 ... h?

(2:~ ~)!

j<2k+2)(O for a suitable ~ E (a, b) and

f

^E C2k+ 2[a, b] [see Bauer, Rutishauser, and Stiefel (1963), Bulirsch (1964)].

one instance. In all these cases, the asymptotic expansion of the result T(h) is of the form

(3.5.1) T(h) =!o

+

!lhYl

+

!zhY2

+ ... +

!mhYm

+

hYm⁺¹rim+l(h),

°

^{< Yl < Yz} < ... < Ym+ ^1,

where the exponents Yi need not be integers. The coefficients !i are indepen- dent of h, the function ^{rim +} l(h) is bounded for h ^-+ 0, and!o = limh^-+O T(h) is the exact solution of the problem at hand.

Consider, for example, numerical differentiation. For h

+-

0, the central difference quotient

T(h) = f(x

+

h) - f(x - h) 2h

is an approximation to f'(x). For functions fE CZm +3[x - a, x

+

a] and

Ihl :::; lal,

Taylor's theorem gives

1 1 h z hZm+ 3

T(h)

=

2h If (x)

+

hf'(x)

+

2! f"(x)

+ ... +

(2m

+

3)! (f(Zm+3)(x)

+

0(1)]

hZ hZm +3 \

- f(x)

+

/if'(x) - 2! f"(x)

+ ... +

(2m

+

3)! (f(Zm+3)(x)

+

0(1)]1

=!o

+

!lhZ

+ ... +

!mhZm

+

hZm +zrim+l(h),

where !o=f'(x), !k=PZk+l)(X)/(2k+1)! for k=1,2, ... ,m+1, and rim+l(h)

=

^!m+l

+

0(1).

Using the one-sided difference quotient T(h):= f(x

+

h) - f(x)

h leads to the asymptotic expansion

with

T(h) =!o

+

!lh

+

!zhz

+ ... +

!mhm

+

hm+l(!m+l

+

0(1))

pk+

1)(X)

!k=(k+1)!' ^k

=

0, 1, 2, ... , m

+

^1.

We will see later that the central difference quotient is a better approxi- mation to base an extrapolation method on, as far as convergence is con- cerned, because its asymptotic expansion contains only even powers of the step length h. Other important examples of discretization methods which lead to such asymptotic expansions are those for the solution of ordinary differential equations (see Sections 7.2.3 and 7.2.12).

In order to derive an extrapolation method for a given discretization method, we select a sequence of step lengths

ho > hI > hz > ... > 0,

and calculate the corresponding approximate solutions T(h;), i = 0, 1,2, ....

For i ~ k, we introduce the" polynomials"

Tik(h)

=

bo

+

^b1hY1

+ ... +

^{bkhYk ,}

for which

Tik(hj )

=

T(hJ, and we consider the values

j = ^{i -} k, i - k

+

1, ... , i,

T;k ^:= Tik(O)

as approximations to the desired value to. Rational functions T;k(h) are frequently preferred over polynomials. Also the exponents Yk need not be integer [see Bulirsch and Stoer (1964)].

For the following discussion of the discretization errors, we will assume that the 1;k(h) are polynomials with exponents of the form Yk = kyo Romberg integration (see Section 3.4) is a special case with y = 2. We will use the abbreviations

Zj := h}, j

=

0, 1, ... , m.

Applying Lagrange's interpolation formula (2.1.1.3) to the polynomial

- 2 J<

Tik(h) = Pik(Z) = bo

+

biz

+

b2z

+ ... +

bkz yields for Z =

°

with

Then

(3.5.2)

i i

T;k = Pik(O) =

L

cL~Pik(zj) =

L

c~JT(hj)

j=i-k j=i-k

i

C^(i):= kj

n

^Z"

"",j Z" - Zj n=i-k

if t = 0,

if ^t = 1, 2, ... , k, if ^t

=

k

+

1.

PROOF. The Lagrange coefficients 4~ depend only on the support abscissas Zj

and not on the functions to be interpolated. Selecting the polynomials ZI,

1= 0, 1, ... , k, Lagrange's interpolation formula gives therefore

i i Z "'

1- " 1

n --"

Z - 1... Zj - - ,

j=i-k "",j Zj - Z"

1= 0, ... , k.

,,=i-k

For Z

=

0, all but the last one of the relations (3.5.2) follow.

To prove the last of the relations (3.5.2), we note that (3.5.3 )

Indeed, since the coefficients of ?!'+ 1 are the same on both sides, the differ- ence polynomial has degree at most k. Since it vanishes at the k

+

1 points

Za, (j = i - k, ... , i, it vanishes identically, and (3.5.3) holds. Letting Z = 0 in

(3.5.3) completes the proof of (3.5.2). D

(3.5.2) can be sharpened for sequences hj for which there exists a constant b such that

\+.1 ,.;;

^{b <} 1 for allj.

J

In this case, there exists a constant Ck which depends only on b and for which

i

(3.5.4) ^~ _L...,

I (i)1

_{Ckj Zj} ^{k+1 ~} "" ^C k Zi-k Zi-k+1 •.• Zi' j=i-k

We prove (3.5.4) only for the special case of geometric sequences {hj} with

o

< b < 1, j

=

0, 1, ....

For the general case see Bulirsch and Stoer (1964). It suffices to prove (3.5.4) for i = k. With the abbreviation () := bY we have

In view of (3.5.2), the polynomial

k Pk(z):=

L

^4~)zj

j=O

satisfies

(() ) ~ (k)()jt _ - t ~ ^{(k) t _} J 1 for r

=

⁰

P k ^t

=

^{L..., Ckj - Zo L..., CkjZj -}

I

j=O j=O 0 for r = 1,2, ... , k,

so that Pk(z) has the n different roots 0', r

=

^{1, ... ,} k. Since P k(l)

=

^{1, the}

polynomial Pk must have the form

The coefficients of Pk alternate in sign, so that

with (3.5.5)

This proves (3.5.4) for the special case of geometrically increasing step lengths hj •

We are now able to make use of the asymptotic expansion (3.5.1) which gives for k < m

i

1ik

= L

^cr}T(hJ

j=i-k

i

= L

cm,o + 'lZj + '2 Z

J

+ ... + 'k Z' + Z,+l('k+l + O(hj))],

j=i-k

and for k

=

m

i

1im

= L

c~~[,o+

'1

^Zj + '2 Z

J

+ ... + 'mz}' + zj+ 11Xm+ 1 (hj )].

j=i-m

By (3.5.2) and (3.5.4),

(3.5.6) 1ik

=

^'0

+

(-1 fZi-kZi-k+ 1 ... Zi('k+ 1

+

O(hi- k)) for k < m, and

if IlXm+1(hj

)1 ::::;

Mm+ 1 for j ~ 0 [see (3.4.2)]. Consequently, for fixed k and i -+ 00,

7"' _ _~ik

'0 -

_ 0(-1<+ ^Zi-k 1) _ ^- O(h(k+ ^{j-k .} l)Y)

In other words, the elements 1ik of the (k

+

1)st column of the tableau (3.4.4) converge to

'0

like a method of order (k

+

1 )'Y. Note that the increase of the order of convergence from column to column which can be achieved by extrapolation methods is equal to 'Y: 'Y = 2 is twice as good as 'Y = 1. This explains the preference for discretization methods whose corresponding asymptotic expansions contain only even powers of h, e.g., the asymptotic

0 1 2 3 4 5 6

0 1 2 3 4 5 6

expansion of the trapezoidal sum (3.4.1) or the central difference quotient discussed in this section.

The formula (3.5.6) shows .furthermore that the sign of the error remains constant for fixed k < m and sufficiently large i provided 7:k + ¹ =f. O. Advantage can be taken of this fact in the many cases in which

(3 5 7) 0 ~ 17i+l,k - 7:01 ~ hI+l ~ by(k+l)

<!

. . ~ l7ik - 7:01 hI-k ~ 2'

If we put then

Uik - 7:0 = 2(7i+ I, k - 7:0) - (7ik - 7:0)' For s := stgn(7i+ I, k - 7:0)

=

^{sign(7i, k -} 7:0), we have

S(Uik - 7:0) = 217i, k+ 1 - 7:01 - l7ik - 7:0 1 ~ - l7ik - 7:01 < O.

Thus Uik converges monotonically to 7:0 for i ~ 00 at roughly the same rate as tik but from the opposite direction, so that eventually tik and Uik will include the limit 7:0 between them. This observation yields a convenient stopping criterion.

EXAMPLE. The exact value of the integral

r

nl2 5(en - 2t le^2x cos x dx '0

is 1. Using the polynomial extrapolation method of Romberg, and carrying 12 digits, we obtain for 7;k, Uik , 0 ~ i ~ 6, 0 ~ k ~ 3, the values given in the following table.

7;0 7;1 7;2 7;33

0.185 755 068 924

0.724 727 335 089 0.904 384 757 145

0.925 565 035 158 0.992 510 935 182 0.998 386 013 717

0.981 021 630069 0.999 507 161 706 0.999 973 576 808 0.999 998 776 222 0.995 232 017 388 0.999 968 813 161 0.999 999 589 925 1.000 000 002 83 0.998 806 537 974 0.999 998 044 836 0.999999993 614 1.000 000 000 02 0.999 701 542 775 0.999 999 877 709 0.999 999 999 901 1.000 000 000 00

Uw Ui! Uu Ui3

1.263 699 60 1 26

1.126 402 735 23 1.080 637 113 22

1.036 478 224 98 1.006 503 388 23 1.001 561 139 90

1.009 442 404 71 1.000 430 464 62 1.000 025 603 04 1.000 001 229 44 1.002 381 058 56 1.000 027 276 51 1.000 000 397 30 0.999999997211 1.000 596 547 58 1.000 001 710 58 1.000000 006 19 0.999 999 999 978 1.000 149217 14 1.000 000 107 00 1.000000 00009 1.000 000 000 00