Chapter 19 Numerics in General in General

(1)

Chapter 19 Chapter 19 Chapter 19 Chapter 19 Numerics

Numerics in General in General

Bo-Yeong Kang, 2009

Kyungpook National University

1

(2)

What is What is

numerical analysis ? y

(3)

The study of algorithms to sol e the problems that

solve the problems that

there is no solution formula

3

(4)

19.1 Introduction

The steps from problems p p to answer

Modeling Modeling

Choosing a numeric methods programming

programming

Doing the computation

Interpreting the results

(5)

19.1 Introduction

Real number

Fl ti i t f f Floating point form of

numbers numbers

0.6247*10

³

, 0.735*10

^-13

, -0.2000*

^10-1

Number of significant digits is kept fixed g g p Decimal point “floating”

5

(6)

19.1 Introduction

Floating point form of Floating point form of numbers

numbers

a = m*10

ⁿ

, 0.1≤|m| < 1

On computers, m & n is limited p ,

ā = m*10

ⁿ

, m=0.d

₁

d

₂

…d

_k

, d

₁

>0

mantissa

exponent mantissa

(7)

19.1 Introduction

Floating point form of n mbe s

numbers

ā = m*10

ⁿ

, m=0.d

₁

d

₂

…d

_k

, d

₁

>0

mantissa

exponent exponent

sign mantissati

IBM 3000 & 4000 series Single precision floating point number 1 sign, 7 exponent, 24 mantissa, 32 bits

7

(8)

19.1 Introduction

Floating point form of oat g po t o o numbers

The number outside that range occurs

Underflow when the number is smaller (zero) Overflow when it is larger (halt)

exponent

sign mantissa

(9)

19.1 Introduction

Roundoff(

^끝처리

)

mantissa

Roundoff(

^끝처리

)

0.d

₁

d

₂

…d

_k

d

_k+1

d

_{k +2}

d

_k+3

… Roundoff rule

Discard (k+1)th and all subsequent decimals

If the number discarded is less than half a unit in kth place, leave kth decimal unchanged(rounding down)

If it is greater than half a unit, round add one to the kth decimal If it is greater than half a unit, round add one to the kth decimal (rounding up)

If it is exactly half a unit, round off to the nearest even decimal

Rounding up/down happens about equallyg p/ pp q y ex)3.45-3.4, 3.55-3.6

9

(10)

19.1 Introduction

Error in Rounding Error in Rounding

ā= fl(a),

floating point approximation of a by rounding

|m-m| ≤ ½ * 10

^-k (ex. |0.45-0.4| ≤ ½ * 10^-1)

|m| ≥ 0.1 , 1/|m| ≤ 10

|(a- ā)/a| ≈|(m-m)/m| ≤ ½ * 10

^1-k

Error bound in rounding

(11)

19.1 Introduction

Errors of Numeric Results

Finding the results – Approximation

Rounding errors

Experimental errors from measurements

T ncating e o s f om p emat el b eaking off Truncating errors from prematurely breaking off

Formulas for Errors Formulas for Errors

є = a – ã

a = ã + є , True value = Approximation + error a ã + є , True value Approximation + error ã = 10.5, a=10.2, є = -0.3

11

(12)

19.1 Introduction

Errors of Numeric Results

є = |a – ã|(absolute error) or ã – a | |( )

є

_r

= є/a = (a-ã)/a = Error/True value (relative error ) This look useless because a is unknown

/ã є

_r

= є/ã

This still looks problematic because є is unknown

One can obtain in practice is an error bound for ã, 

|є| ≤  , hence |a-ã| ≤ 

| |  | | 

|є

_r

|≤ 

_r

, hence |(a-ã)/a| ≤ 

_r

(13)

19.1 Introduction

E P ti

Error Propagation

How errors at the beginning and in later steps

propagate into the computation and affect accuracy (a) Subtraction

x = x+ є y=ỹ+є |є | ≤  |є | ≤ 

x = x+ є

₁

, y=ỹ+є

₂

, |є

₁

| ≤ 

_{1 ,}

|є

₂

| ≤ 

₂

є = |x-y – (x-ỹ)|

= |x-x – (y-ỹ)|

True value subtraction

Approximation subtraction

= |x x (y ỹ)|

= | є

₁

– є

₂

| ≤ |є

₁

|+|є

₂

| ≤ 

₁

+ 

₂

13

(14)

19.1 Introduction

Error Propagation Error Propagation

(b) Relative error of multiplication

x = x+ є

₁

, y=ỹ+ є

₂

, |є

_r1

| ≤ 

_{r1 ,}

|є

_r2

| ≤ 

_r2

|( ỹ)/ | є

_r

= |(xy – xỹ)/xy|

= |(xy – (x- є

₁

)(y-є

₂

))/xy|

|( )/ |

= |(є

₁

y+є

₂

y-є

₁

є

₂

)/xy|

≈ |(є

₁

y+є

₂

x)/xy|

| / | | / | | | | |

≤ | є

₁

/x |+ |є

₂

/y| = |є

_r1

|+|є

_r2

| ≤ 

_r1

+ 

_r2

(15)

19.1 Introduction

Error Propagation Error Propagation

Th 1 E P i

Th.1 Error Propagation Th.1 Error Propagation

(a)In addition and subtraction, an error bound

f th lt i i b th f th

for the results is given by the sum of the error bounds for the terms

(b)I lti li ti d di i i b d

(b)In multiplication and division, an error bound for the relative error of the results is given by

h f h b d f h l i f

the sum of the bounds for the relative error of the given numbers

15

(16)

19 2 Solution of 19.2 Solution of

Equation by Iteration q o y o

(17)

19.2 Solution of Equation by Iteration

F(x) = 0

17

(18)

F(x) = 0 F(x) 0

Zeros of Bessel function ?

(19)

19

(20)

Algebraic equation Algebraic equation

Transcendental equation

(21)

F(x) = 0

No formula for exact solution Approximation

Iteration method

21

(22)

Iteration method Iteration method

Initial guess x ₀ step by step x ₁ x ₂ step by step x ₁ x _{2 …} Fi d i t it ti

Fixed point iteration Newton’s method

Secant method

(23)

Fixed-point iteration p

Transform f(x) 0 to x g(x) Transform f(x)=0 to x=g(x)

Choose x ₀ x ₁ =g(x ₀ ) x ₂ ₂ =g(x g( ₁ ₁ ) )

… X _n+1 _n+1 =g(x g _n _n )

23

(24)

Ex1. Fixed-point iteration p

Transform to x=g(x)

Converge

Diverge

(25)

Ex1. Fixed-point iteration p

Divide by x

25

(26)

Indexing Indexing

Extract the index terms Extract the index terms Weights

Data structure

(27)

Header file Ex. stdlib.h

27

(28)

Data type Data type Int ^integer Int, ^integer

Long, long integer

Float, floating-point

Double, double precision floating point

(29)

Indexing Indexing

Extract the index terms Extract the index terms Weights

Data structure

29

(30)

Indexing Indexing

Extract the index terms Extract the index terms Weights

Data structure

(31)

x ₀ =1 0 x ₀ =1.0

31

(32)

3 0 x ₀ =3.0

(+,-)1.#INF, infinite number NaN not a number =

NaN, not a number =

(+,-)1.#IND (indeterminate)

(33)

Ex1. Fixed-point iteration p

lower part, the slope of g₁(x) is less than the slope of y=x

of y x

upper part, the slope of g₁(x) is steeper than the slope of y=x

|g₁’(x)| < 1 is sufficient for convergence

1.0 3.0 ₃₃

(34)

C f Fi d P i t

19.1 Introduction

Convergence of Fixed-Point Iteration

Iteration

Th 1 C Th 1 C

Th.1 Convergence Th.1 Convergence

Let x=s be a solution of x=g(x) and suppose

that g has a continuous derivative in some

interval J containing s. Then if |g’(x)| ≤ K < 1 in

J, the iteration process defined by (3) converges

for any x

₀

in J, and the limit of the sequence {x

_n

}

is s.

(35)

C f Fi d P i t

Convergence of Fixed-Point Iteration

Iteration

|g’(x)|<1

S S

J ³⁵

(36)

Proof

|g’(x)|<1

S S

(37)

Ex1. Fixed-point iteration p

Transform to x=g(x)

| ’( )| |2 /3|

|g₁’(x)| = |2x/3|<1

-3/2 < x < 3/2 의 어떤 x₀에서라도 수렴

37

(38)

Ex1. Fixed-point iteration p

Divide by x

|g₁’(x)| = |1/x²|<1

1 1 의 어떤 에서라도 수렴

-∞< x <-1, 1<x<∞ 의 어떤 x₀에서라도 수렴

(39)

Iteration method Iteration method

Initial guess x ₀ step by step x ₁ x ₂ step by step x ₁ x _{2 …} Fi d i t it ti

Fixed point iteration Newton’s method

Secant method

39

(40)

Newton’s Method

F(x)=0

Commonly used

Simplicity and great speed

(41)

Newton’s Method

Idea Tangent

41

(42)

Newton’s Method

(43)

Fail, Try another x₀

Success

Fail, Try another x, y ₀₀

43

(44)

Ex3. Square Root q

(45)

45

(46)

(47)

Ex4. Transcendental Equation q

47

(48)

(49)

49

(50)

Ex5. Algebraic Equation g q

(51)

51

(52)

(53)

Order of an Iteration method Speed of Convergence

53

(54)

Subtract g(s)=s

x_n+1 – s = -g’(s) є_n+ 1/2 g’’(s) є_n²+ … є = g’(s) є + 1/2 g’’(s) є ²+

-є_n+1 = -g (s) є_n+ 1/2 g (s) є_n²+ … є_n+1 = g’(s) є_n- 1/2 g’’(s) є_n²+ …

(a) є_n+1 ≈ g’(s) є_nin the case of first order

(b) є ₊₁ ≈ -1/2 g’’(s) є ² in the case of second order (b) є_n+1 ≈ 1/2 g (s) є_n in the case of second order Ex. є_n=10^-k, є_n+1 = c*(10^-k)²=c*10^-2k

(55)

C f N t ’

Convergence of Newton’s Method

Method

є_n+1

= g’(s)

є_n

- 1/2 g’’(s)

є_n²

+ …

є_n+1

g (s)

є_n

1/2 g (s)

є_n

+ …

 є_n+1 ≈ -1/2 g’’(s) є_n² , second order if f’ and f’’ are not zero at a solution s

55

(56)

Taylor Series

미분가능한 어떤 함수를 다항식의 형태 미분가능한 어떤 함수를 다항식의 형태 로 근사화하는 방법

어떤 한 점에서 그것의 미분값으로 계 산되는 항들의 무한 합으로 표현

(57)

Taylor Series Taylor Series

f(x) = c

₀

+c

₁

(x-a)+c

₂

(x-a)

²

+c

₃

(x-a)

³

+…

(

단

, |x-a|<R

때 수렴

)

( | | )

x=a, f(a) = c₀

f’(x) = c₁+2c₂(x-a)+3c₃(x-a)²+…

x=a, f’(a) = c₁

f’’(x) = 2c₂+3*2c₃(x-a)+…

x=a, f’’(a) = 2c₂, f’’(a)/2 = c₂

Cⁿ =

f’’’(x) = 3*2c₃+…

x=a, f’’’(a) = c₃, f’’’(a)/(3*2) = c₃

…

57

(58)

Taylor Series Taylor Series

The Taylor series of a real or complex function ƒ(x) y p ƒ( ) that is infinitely differentiable in a neighborhood of a real or complex number a, is the power series

where n! denotes the factorial of n and ƒ ⁽ⁿ⁾(a) denotes the nth

derivative of ƒ evaluated at the point a; the zeroth derivative of ƒ is derivative of ƒ evaluated at the point a; the zeroth derivative of ƒ is defined to be ƒ itself and (x − a)⁰ and 0! are both defined to be 1.

In the particular case where a 0 the series is also

In the particular case where a = 0, the series is also

called a Maclaurin series.

(59)

처음 몇 항까지를 선택함으로써 x = a 주변에서의 f(x)의 근사식으로 사용

The exponential function y = e^x (continuous red line) and the corresponding Taylor polynomial of degree four (dashed green line) around the origin

line) around the origin.

59

(60)

The exponential function (in blue) The exponential function (in blue), and the sum of the first n+1 terms of its Taylor series at 0 (in red).

(61)

Iteration method Iteration method

Initial guess x ₀ step by step x ₁ x ₂ step by step x ₁ x _{2 …} Fi d i t it ti

Fixed point iteration Newton’s method

Secant method

61

(62)

Secant Method

f’(x)

A far more difficult expression

Computationally expensive

(63)

Secant Method

초기치 2개 필요 초기치 2개 필요

63

(64)

Ex8. Secant Method

(65)

Convergence of Secant Method Evaluation of derivatives

Avoided

Convergence

Almost quadratic like Newton Almost quadratic like Newton

|є | ≈ const| є |

^{1 62}

|є

_n+1

| ≈ const| є

_n

|

^1.62

65

(66)

Iteration method Iteration method

Initial guess x ₀ step by step x ₁ x ₂ step by step x ₁ x _{2 …} Fi d i t it ti

Fixed point iteration Newton’s method

Secant method

(67)

19.3 Interpolation

67

(68)

19.3 Interpolation

approximate values approximate values

of a function f(x) for an x

between different x-values x ₀ ,x ₁ ,…,x _n

(69)

69

(70)

Lagrange Interpolation

(71)

y-f ₁ =(f ₁ -f ₀ /x ₁ -x ₀ )(x-x ₁ )

71

(72)

Lagrange Interpolation g g p Idea

Idea

(x ₀ ,f ₀ ), (x ₁ ,f ₁ )

x=x ₀ P ₁ (x)=f ₀

x=x ₀ , P ₁ (x)=f ₀

x=x ₁ , P ₁ (x)=f ₁

(73)

(x ₀ ,f ₀ ), (x ₁ , f ₁ ) x=x ₀ , P ₁ (x)=f ₀ x=x ₁ P ₁ (x)=f ₁ x=x ₁ , P ₁ (x)=f ₁

P (x ) =f L +f L =f (L =1 L =0) P ₁ (x ₀ ) =f ₀ L ₀ +f ₁ L ₁ =f ₀ (L ₀ =1, L ₁ =0) P ₁ (x ₁ ) =f ₀ L ₀ +f ₁ L ₁ = f ₁ (L ₀ =0, L ₁ =1) L ₀ = (x-x ₁ )/(x ₀ -x ₁ )

L ₀ = (x x ₁ )/(x ₀ x ₁ ) L ₁ = (x-x ₀ )/(x ₁ -x ₀ )

73

(74)

y-f

₁

=(f

₁

-f

₀

/x

₁

-x

₀

)(x-x

₁

) y=(f

₁

-f

₀

/x

₁

-x

₀

)(x-x

₁

)+f

₁

={(f

₁

-f

₀

)(x-x

₁

)+f

₁

(x

₁

-x

₀

)}/(x

₁

-x

₀

)

={(-f

₀

)(x-x

₁

)+f

₁

(x-x

₁

)+f

₁

(x

₁

-x

₀

)}/(x

₁

-x

₀

)

={(-f

₀

)(x-x

₁

)+f

₁

(x-x

₀

)}/(x

₁

-x

₀

)

=(-f

₀

)(x-x

₁

) /(x

₁

-x

₀

)+f

₁

(x-x

₀

) /(x

₁

-x

₀

)

= f

₀

(x-x

₁

) /(x

₀

-x

₁

)+f

₁

(x-x

₀

) /(x

₁

-x

₀

)

= f

₀

L

₀

+f

₁

L

₁

(75)

Ex1. Linear Lagrange Interpolation

P

₁

(x) =f

₀

L

₀

+f

₁

L

_{1 ,}

L = (x x )/(x x ) L = (x x )/(x x ) L

₀

= (x-x

₁

)/(x

₀

-x

₁

), L

₁

= (x-x

₀

)/(x

₁

-x

₀

)

75

(76)

L

₀

= (x-x

₁

)(x-x

₂

)/(x

₀

-x

₁

)(x

₀

-x

₂

)

L

₀

(x x

₁

)(x x

₂

)/(x

₀

x

₁

)(x

₀

x

₂

)

L

₁

= (x-x

₀

)(x-x

₂

)/(x

₁

-x

₀

)(x

₀

-x

₂

)

L

₂

= (x-x

₀

)(x-x

₁

)/(x

₂

-x

₀

)(x

₂

-x

₁

)

(77)

Quadratic Interpolation

Q p

three points three points

(x ₀ ,f ₀ ), (x ₁ ,f ₁ ), (x ₂ ,f ₂ ) x=x ₀ , P ₂ (x)=f ₀

x=x ₁ P ₂ (x)=f ₁ x=x ₁ , P ₂ (x)=f ₁ x=x ₂ , P ₂ (x)=f ₂

77

(78)

P ₂ (x) =f ₀ L ₀ +f ₁ L ₁ +f ₂ L ₂ (x ₀ ,f ₀ ), (x ₁ ,f ₁ ), (x ₂ ,f ₂ )

x=x (L =1 L =0 L =0) x=x ₀ , (L ₀ =1, L ₁ =0, L ₂ =0) x=x ₁ , (L ₀ =0, L ₁ =1, L ₂ =0) x=x ₂ , (L ₀ =0, L ₁ =0, L ₂ =1)

L ₀ = (x-x ₁ )(x-x ₂ )/(x ₀ -x ₁ )(x ₀ -x ₂ )

L ( )( )/( )( )

L ₁ = (x-x ₀ )(x-x ₂ )/(x ₁ -x ₀ )(x ₀ -x ₂ )

L ₂ ₂ = (x-x ( ₀ ₀ )(x-x )( ₁ ₁ )/(x )/( ₂ ₂ -x ₀ ₀ )(x )( ₂ ₂ -x ₁ ₁ ) )

(79)

Ex2. Quadratic Lagrange Interpolation

79

(80)

L ( )( )/( )( )

L

₀

= (x-x

₁

)(x-x

₂

)/(x

₀

-x

₁

)(x

₀

-x

₂

)

L

₁

= (x-x

₀

)(x-x

₂

)/(x

₁

-x

₀

)(x

₀

-x

₂

)

L

₂

= (x-x

₀

)(x-x

₁

)/(x

₂

-x

₀

)(x

₂

-x

₁

)

(81)

L = (x x )(x x )/(x x )(x x ) L

₀

= (x-x

₁

)(x-x

₂

)/(x

₀

-x

₁

)(x

₀

-x

₂

) L

₁

= (x-x

₀

)(x-x

₂

)/(x

₁

-x

₀

)(x

₀

-x

₂

) Code

작성

Code

작성

81

(82)

General Lagrange

I t l ti P l i l

Interpolation Polynomial

(83)

Error Estimate

83

(84)

Ex3. Error Estimate of Linear Interpolation

9 0 ≤ t ≤ 9 5 9.0 ≤ t ≤ 9.5

(85)

Ex3. Error Estimate of Linear Interpolation

85

(86)

Ex3. Error Estimate of Linear Interpolation

(87)

Interpolation Interpolation

Linear Lagrange Interpolation

Quadratic Lagrange Interpolation Quadratic Lagrange Interpolation

General Lagrange Interpolation Error Estimate

87

(88)

19 5 Numeric Integration 19.5 Numeric Integration

& Differentiation

(89)

N i l ti f Numeric evaluation of

integrals integrals

89

(90)

Rectangular Rule

middle point of 1-st subinterval

(91)

Trapezoidal Rule

91

(92)

19.5 Numeric Integration

Ex1. Trapezoidal Rule

F(0)+f(1)

(93)

93

(94)

0.746824

(95)

95

(96)

0.746824

(97)

Error Bounds and

Estimate for Trapezoidal Rule

97

(98)

Error Bounds and

Estimate for Trapezoidal Rule

(99)

Error Bounds and

Estimate for Trapezoidal Rule

99

(100)

Error Bounds and

Estimate for Trapezoidal Rule

f

(101)

Ex2. Error Estimation for Trapezoidal Rule

101

(102)

Simpson’s Rule Simpson s Rule

of Integration

(103)

Piecewise constant approximation pp Rectangular rule

Piecewise linear approximation Trapezoidal rule

Trapezoidal rule

Piecewise quadratic approximation Simpson’s rule p

103

(104)

Divide the interval

i t b

into even number

of equal subintervals

(105)

105

(106)

-h -2h h -h 2h h

(107)

107

(108)

(109)

Error of Simpson’s Rule Error Bound

Error of Simpson s Rule, Error Bound, Degree of Precision

Because Simpson rule uses second order, if n=2, error of Simpson rule has f⁽³⁾ derivatives ((5) in pp.800 & (3*) in pp.819).

B hi th bl i T l l i l

By approaching the problem using Taylor polynomial, we can derive the error involves the fourth derivative of f⁽⁴⁾ .

Simpson’s rule gives exact results when applied to any polynomial of degree three or less

polynomial of degree three or less.

109

(110)

Error of Simpson’s Rule Error Bound

Error of Simpson s Rule, Error Bound,

Degree of Precision

(111)

Error of Simpson’s Rule Error Bound

Error of Simpson s Rule, Error Bound, Degree of Precision

111

(112)

Error of Simpson’s Rule Error Bound

Error of Simpson s Rule, Error Bound, Degree of Precision

• Newton–Cotes formulas are a group of formulas for numerical integration based on

evaluating the integrand at equally-spaced points

Closed Newton-cotes formula, [a,b]

Open Newton-cotes formula, (a,b)

(113)

Numeric Differentiation

113

(114)

Computation of values of derivative of f(x)

from a given value of f(x)

(115)

Numeric differentiation should be avoided

whenever possible whenever possible

115

(116)

큰 값의 작은 차를 큰 값의 작은 차를

작은 양으로 나눈 것의 극한

약간의 부정확한

f(x)

값이라도 약간의 부정확한

f(x)

값이라도

f’(x)

에 큰 영향을 줄 수 있음

(117)

19.5 Numeric Differentiation

f(x ) f(x₁) f(x₀)

x₀ x_1/2x₁ h

117

(118)

f(x) = x

⁴

f’(1.5) = ?

f(x ) f(x₁) f(x₀)

1.5 x_1/2 2.0

x₁ 1.0

x₀

118 x_1/2

h=1

x₁ x₀

(119)

f’’(x ) ?

f’’(x

₁

) = ?

119

(120)

f(x) x

⁴

f(x) = x

⁴

f’’(2) = ?

(121)

Lagrange polynomial Lagrange polynomial

approximation, differentiation

P

₂

(x) =f

₀

L

₀

+f

₁

L

₁

+f

₂

L

₂

-h -2h

L

₀

= (x-x

₁

)(x-x

₂

)/(x

₀

-x

₁

)(x

₀

-x

₂

) L

₁

= (x-x

₀

)(x-x

₂

)/(x

₁

-x

₀

)(x

₀

-x

₂

)

L ( )( )/( )( )

L

₂

= (x-x

₀

)(x-x

₁

)/(x

₂

-x

₀

)(x

₂

-x

₁

)

121

(122)

Lagrange polynomial Lagrange polynomial

approximation, differentiation

x=x₀ x=x₁ x=x₂

(123)

Numeric Integration g

Rectangular Rule Trapezoidal Rule Trapezoidal Rule

Simpson Rule Error Estimate

Differentiation Differentiation

123