What is Numerical Analysis?
Numerical analysis deals with the mathematical derivation, description and analysis of methods of obtaining numerical solutions of mathematical problems.
Numerical methods are mathematical techniques used for solving mathematical problems that cannot be solved or are difficult to solve. The numerical solution is an approximate numerical value for the solution. Although numerical solutions are an approximation, they can be very accurate. Approximation is unavoidable in mathematical modelling the real world phenomena. Approximation leads to errors. Estimating the errors in computation is necessary for reliability of computed results.
In many numerical methods, the calculations are executed in an iterative manner until a desired accuracy is achieved.
Today, numerical methods are used in fast electronic digital computers that make it possible to execute many tedious and repetitive calculations that produce accurate (even though not exact) solutions in a very short time.
For every type of mathematical problem there are several numerical techniques that can be used. The techniques differ in accuracy, length of calculations, and difficulty in programming. However, one cannot choose the best technique for the problem in hand without having a sound understanding of the theoretical principles involved.
We shall discuss these techniques this branch of mathematics.
Historical background.
Numerical algorithms are almost as old as human civilization. The Rhind Papyrus (1650 BC) of ancient Egypt describes a root finding method for solving a simple equation. Archimedes of Syracuse (287- 212 BC) created much new mathematics, including the “method of exhaustion” for calculating lengths, areas, and volumes of geometric figures. When used as a method to find approximations, it is in much the spirit of modern numerical integration; and it was an important precursor to the development of the calculus by Isaac Newton
and Gottfried Leibnitz. A major impetus to developing numerical procedures was the invention of the calculus by Newton and Leibnitz, as this led to accurate mathematical models for physical reality, first in the physical sciences and eventually in the other sciences, engineering, medicine, and business. These mathematical models cannot usually be solved explicitly, and numerical methods to obtain approximate solutions are needed. Another important aspect of the development of numerical methods was the creation of logarithms by Napier (1614) and others, giving a much simpler manner of carrying out the arithmetic operations of multiplication, division, and exponentiation. Newton created a number of numerical methods for solving a variety of problems, and his name is attached today to generalizations of his original ideas. Of special note is his work on root finding and polynomial interpolation. Following Newton, many of the giants of mathematics of the 18th and 19th centuries made major contributions to the numerical solution of mathematical problems. Foremost among these are Leonhard Euler (1707-1783), Joseph-Louis Lagrange (1736-1813), and Karl Friedrich Gauss (1777-1855). Up to the late 1800’s, it appears that most mathematicians were quite broad in their interests, and many of them were interested in and contributed to numerical analysis.
Characteristic of Numerical Methods
1. The solution procedure is iterative and generally the accuracy of the solution improving with each iteration.
2. The solution procedure provides only an approximation to the true solution.
3. An initial estimate of the solution may be required.
4. The algorithm is simple and can be easily programmed.
5. The solution procedure may occasionally diverge from rather than converge to the true solution.
Errors
Since numerical solutions are an approximation, and since the computer program that executes the numerical method might have errors, a numerical solution needs
to be examined closely. There are three major sources of error in computation:
human errors, truncation errors, and round-off errors.
Error in solving an engineering or science problem can arise due to several factors.
First, the error may be in the modeling technique. A mathematical model may be based on using assumptions that are not acceptable. For example, one may assume that the drag force on a car is proportional to the velocity of the car, but actually it is proportional to the square of the velocity of the car. This itself can create huge errors in determining the performance of the car, no matter how accurate the numerical methods you may use are. Second, errors may arise from mistakes in programs themselves or in the measurement of physical quantities.
In any applied numerical computation, there are four key sources of error:
(i ) Inexactness of the mathematical model for the underlying physical phenomenon.
(ii ) Errors in measurements of parameters entering the model.
(iii ) Round-off errors in computer arithmetic.
(iv) Approximations used to solve the full mathematical system.
Of these, (i ) and (ii) are the domain of mathematical modeling and experimentalists respectively, will not be discussed here. Round-off error arises due to the finite numerical precision imposed by the computer. Approximations is the true domain of numerical analysis, and refers to the fact that most systems of equations are too complicated to solve explicitly, or, even in cases when an analytic solution formula is known, directly obtaining the precise numerical values may be difficult.
In general, errors can be classified based on their sources as non-numerical and numerical errors.
Non-numerical errors:
(1) modeling errors: generated by assumptions and limitations.
(2) blunders and mistakes: human errors (3) uncertainty in information and data
Numerical errors:
(1) round-off errors: due to a limited number of significant digits
(2) truncation errors: due to the truncated terms e.g. infinite Taylor series
(3) propagation errors: due to a sequence of operations. It can be reduced with a good
computational order. e.g. In summing several values, we can rank the values in ascending order before performing the summation.
(4) mathematical-approximation errors:
e.g. To use a linear model for representing a nonlinear expression.
In applications of numerical methods itself, the two errors we need to focus on are
1. Truncation error.
2. Round off error
1. Truncation error.
Error(e): the difference between the computed (xc) and true (xt) values of a number
𝑒 = 𝑥𝑐− 𝑥𝑡
The relative true error (er) :
𝑒𝑟= 𝑥𝑐− 𝑥𝑡 𝑥𝑡
In practice, the true value is not known, so we cannot get the relative true error.
We use ∆𝑒𝑖 = 𝑥𝑖+1− 𝑥𝑖 to measure the error, where 𝑥𝑖 is the computed vale of x at ith iteration.
Significant digits
If 36.13 is exact to the four digits shown, it has four significant digits (The last digit is imprecise). The error is no more than 0.005.
The digits from 1 to 9 are always significant, with zero being significant where it is not being used to set the position of the decimal point.
• Precision: The ability to give multiple estimates that are near to each other (a measure of random deviations).
• Bias: The difference between the center of the holes and the center of the target (a systematic deviation of values from the true value).
• Accuracy: The degree to which the measurements deviate from the true value.
(a) inaccurate and imprecise,
(b) accurate and imprecise,
(c) inaccurate and precise
(d) accurate and precise.Round-off error
Round off errors occur due to the way in which computers represent numerical values . Computer representation of numerical values is limited in terms of Magnitude – there are upper and lower bounds on the magnitude of numbers that can be represented .
Precision – not all numbers can be represented exactly.
Round-off errors arise because digital computers cannot represent some quantities exactly. There are two major facets of round-off errors involved in numerical calculations: Digital computers have size and precision limits on their ability to represent numbers. Certain numerical manipulations are highly sensitive to round-off errors.
The Patriot Missile catastrophically missed its target due to rounding errors.
(Courtesy of the US Armed Forces,
http://www.redstone.army.mil/history/archives/patriot/patriot.html)
The total numerical error is the sum of the truncation and round off errors. The truncation error generally increases as the step size increases, while the round off error decreases as the step size increases - this leads to a point of diminishing returns for step size.
Referrences
[1] H. Goldstine (1977) A History of Numerical Analysis: From the 16th Through the 19th Century, Springer-Verlag.
