Although there are many books on numerical methods where many of the algorithms have corresponding computer implementations (see, e.g., the last two are the only texts we know of that use Python), it is assumed that the reader "knows how to program" in advance. With this attitude in mind, many topics have been omitted from the present book, simply because they were not absolutely necessary in the process of solving mathematical problems.
What Is a Program? And What Is Programming?
With some programming knowledge, you can write your own little program that can translate one data format to another. Well, you can write down the recipe in these three languages and send it.
A Python Program with Variables
- The Program
- Dissection of the Program
- Why Not Just Use a Pocket Calculator?
- Why You Must Use a Text Editor to Write Programs
- Installation of Python
- Write and Run Your First Program
You must know the consequences of every instruction in the program and be able to determine the consequences of the instructions. This makes Python print the value of y in that window on the screen where you ran the program.
A Python Program with a Library Function
This means that where we see atan(y/x), a calculation is performed (tan1.y=x/) and the result "replaces" textatan(y/x). This is actually no more magic than if we had written justy/x: then the calculation ofy/x would take place, and the result of that division would replace texty/x.
A Python Program with Vectorization and Plotting
However, we actually only use a few functions in the actual program: linspace,plot,xlabel, andylabel. It is based on the example above, but is much simpler in terms of the math and the amount of numbers involved.
More Basic Concepts
- Using Python Interactively
- Arithmetics, Parentheses and Rounding Errors
- Variables and Objects
- Integer Division
- Formatting Text and Numbers
- Arrays
- Plotting
- Error Messages and Warnings
- Input Data
- Symbolic Computations
- Concluding Remarks
Very often you will find that there are errors in the program you have written. A special program (debugger) can be used to help you check (and do) various things in the program you need to fix.
Exercises
Write a program that calculates the volumeV of a cube with sides of lengthLD4 cm and print the result on the screen. Both V and L must be defined as separate variables in the program. Then let the program calculate the product of these two variables and print the result on the screen as .
If Tests, Colon and Indentation
If the answer to the "if" question is positive (true), we are done and can skip the next questions. If the condition is false, the program flow jumps to the next elif or other branch.
Functions
Python does not look at the function at this point (other than checking the code for syntax errors). When calling the function (and print), arguments must appear in the same order as in the function definition.
For Loops
The value iny[0] is used as the provisional largest height, so that the very first check that is made, for example, is to test whether [1] is greater than this height. At the same time, however, it is something that programmers often do, so it is important to develop the right skills on these issues.
While Loops
The value ofx is then updated with each iteration of the loop, and not until the loop is finished will x have the correct value. This way of building the value is very common in programming, so make sure you understand it by simulating the code segment above by hand.
Lists and Tuples – Alternatives to Arrays
We can add and/or delete elements anywhere in the list as shown in the example below. That is, writing += 2 will not change anything inx, since it can only be used to read (as opposed to writing) the elements of the list.
Reading from and Writing to Files
Exercises
Have the program make a formatted printout of the array to the screen, both before and after sorting. The length of the string is given bylen(none), so an alternative traversal over an index isfor i in range(len(none)).
Basic Ideas of Numerical Integration
Starting from (3.5), the different integration methods will differ in the way they approach each integral on the right-hand side. We can therefore calculate the exact value of the integral as V .1/V .0/1:718 (rounded to 3 decimal places for convenience).
The Composite Trapezoidal Rule
- The General Formula
- Implementation
- Making a Module
- Alternative Flat Special-Purpose Implementation
Strictly speaking, writing, say, "the trapezoidal method" should imply the use of only one trapezoid, while "the compound trapezoidal method" is the most correct name when multiple trapezoids are used. How much do we need to change in the previous code to calculate the new integral.
The Composite Midpoint Method
The General Formula
Implementation
Comparing the Trapezoidal and the Midpoint Methods
The trapezoid and midpoint methods are just two examples in a jungle of numerical integration rules. The different methods differ in the way they construct the evaluation pointsxi and the weightswi.
Testing
- Problems with Brief Testing Procedures
- Proper Test Procedures
- Finite Precision of Floating-Point Numbers
- Constructing Unit Tests and Writing Test Functions
In the trapezoidal and midpoint rules it is known that the error depends on n as n2 as n. As we demonstrate below, these tolerances depend on the size of the numbers in the calculations.
Vectorization
Vectorized code performs all the loops very efficiently in compiled code, resulting in much faster execution. Furthermore, many readers of the code will also say that the algorithm looks more clear than in the loop-based implementation.
Measuring Computational Speed
The results for the trapezoidal method are very similar and the factor of about 20 is independent of the number of intervals.
Double and Triple Integrals
The Midpoint Rule for a Double Integral
Direct derivation The formula (3.25) can also be derived directly in the two-dimensional case by applying the idea of the midpoint method. Deriving a formula for the double integral and its implementations follow exactly the same ideas as we explained with the midpoint method, but there are more terms to write in the formulas.
The Midpoint Rule for a Triple Integral
Note that we can use the ideas under Direct derivation at the end of Section 3.7.1 to directly arrive at (3.26): divide the domain into cells of volumes hxhyhz; approximateg with a constant evaluated in the middle.xi; yj; zk/, in each cell; and sum cell integralshxhyhzg.xi; yj; zk/. Implementation Let's follow the ideas for implementations of the central rule for the double integral.
Monte Carlo Integration for Complex-Shaped Domains
All we know is that the evaluated integral must approach 3 as the number of random points goes to infinity. Mathematically, it is known that the standard deviation of the Monte Carlo estimate of an integral converges to asn1=2, where is the number of samples.
Exercises
The height of the rectangle can be based on the left or right end or center. A remarkable property of the trapezoidal rule is that it is exact for integralR sinnt dt (when the subintervals are of equal size).
Population Growth
Derivation of the Model
We are not concerned with the spatial distribution of animals, but with their number in a spatial area where there is no exchange of individuals with other spatial areas. Also, we introduce Dbd, which is the net population growth rate per unit time.
Numerical Solution
Such an algorithm is called an numerical scheme for the differential equation and is often compactly written as However, the formula (4.8) arises from a detour via a differential equation and a numerical method for the differential equation.
Programming the Forward Euler Scheme; the Special Case 102
The beauty of the Forward Euler method is that it gives an understanding of what a differential equation is and a geometric view of how to construct the solution. We know that the line must pass through the solution attn, i.e., the point.tn; i/.
Programming the Forward Euler Scheme; the General Case 105
We now encourage you to do Exercise 4.1 to become more familiar with the geometric interpretation of Forward Euler's method. Another option is to use the Forward Euler formula for the general problem u0Df .u; t /in usef .u; t /Dr .u/uin replace zN. The simplest choice of r .N /is a linear function that starts at some value of growth rN and decreases until the population reaches its maximum,M, given the available resources:.
Verification: Exact Linear Solution of the Discrete Equations 109
Currently, world population forecasts show growth to 9.6 billion before declining. The test functions should make the assert test succeed (here success can also be a boolean expression like inassert diff < tol).
Spreading of a Flu
The expected number of individuals in category S who catch the virus and become infected in the time interval t is then ptSI. Since there is no loss in the R category (people are either healed and immune or dead), we are done modeling this category.
A Forward Euler Method for the Differential Equation
This differential equation model (and also its discrete counterpart above) is known as the SIR model. The input data for the differential equation model consists of parametersˇ and initial conditions S.0/ D S0,I.0/ D I0 and R.0/DR0.
Programming the Numerical Method; the Special Case
Outbreak or Not
Abstract Problem and Notation
The equationnu0Df .u; t / means to set the two vectors equal, ie. the components must be pairwise identical.
Programming the Numerical Method; the General Case
For a user, who only needs to define thef in the ODE system, it is convenient to put the various mathematical expressions on the right into a list and return that list. The user's function f(u, t) takes a vector u, with three components corresponding to the argument S, I, and Ras, along with the current time t[n], and must take the values of the formulas of the right sides render in the vector ODE.
Time-Restricted Immunity
Recall that theureturned fromode_FE contains all components (S,I,R) in the solution vector at all time points. Therefore, loss in category R can be written as astR v timet, where 1 is the typical time required for loss of immunity.
Incorporating Vaccination
Modeling loss of immunity is very similar to modeling recovery from disease: the number of people who lose immunity is proportional to the number of patients recovered and the length of the time interval. Setting 1 in 50 days, reducing by a factor of 4 compared to the previous example (ˇ D0:00033), and simulating for 300 days produces an oscillating behavior in the categories, as shown in Fig.4.11.
Discontinuous Coefficients: A Vaccination Campaign
Oscillating One-Dimensional Systems
Derivation of a Simple Model
At x D0 the spring is not stretched, so the force is zero, and x D0 is therefore the equilibrium position of the body. Equation (4.42) is a second-order differential equation, so we need two initial conditions, one for the position x.0/ and one for the velocity x0.0/.
Numerical Solution
Programming the Numerical Method; the Special Case
Simulating for three periods the cosine function, T D 3P, and choosing such that there are 20 intervals per period gives DP =20 and a total of Nt DT = tintervals. Figure 4.16 shows a comparison between the numerical solution and the exact solution of the differential equation.
A Magic Fix of the Numerical Method
The error in the backward difference is total proportional, the same as for the forward difference (but the proportionality constant in the error term has a different sign). The standard way of expressing this scheme in physics is to change the order of the equations.
The 2nd-Order Runge-Kutta Method (or Heun’s Method) . 131
This simply means that we redefine the name inside the function to mean the solution in terms of the first component of the ODE system. This method is also popularly known as ode45, because this is the name of the function that implements the method in Matlab.
The 4th-Order Runge-Kutta Method
Implementation The phases of the 4th order Runge-Kutta method can be easily implemented as a modification of theosc_Heun.pycode. Note that the 4th-order Runge-Kutta method is completely explicit, so there is never any need to solve linear or non-linear algebraic equations, no matter how it looks.
More Effects: Damping, Nonlinearity, and External Forces 141
We can choose simple values for the constants to illustrate basic effects of damping (and later excitations). But the solution of the dimensionless problem is more general: if we have a solutionu.N tNIˇ/, we can find the physical solution of a number of problems since.
Illustration of Linear Damping with Sinusoidal Excitation . 146
Due to the contact between the body and the plane, the friction force f .u0/ also acts on the body. The initial displacement of the body is 10 cm, and the parameter ins.u/ is set to 60 1/m.
A finite Difference Method; Undamped, Linear Case
It turns out that this method is mathematically equivalent to the Euler-Cromer scheme. Or more precisely, the general formula (4.76) is equivalent to the Euler-Cromer formula, but the scheme for the.
A Finite Difference Method; Linear Damping
There will be a phase error as in the Euler-Cromer scheme, but this error is effectively reduced by reduction, as already shown. Using the same trick in the finite difference scheme for the second order differential equation, i.e. using the backward difference inf .u0/, this scheme is as convenient and accurate as the Euler-Cromer scheme in general nonlinear casemu00Cf .u0/Cs .u/DF.
Exercises
Plot the solutions corresponding to the last two time steps entk1 in the same plot. Assume that the initial condition onu0 is non-zero in the finite difference method of Sect.4.3.12:u0.0/DV0.
Finite Difference Methods
- Reduction of a PDE to a System of ODEs
- Construction of a Test Problem with Known Discrete
- Implementation: Forward Euler Method
- Application: Heat Conduction in a Rod
- Vectorization
- Using Odespy to Solve the System of ODEs
- Implicit Methods
When we have the solution u.N x;N t /, the solution with N dimension Kelvin, which reflects the true temperature in our medium, is given by. Figure 5.3 shows four snapshots of the scaled (dimensionless) solutionN.x;N t /.N The power of scale is to reduce the number of physical parameters in a problem, and in the present case we have found one single problem that is independent of the material (ˇ) and the geometry (L).
Exercises
There is no source term in the equation (actually, if rocks in the ground are radioactive, they radiate heat and this can be modeled by a source term, but this effect is neglected here). Filename:ground_temp_adaptive.py.. a) The Crank-Nicolson method for ODEs is very popular when combined with diffusion equations.
Brute Force Methods
Brute Force Root Finding
Brute Force Optimization
Model Problem for Algebraic Equations
Newton’s Method
Deriving and Implementing Newton’s Method
Making a More Efficient and Robust Implementation
The Secant Method
The Bisection Method
Rate of Convergence
Solving Multiple Nonlinear Algebraic Equations
Abstract Notation
Taylor Expansions for Multi-Variable Functions
Newton’s Method
Implementation
Exercises
Required Software
Anaconda and Spyder
Spyder on Mac
Installation of Additional Packages
How to Write and Run a Python Program
The Need for a Text Editor
Text Editors
Terminal Windows
Using a Plain Text Editor and a Terminal Window
Spyder
The SageMathCloud and Wakari Web Services
Basic Intro to SageMathCloud
Basic Intro to Wakari
Installing Your Own Python Packages
Writing IPython Notebooks
A Simple Program in the Notebook
Mixing Text, Mathematics, Code, and Graphics
Error messages
An important source of information is the official Python documentation website (http://docs.python.org/), which provides a Python tutorial, the Python library reference, a language reference, and more. Several excellent books are also available (http://wiki.python.org/moin/PythonBooks), but not so many with a scientific computing focus.
Volume of a cube
Area and circumference of a circle
Volumes of three cubes
Average of integers
Interactive computing of volume and area
Peculiar results from division
Update variable at command prompt
Formatted print to screen
Python documentation and random numbers
The arguments named in the header of a function definition are line-by-line local variables within the function. If we do that with our top-down xy function, we can write the first lines of the function as .
Errors with colon, indent, etc
Compare integers a and b
Functions for circumference and area of a circle
Function for area of a rectangle
Area of a polygon
Write a function polyarea(x, y) that takes two coordinate arrays with the vertices as arguments and returns the area. Test the function on a triangle, a quadrilateral and a pentagon where you can calculate the area by alternative methods for comparison.
Average of integers
While loop with errors
Area of rectangle versus circle
Find crossing points of two graphs
Sort array with numbers
Letter numberi is accessed through gen[i], and a substring from indexiup to but not includingj is created by gen[i:j]. a) Write a function freq(letter, text) that returns the frequency of the letter in the string text, i.e. the number of occurrences of letter divided by the length of text. The leading sequence completes the series for the error, ie. the error with the smallest strength is often a good approximation of the error.
Compute
Compute combinations of sets
Write statements that create a set of cards, ie. all combinations of CA, C2, C3 and so on up to SK. b). The vehicle registration number is on form DE562 where letters vary from A to Z and numbers from 0 to 9. Write statements that calculate all possible registration numbers and store them in a list.
Frequency of random numbers
Game 21
Linear interpolation
Count how many combinations where the sum of the eyes is equal to 7. File name: combine_sets.py.
Test straight line requirement
Fit straight line to data
Fit sines to straight line
Call trial(f, 3) and try to find by experimentation some values b1,b2, andb3 such that the sum of sinesSN.t / is a good approximation to the straight line. g) Now we will try to automate the procedure in f). In real applications, f .t / is not known to be a continuous function, but function values off .t / are provided.
Count occurrences of a string in a string
We see that all the numbers have an inaccurate digit in the 17th decimal place. As in the test case above, we experience better results with a larger number of points.
Hand calculations for the trapezoidal method
Hand calculations for the midpoint method
Compute a simple integral
Hand-calculations with sine integrals
Make test functions for the midpoint method
Explore rounding errors with large numbers
One common way to do this is to first set up a general expression for the approximation error measured by .. summing” the squared deviation of SN from: ED. The probability that people meet in pairs at time T is (using the empirical frequency definition of probability) equal tom=n, i.e. the number of successes divided by the number of possible outcomes.
Write test functions for R 4
Rectangle methods
Adaptive integration
Integrating x raised to x
Note that Wolfram alpha gives you an answer, but that answer is an approximation, not exact. This is because Wolfram alpha also uses numerical methods to arrive at the answer, as you will in this exercise).
Integrate products of sine functions
Integrandxx does not have an anti-derivative that can be expressed in terms of standard functions (visit http://wolframalpha.com and typeintegral(x**x,x) to convince yourself that our claim is correct.
Revisit fit of sines to a function
Use this property to create a function test_integrate_coeff to verify the implementation of integrate_coeffs. e) Implement the choicef .t / D 1t as a Python function f(t) and call integrate_coeffs(f, 3, 100) to see what the optimal choice of b1 is; b2; b3is. f) Make a function plot_approx(f, N, M, filename) where you plotf(t) together with the best approximationSN calculated as above, with M intervals for numerical integration.
Derive the trapezoidal rule for a double integral
Compute the area of a triangle by Monte Carlo integration
The error in the forward difference is proportional tot (often written as O.t /, but we will not use this notation in this book). We can check that this relation holds by comparing SnCInCRn with the sum of the initial conditions.
Geometric construction of the Forward Euler method
Make test functions for the Forward Euler method
Implement and evaluate Heun’s method
It is interesting to see how far off the curve the Forward Euler method is when Heun's method can be considered "exact" (for visual purposes).
Find an appropriate time step; logistic model
Find an appropriate time step; SIR model
Model an adaptive vaccination campaign
Make a SIRV model with time-limited effect of vaccination
Write up the complete model, implement it and repeat the case from Section 4.2.8 with various choices of parameters to illustrate various effects.
Refactor a flat program
Simulate oscillations by a general ODE solver
Compute the energy in oscillations
Use a Backward Euler scheme for population growth
Use a Crank-Nicolson scheme for population growth
Understand finite differences via Taylor series
Write down the Taylor series foru.tn/aroundtnC12t and the Taylor series for u.tnCt / aroundtnC 12t. Subtract the two series, solve with respect to u0.tnC12t/, identify the finite difference approximation and the error terms on the right, and note the leading-order error term. Can you use the leading error terms in a)-c) to explain the visual observations in the numerical experiment in Exercise 4.12?
Use a Backward Euler scheme for oscillations
Use Heun’s method for the SIR model
Use Odespy to solve a simple ODE
Set up a Backward Euler scheme for oscillations
Set up a Forward Euler scheme for nonlinear and damped
Discretize an initial condition
The solution of the equation is not unique unless we also prescribe initial and boundary conditions. In addition, the diffusion equation needs one boundary condition at each point of the boundary@˝of˝.
Simulate a diffusion equation by hand
Compute temperature variations in the ground
Compare implicit methods
Explore adaptive and implicit methods
Investigate the rule
File name: rod_BE_vs_B2Step.py. b) The Backward Euler, Forward Euler and Crank-Nicolson methods can have a uniform implementation. For D0 we restore the Forward Euler method, D 1 gives the Backward Euler scheme and D 1=2 corresponds to the Crank-Nicolson method.
Compute the diffusion of a Gaussian peak
The approximation error in the rule is proportional to t, except for D 1=2 where it is proportional to 2. Remarks Despite the fact that the Crank-Nicolson method, or the rule with D 1=2, is theoretically more accurate than the Backward Euler and Forward Euler schemes, it can exhibit non-physical oscillations as in the present example if the solution is very steep.
Vectorize a function for computing the area of a polygon
Using the Gauss divergence theorem over the integral on the right and shifting the time derivative outside the integral on the left leads to the result. If the PDE gives an interpretation in terms of heat conduction, the result can be easily explained: with Neumann conditions, no heat can escape from the domain, so the initial heat will only be spread evenly, but not leak out, so the temperature does not go to zero (or the scaled and translated temperature u, to be precise).
Explore symmetry
Remarks If the interest lies in the stationary limit of a diffusion equation, one can either solve the associated Laplace or Poisson equation directly, or use a Backward Euler scheme for the time-dependent diffusion equation with a very long time step. When we solve algebraic equations f .x/D0, we often say that the solution x is a root of the equation.
Solve a two-point boundary value problem
So, when do we really need to solve algebraic equations as the simplest types that we can handle with pen and paper. Systems of nonlinear algebraic equations with many variables arise from implicit methods for ordinary and partial differential equations as well as in multivariable optimization.
Understand why Newton’s method can fail
See if the secant method fails
Understand why the bisection method cannot fail
Combine the bisection method with Newton’s method
Write a test function for Newton’s method
Solve nonlinear equation for a vibrating beam
