Our attention will be limited to Newton's method for such systems of nonlinear algebraic equations. For example, Newton's method is very fast but not reliable, while the halving method is the slowest but absolutely reliable. Newton's method, also known as Newton-Raphson's method, is a very famous and widely used method for solving non-linear algebraic equations.
The fundamental idea of Newton's method is to approximate the original function f .x/ by a straight line, i.e. a linear function, since it is simple to solve linear equations. This speed of the search for the solution is the primary strength of Newton's method compared to other methods. The underlying problem, which leads to the division by zero in the example above, is that Newton's method differs: the approximations move further and further away from x D 0.
Divergence of Newton's method occasionally occurs and the solution is to terminate the method when a maximum number of iterations is reached. If something goes wrong here, or more precisely, if Python raises an exception caused by a problem (such as division by zero, array index out of bounds, use of undefined variable, etc.), execution immediately jumps into the exception block. Because of its speed, Newton's method is often the method of first choice for solving nonlinear algebraic equations, even if the scheme is not guaranteed to work.
In cases where the initial guess may be far from the solution, a good strategy is to run a few iterations with the doubling method (see Chapter 6.4) to narrow the region where it is close to zero, and then switch to Newton's method for fast. convergence with the solution.
The Secant Method
As with Newton's method, the procedure is repeated until .xn/ is below a chosen threshold value or a limit on the number of iterations is reached. We also use a repetition counter here, based on the same reasoning as in the implementation of Newton's method. We can store the approximationsxn in an array, but as in Newton's method, we note that computing exnC1 only needs knowledge of xnandxn1, no.
The number of function calls is now related to tono_iterations ie. the number of iterations, as2 + no_iterations, since we need two function calls before entering the whileloop, and then one function call per loop iteration. Note that although we need two points on the graph to calculate each updated estimate, only .
The Bisection Method
Note that we first check if changes sign inŒa; b, because it is a requirement for the algorithm to work. The algorithm also relies on a continuousf .x/function, but this is very challenging for a computer code to check. We notice that the number of function calls is much higher than with the previous methods.
If the start interval of the bisection method is bounded by ondb, and the solution at step is considered to be the middle value, the error is bounded as . To satisfy a tolerance, we need iterations such that the length of the current interval is equal.
Rate of Convergence
Knowing the exact solution x off .x/D0, we can calculate all errors and all associated qn values with a compact function. However, for the halving method, it works well at the beginning, but not when it gets close to the solution.
Solving Multiple Nonlinear Algebraic Equations
- Abstract Notation
- Taylor Expansions for Multi-Variable Functions
- Newton’s Method
- Implementation
We follow the ideas of Newton's method for one equation in one variable: approximate the non-linear function by a linear function and find the root of that function. The technique to approximate F by a linear function is to use the first two terms in a Taylor series expansion. The following terms in the expansions are omitted here and of size jjxiC1xijj2, which are assumed to be small compared to the two terms above.
The idea of Newton's method is to have some approximation of x to the root and find a new (and hopefully better) approximation of xiC1 by approximating F.xiC1/ with a linear function and solving the corresponding linear system of algebraic equations. Gaussian elimination is the most common and generally the most robust method for this purpose. When nonlinear systems of algebraic equations arise from the discretization of partial differential equations, the Jacobian is very often sparse, i.e. most of its elements are zero.
In such cases, it is important to use algorithms that can take advantage of many zeros. Gaussian elimination is therefore a slow method, and (much) faster methods are based on iterative techniques.
Exercises
Understand why Newton’s method can fail
See if the secant method fails
Understand why the bisection method cannot fail
Combine the bisection method with Newton’s method
Write a test function for Newton’s method
Solve nonlinear equation for a vibrating beam
We want to calculate the frequencies of a vibrating steel beam with a rectangular cross-section with width b D25mm and height hD8mm. The moment of inertia of a rectangular section is I Dbh3=12. a) Plot the equation to be solved so that one can inspect where the zero crossings occur. Open Access This chapter is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any non-commercial use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link to the Creative Commons license is provided, and any changes made are indicated.
The images or other third-party material in this chapter are included in the work's Creative Commons license, unless otherwise noted in the credit line; if such material is not included in the work's Creative Commons license and the action in question is not permitted by statutory regulation, users must obtain permission from the licensee to duplicate, adapt or reproduce the material.
Getting Access to Python
- Required Software
- Anaconda and Spyder
- Spyder on Mac
- Installation of Additional Packages
- How to Write and Run a Python Program
- The Need for a Text Editor
- Text Editors
- Terminal Windows
- Using a Plain Text Editor and a Terminal Window
- Spyder
- The SageMathCloud and Wakari Web Services
- Basic Intro to SageMathCloud
- Basic Intro to Wakari
- Installing Your Own Python Packages
- Writing IPython Notebooks
- A Simple Program in the Notebook
- Mixing Text, Mathematics, Code, and Graphics
Since many tools for doing scientific computing in Python are still only available for Python version 2, we use this version in this book, but emphasize that it must be v2.7 and not older versions. Spaces are often important in Python programs, and plain text editors give you full control over the spaces and all other characters in the program file. This is most conveniently done in the terminal window, since you will need to use this window to run the program anyway.
Select Run from the Run drop-down menu and watch the Hello! in the lower right window, where the output from the programs is visible. The drawing file we create in the program above, tmp.png, is found by default in the Spyder folder listed in the default text at the top of the program. The program you write is written in a file.temp.py in the same default folder, but you can specify any name and folder in the standard file - Save As.
A useful feature of Spyder is that the upper right window continuously displays documentation of the statements you write in the editor on the left. Both sites require you to create an account before you can write notebooks in the web browser and download them to your own computer. Sign in, click New Project, give your project a title and decide whether it should be private or public, click the project when it appears in the browser, and click Create or import a file, worksheet, terminal, or folder.
If your Python program needs graphics, you must select IPython Notebook, otherwise you can select File. Clicking File opens a browser window with a text editor where you can write Python code. To run the program, click the plus (New) icon, select Terminal, and you'll have a regular Unix terminal window where you can type python py1.py to run the program.
Wakari allows you to create and edit simple Python files as well: click the Add File icon in the left panel, fill in the name of the program, and enter an editor where you can write a program. Pressing Execute opens an IPython session in a terminal window, where you can run the program dukerun prog.pyifprog.py is the name of the program. The notebook interface is a web browser: you type all the code and see all the results in the browser window.
You can then type print in a new cell, execute that cell, and see the output of that statement in the browser. As a teaser, open a new notebook, click in the first cell and select Markdownas format (notebook runs locally) or switch from Code to Markdown in the drop-down menu (notebook in the cloud).
