You can test the complexity of a sorting algorithm in the following way:
•Write a function or functions that will do the sorting algorithm.
•Put in two global counters called compareCount and swapCount. The first should be incremented before every comparison of list values in every routine. The second should be incremented whenever list elements are exchanged or moved.
•Write a main program with a loop that generates a random list and then sorts it. In each pass of this loop, you should keep track of the maximum, minimum, and total values for both compareCount and swapCount. At the end, you can report the overall maximum, minimum, and averages for these two counters. The more times you perform this loop, the more accu- rate your results will be.
•If you are trying to compare sort routines, you should run them on the same list or lists. The easiest way to do this is to have a set of counters for each sort, and then when you generate a random list, make a copy of the list to pass into each sort. You would then run all of the sorts on the first list before generating the next list.
1. Use the technique above with insertion sort and bubble sort. Even though both are O(N2) sorts, does your test show any differences? How do your results relate to the analysis done in this chapter? Try to explain any differ- ences.
2. Use the technique above with heapsort, merge sort, and quicksort. Even though all are O(N lg N) sorts on average, does your test show any differ- ences? How do your results relate to the analysis done in this chapter? Try to explain any differences.
3. Use the technique above with the version of bubble sort given in the chap- ter and one or more of the versions described in the exercises in Section
3.2.4. Does your test show any differences? How do your results relate to the analysis done in this chapter and as part of the exercises? Try to explain any differences.
4. Use the technique above with quicksort using the versions of PivotList given in the chapter and in the exercises in Section 3.7.3. Does your test show any differences? How do your results relate to the analysis done in this chapter and as part of the exercises? Try to explain any differences.
5. Using the previous general technique, investigate the impact of the incre- ment on shellsort. You should create a version of the ShellSort function that has an additional parameter, which is just an array with the increment values to use in decreasing order. You will need to alter this function so it uses the values passed in instead of the increments that are calculated based on the powers of 2. You should work with random lists of 250 elements and make sure that each of the sets of increments discussed in Section 3.3 is used with each list generated. How do your results relate to the analysis done in Section 3.3? Try to explain any differences.
6. Some people find it easier to understand an algorithm if they can visualize it in action. The numeric values in a list can be visualized by drawing a vertical bar for each list element with the first element to the left and the last to the right. The height of each bar is based on the value stored at that element.
For example, if we have a list with the values from 1 to 500, the location where the 1 is stored would have a bar 1 unit high and the location where the 500 is stored would have a bar 500 units high. A random list would have the bars all mixed up, but a list sorted in increasing order would appear as a triangle with the shortest bar to the left and the largest to the right.
The operation of a sorting algorithm can be seen if the list is displayed as described above after each pass of the sorting algorithm. This allows the viewer to watch as the elements are moved into their proper position by the sort algorithm. Write a program (or programs) using randomly arranged lists for the cases below. You should use lists of between 250 and 500 elements, depending on capabilities of the computer(s) you are using. On fast com- puters, you may need to put in a short delay so that the visualization does not happen too quickly.
a. Insertion sort—displaying the list at the end of each pass of the outer loop
b. Bubble sort—displaying the list at the end of each pass of the outer loop c. Shellsort—displaying the list after each call to the modified insertion sort d. Heapsort—displaying the list after each call to FixHeap
e. Merge sort—displaying the list after each call to MergeLists f. Quicksort—displaying the list after each call to PivotList
C H A P T E R
4
Numeric Algorithms
PREREQUISITES
Before beginning this chapter, you should be able to
•Do simple algebra
•Evaluate polynomials
•Describe growth rates and order GOALS
At the end of this chapter, you should be able to
•Evaluate a polynomial by Horner's method
•Evaluate a polynomial by preprocessing its coefficients
•Explain the analysis of preprocessed coefficients
•Explain matrix multiplication
•Trace Winograd's matrix multiplication
•Explain the analysis of Winograd's matrix multiplication
•Use Strassen's matrix multiplication STUDY SUGGESTIONS
As you are working through the chapter, you should rework the examples to make sure you understand them. In addition, you should trace Horner’s
method and calculate the preprocessed coefficients for the polynomials x3 + 4x2 3x + 2 and x7 8x6 + 3x5 + 2x4 4x3 + 5x 7. You should trace the standard matrix multiplication algorithm, Winograd's matrix multiplication algorithm, and Strassen's matrix multiplication with
You should trace the Gauss-Jordan method with the equations 3x1 + 9x2 + 6x3 = 21, 5x1 + 3x2 + 22x3 = 23, and 2x1 + 8x2 + 7x3 = 26. You should also try to answer any questions before reading on. A hint or the answer is in the sen- tences following the question.
athematical calculation forms the basis for a wide range of pro- grams. Computer graphics and vision both require a large number of calculations involving polynomials and matrices. Because these are typically done for each location in an image, small improvements can have a great impact. A typical image can be created with 1024 pixels per row and 1024 pixels per column. Improving the calculation for each of these locations by even one multiplication would reduce the creation of this image by 1,048,576 multiplications overall. So, even though the techniques in this chap- ter don't seem to show a dramatic improvement, the real savings come from the number of times that these are used.
Some software will repeatedly evaluate complex polynomial equations. This can be part of a monitoring task where input from an external device is the value “plugged into” the equation, and the result tells if there is some condi- tion that needs attention. Another application is trigonometric functions. What most programmers do not realize is that standard trigonometric functions like sine and cosine have power series expansions that take the form of polynomial equations, and it is these equations that a computer will use when calculating trigonometric function results. These calculations need to be fast, and so, we begin by looking at methods for more rapidly calculating polynomials.
Matrix multiplication plays a role in a number of applications. Models of physical objects for computer graphics and computer-aided design and manu-
1 4
5 8 and 6 7
3 2
M
facturing can be moved and manipulated through matrix operations. Image analysis will use matrices in convolution operations to improve the quality of an image and to identify the bounds of objects in a picture. Fourier analysis will describe complex wave patterns in terms of simpler sine waves through matrix manipulation.
The common issue in all of these cases is that the matrix operations are done frequently, and so faster matrix multiplication results in faster programs.
Object transformations use 4 4 matrices, convolutions can use square matri- ces from 3 3 up to 11 11 or bigger, but they are used a very large number of times. Convolutions, for example, will take a matrix and multiply it by blocks of pixels in an image for every possible location. This means that for a 5 5 template used with a small image of 512 512 pixels (about one-quarter of a typical computer screen), a convolution will multiply this matrix by 258,064 different locations (508 508). If the standard matrix multiplication algorithm is used, this will result in 32,258,000 multiplications. A more efficient matrix multiplication algorithm can save significant time in this application.
In this chapter we will investigate ways to make polynomial evaluation and matrix multiplication more efficient. Because we are interested in how many calculations are done, we will be counting additions and multiplications. When we considered searching and sorting, we found equations that were based on the size of the list. In analyzing numeric algorithms, we will base our equations on the power of the highest order term in a polynomial equation, or the dimensions of the matrices we are multiplying.