Sequences
4.6 Merge Sort
Fig. 4.4 Selection Sort of a List
106 4 Sequences
4.6.1 The Merge Sort Code
1 def merge(seq, start, mid, stop):
2 lst = []
3 i = start
4 j = mid
5
6 # Merge the two lists while each has more elements
7 while i < mid and j < stop:
8 if seq[i] < seq[j]:
9 lst.append(seq[i])
10 i+=1
11 else:
12 lst.append(seq[j])
13 j+=1
14
15 # Copy in the rest of the start to mid sequence
16 while i < mid:
17 lst.append(seq[i])
18 i+=1
19 # Many merge sort implementations copy the rest
20 # of the sequence from j to stop at this point.
21 # This is not necessary since in the next part
22 # of the code the same part of the sequence would
23 # be copied right back to the same place.
24 # while j < stop:
25 # lst.append(seq[j])
26 # j+=1
27 # Copy the elements back to the original sequence
28 for i in range(len(lst)):
29 seq[start+i]=lst[i]
30
31 def mergeSortRecursively(seq, start, stop):
32 # We must use >= here only when the sequence we are sorting
33 # is empty. Otherwise start == stop-1 in the base case.
34 if start >= stop-1:
35 return
36
37 mid = (start + stop) // 2
38
39 mergeSortRecursively(seq, start, mid)
40 mergeSortRecursively(seq, mid, stop)
41 merge(seq, start, mid, stop)
42
43 def mergeSort(seq):
44 mergeSortRecursively(seq, 0, len(seq))
The merge sort algorithm takes this divide and conquer strategy to the extreme. It divides the list, then divides it again and again, until we are left with lists of size 1.
A sublist of length 1 is already sorted. Two sorted sublists can be merged into one sorted list in O(n) time. A list can be divided into lists of size 1 by repeatedly splitting in O(log n) time. Each of the split lists are then merged together in O(n) time. This results in a complexity of O(n log n) for merge sort. The merge sort code appears in Sect.4.6.1.
Themergefunction takes care of merging two adjacent sublists. The first sublist runs from start to mid-1. The second sublist runs from mid to stop-1. The elements
of the two sorted sublists are copied, in O(n) time, to a new list. Then the sorted list is copied back into the original sequence, again in O(n) time. In themergefunction, the first while loop takes care of merging the two sublists until one or the other sublist is empty. The second and third while loops take care of finishing up whichever sublist had the left-over elements. Only one sublist will have left-over elements so only one condition on the second and third while loops will ever be true.
Notice that the third while loop in the code is commented out. Copying elements fromjtostopin the third while loop is not necessary since they would only be copied right back to the same place when the contents oflst are copied back to theseq sequence. This optimization speeds up merge sort a little bit. One other optimization is to pre-allocate one more list in which to copy values and then alternate between merging in the original and the pre-allocated copy. In this way the overhead of creating and appending to lists is avoided. Coding either of these two optimizations does not improve the computational complexity of the algorithm, but can improve its overall performance slightly. One criticism of the merge sort algorithm is that the elements of the two sublists cannot be merged without copying to a new list and then back again. Other sorting methods, like Quicksort, have the same O(n log n) complexity as merge sort and do not require an extra list.
ThemergeSortfunction calls a helper function to get everything started. It calls mergeSortRecursivelyfunction with the sequence and the start and stop values which indicate the entire list should be sorted. The start and stop parameters are used when splitting the list. The list is not physically split when callingmergeSortRecursively.
Instead, the start and stop values are used to compute the mid point between them and then the two halves are recursively sorted. Since each sublist is smaller, we can rest assured that the recursive call does its job and sorts the two sublists. Then we are left to merge the two sorted sublists by calling themergefunction. The base case for the recursive function is when the sublist size is 1. At that point we have a sorted sublist.
In Fig.4.5the entire left half of the list has been sorted. In addition, three sublists in the right half have been sorted and the third and fourth sublist are in the process of being merged together. The green dots represent the portions of the sequence that are sorted and the black dots indicate the unsorted portion of the original sequence. The red lines at the bottom reflect the recursive calls that are currently on the run-time stack. The length of the red line shows the portion of the sequence that is being sorted by its corresponding recursive call. The blue line underscores the two sublists currently being merged together.
The argument that merge sort runs in O(n log n) time needs just a bit of explanation.
The repetitive splitting of the list results in O(log n) splits. In the end we have lists of size 1. If we were to count every merge that occurs there would be n/2 merges at the bottom, followed by n/4 merges at the next level, and so on leading to this sum.
The number of merges≥
log2n i=0
2i =2log2n−1≈n
This analysis would seem to suggest that the complexity of the merge sort algorithm is O(n2) since there are roughlynmerges each of which is O(n) itself. However, the
108 4 Sequences
Fig. 4.5 Merge Sort Snapshot
Fig. 4.6 Merge Sort Merges
algorithm is not O(n2). To see why, consider sorting the list [5 8 2 6 9 1 0 7]. After repeatedly splitting the lists we get down to lists of size one as depicted in the first list of Fig.4.6. The individual items are merged two at a time to form sorted lists of two, shown in the second list of items. While there are four merges that take place at the lowest level of the merge sort, the four merges are each for lists of two elements (i.e. one from each list) and together they form a list ofnitems. So we can group all these four merges together to find that all the merges at that deepest level take O(n) time. Not each, but all of the merges at the deepest level when combined are O(n).
In the second version of the list, two merges are done for the lists of length two.
However, each merge is done on one half the list. The purple half is one merge, the green half includes the items that are in the second merge. Together, these two merges include alln items again. So, at the second deepest level again at mostn items are merged in O(n) time.
Finally, the last merge is of all the items in yellow from the two sorted sublists.
This merge also takes O(n) time since it merges all the items in the list, resulting in the sorted list seen in the last version of the list.
So, while merging is a O(n) operation, the merges take place on sublists of then items in the list which means that we can count the merging at each level as O(n) and don’t have to count each individual merge operation as O(n). Since there arelog n levels to the merge sort algorithm and each level takes O(n) to merge, the algorithm is O(n log n).