Divide and Conquer Algorithms

(1)

Divide and Conquer Algorithms

Antonio Carzaniga

Faculty of Informatics University of Lugano

(2)

Outline

Merging

Searching

Sorting

Multiplying

(3)

(4)

Merging

Given two sequencesA={a1,a2, . . . ,an}and

B={b1,b2, . . . ,bm}, output a sequenceX ={x1,x2, . . . ,xl} such that

◮ every element ofAappears once inX

◮ every element ofBappears once inX

(5)

Merging

◮ every element ofX appears inAor inBor in both

Example:

A={34,7,11,31,14,51,8,21,10} B={51,21,14,15,27,31,2}

(6)

Merging

◮ every element ofX appears inAor inBor in both

Example:

A={34,7,11,31,14,51,8,21,10} B={51,21,14,15,27,31,2}

(7)

A Simple Merge Algorithm

Algorithm strategy

◮ iterate through every positioni, first throughAand thenB

◮ outputa

iifaiis not in{a1,a2, . . . ,ai−1} ◮ outputb

(8)

A Simple Merge Algorithm

Algorithm strategy

◮ iterate through every positioni, first throughAand thenB

◮ outputa 4 thenoutputA[i]

5 fori ←1tolength(B)

6 do

(9)

Complexity

MERGESIMPLE(A,B) 1 fori ←1tolength(A)

2 do

3 if¬FIND(A[1. .i−1],A[i]) 4 thenoutputA[i]

6 do

(10)

Complexity

2 do

6 do

7 if¬FIND(A,B[i])∧ ¬FIND(B[1. .i−1],B[i]) 8 thenoutputB[i]

(11)

Complexity

2 do

6 do

(12)

Searching

(13)

Searching

Given a sequenceAand a valuekey, returnTRUEifAcontains key, orFALSE ifAdoes not contain a valuekey

FIND(A,key)

1 fori ←1tolength(A)

2 do

3 ifA[i] =key

(14)

Searching

FIND(A,key)

2 do

3 ifA[i] =key

4 then returnTRUE 5 returnFALSE

5 ifvalue(item) =key 6 then returnTRUE

7 i ←i+1

(15)

Searching

FIND(A,key)

2 do

3 ifA[i] =key

7 i ←i+1

8 item←next(item) 9 returnFALSE

(16)

Searching

FIND(A,key)

2 do

3 ifA[i] =key

7 i ←i+1

8 item←next(item) 9 returnFALSE

(17)

Complexity of

M

ERGE

S

IMPLE

2 do

6 do

T(N) = N

∑

i=1

(18)

Complexity of

M

ERGE

S

IMPLE

2 do

6 do

(19)

Complexity of

M

ERGE

S

IMPLE

2 do

6 do

(20)

Complexity of

M

ERGE

S

IMPLE

2 do

6 do

(21)

Searching (2)

(22)

Searching (2)

Given asortedsequenceAand a valuekey, return TRUEifA constains the valuekey, orFALSE otherwise

BINARYSEARCH(A,key) 1 first ←1

2 last ←length(A) 3 whilefirst ≤last

4 domiddle← ⌈(first+last)/2⌉

(23)

Binary Search

BINARYSEARCH(A,key) 1 first ←1

(24)

Binary Search

5 ifA[middle] =key 6 then returnTRUE 7 elseiffirst =last 8 then returnFALSE 9 elseifA[middle]>key 10 thenlast ←middle−1 11 else fisrt ←middle+1

12 returnFALSE ₁2

(25)

Binary Search

(26)

Binary Search

(27)

Binary Search

(28)

Binary Search

(29)

Binary Search

(30)

Binary Search

(31)

Binary Search

(32)

Binary Search

(33)

Binary Search

(34)

Binary Search

(35)

Merging Sorted Sequences

A slightly different problem:

Given twosortedsequencesA={a1,a2, . . . ,an}and B={b1,b2, . . . ,bm}, wherea1≤a2≤. . . ≤an}and

b1≤b2≤ . . .≤bm, output a sequenceX ={x1,x2, . . . ,xl}such that

(36)

A Better Merge Algorithm

MERGESIMPLE2(A,B) 1 fori ←1tolength(A)

2 do

3 if¬BINARYSEARCH(A[1. .i−1],A[i]) 4 thenoutputA[i]

6 do

7 if¬BINARYSEARCH(A,B[i])

(37)

A Better Merge Algorithm

2 do

6 do

8 ∧¬BINARYSEARCH(B[1. .i−1],B[i]) 9 thenoutputB[i]

T(N) = N

∑

i=1

(38)

A Better Merge Algorithm

2 do

6 do

T(N) = N

∑

i=1

(39)

A Better Merge Algorithm

2 do

6 do

(40)

An Even Better Merge Algorithm

Intuition: AandBaresorted e.g.

(41)

An Even Better Merge Algorithm

A={3,7,12,13,34,37,70,75,80} B={1,5,6,7,34,35,40,41,43}

(42)

An Even Better Merge Algorithm

A={3,7,12,13,34,37,70,75,80} B={1,5,6,7,34,35,40,41,43}

so just like in BINARYSEARCHI can avoid looking for an element ofx if thefirstelement I see isy >x

High-level algorithm strategy

◮ step through every positioni ofAand every positionjofB; at

(43)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

(44)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=1

j=1

(45)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=1

j=1

(46)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=1

j=2

(47)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=1

j=2

(48)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=2

j=2

(49)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=2

j=2

(50)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=2

j=3

(51)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=2

j=3

(52)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=2

j=4

(53)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=2

j=4

(54)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=3

j=5

(55)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=3

j=5

(56)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=4

j=5

(57)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=4

j=5

(58)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=5

j=5

(59)

Algorithm

A

B

3 7 12 13 34 37 70 75 80

1 5 6 7 34 35 40 41 43

i=5

j=5

(60)

(61)

Algorithm (2)

(62)

(63)

Complexity of

M

ERGE

MERGE(A,B) 1 i,j←1 2 X ← ∅

3 whilei ≤length(A)∨j ≤length(B)

4 do ifi ≤length(A)∧(j>length(B)∨A[i]<B[j]) 5 thenX ←X◦A[i]

6 i ←i+1

7 else X ←X◦B[j]

8 j ←j+1

(64)

Complexity of

M

ERGE

MERGE(A,B) 1 i,j←1 2 X ← ∅

6 i ←i+1

7 else X ←X◦B[j]

8 j ←j+1

(65)

Complexity of

M

ERGE

MERGE(A,B) 1 i,j←1 2 X ← ∅

6 i ←i+1

7 else X ←X◦B[j]

8 j ←j+1

9 returnX

(66)

Complexity of

M

ERGE

MERGE(A,B) 1 i,j←1 2 X ← ∅

6 i ←i+1

7 else X ←X◦B[j]

8 j ←j+1

9 returnX

T(N) = Θ(N)

(67)

Complexity of

M

ERGE

MERGE(A,B) 1 i,j←1 2 X ← ∅

6 i ←i+1

7 else X ←X◦B[j]

8 j ←j+1

9 returnX

T(N) = Θ(N)

(68)

Complexity of

M

ERGE

Can we do better? No!

(69)

Using

M

ERGE

So now we have alinear-complexitymerge procedure ◮ merges twosortedsequences

(70)

Using

M

ERGE

◮ produces a sorted sequence

(71)

Using

M

ERGE

We could use that to implement a sort algorithm

Idea

◮ assume that two parts,A

L andAR, of the input sequenceAhave

(72)

Using

M

ERGE

Idea

already been sorted

◮ then you can use M_ERGEto combine them to obtain the final

(73)

Using

M

ERGE

Idea

already been sorted

◮ then you can use M_ERGEto combine them to obtain the final

result

(74)

(75)

Merge Sort

MERGESORT(A) 1 iflength(A) =1 2 then returnA 3 m← ⌊length(A)/2⌋

4 AL← MERGESORT(A[1. . .m])

(76)

Merge Sort

5 AR ←MERGESORT(A[m+1. . .length(A)]) 6 returnMERGE(AL,AR)

(77)

Merge Sort

The complexity of MERGESORTis

(78)

Merge Sort

The complexity of MERGESORTis

T(N) =2T(N/2) +O(N)

(79)

Divide and Conquer

(80)

Divide and Conquer

MERGESORTexemplifies thedivide and conquerstrategy

General strategy: given a problemP on input dataA ◮ dividethe inputAinto partsA

1,A2, . . . ,Ak with|Ai|<|A|=N ◮ solveproblemPfor the individualk parts

(81)

Divide and Conquer

MERGESORTexemplifies thedivide and conquerstrategy

General strategy: given a problemP on input dataA ◮ dividethe inputAinto partsA

1,A2, . . . ,Ak with|Ai|<|A|=N ◮ solveproblemPfor the individualk parts

◮ combinethe partial solutions to obtain the solution forA

Complexity analysis

(82)

(83)

A Divide-and-Conquer Merge

MERGER(A,B) 1 iflength(A) =0 2 then returnB 3 iflength(B) =0 4 then returnA 5 ifA[1]<B[1]

(84)

A Divide-and-Conquer Merge

(85)

A Divide-and-Conquer Merge

6 then returnA[1]◦MERGER(A[2. . .length(A)],B) 7 else returnB[1]◦MERGER(A,B[2. . .length(B)]) ◮ again, this algorithm is a bit incorrect (Exercise: Fix it.)

(86)

A Divide-and-Conquer Merge

The complexity of MERGER is

(87)

A Divide-and-Conquer Merge

(88)

A Divide-and-Conquer Merge

T(N) =C1+T(N−1) =C1N=O(N)

(89)

A Divide-and-Conquer Merge

(90)

(91)

Divide-and-Conquer Multiplication

(92)

Divide-and-Conquer Multiplication

Going back to multiplication. . .

(93)

Divide-and-Conquer Multiplication

x = XL XR and y = YL YR

which meansx =2ℓ/2_x

L+xRandy =2ℓ/2yL+yR, so. . .

xy = (2ℓ/2xL+xR)(2ℓ/2yL+yR)

=2ℓxLyL+2ℓ/2(xLyR+xRyL) +xRyR

(94)

Divide-and-Conquer Multiplication

we reduced the problem of multiplying two numbers ofℓbits into the problem of multiplyingfournumbers ofℓ/2 bits. . .

(95)

Divide-and-Conquer Multiplication

we reduced the problem of multiplying two numbers ofℓbits into the problem of multiplyingfournumbers ofℓ/2 bits. . .

T(ℓ) =4T(ℓ/2) +O(ℓ)

(96)

(97)

Divide-and-Conquer Multiplication (2)

Again, we have

(98)

Divide-and-Conquer Multiplication (2)

Again, we have

(99)

Divide-and-Conquer Multiplication (2)

Again, we have

but notice thatxLyR+xRyL= (xL+xR)(yR+yL)−xLyL−xRyR, so

(100)

Divide-and-Conquer Multiplication (2)

Again, we have

xy =2ℓxLyL+2ℓ/2((xL+xR)(yR+yL)−xLyL−xRyR) +xRyR

(101)

Divide-and-Conquer Multiplication (2)

Again, we have

Only 3 multiplications: xLyL,(xL+xR)(yR+yL), andxRyR

(102)

Divide-and-Conquer Multiplication (2)

Again, we have

Only 3 multiplications: xLyL,(xL+xR)(yR+yL), andxRyR

T(ℓ) =3T(ℓ/2) +O(ℓ)

which, as we will see, leads to a much better complexity

T(ℓ) =O(ℓlog23_{) =}_O(_ℓ1.59

(103)

Computing the Median

Themedianof a sequenceAis a valuem∈Asuch that half the values inAare smaller thanmand half are bigger thanm

◮ e.g., what is the median ofA={2

(104)

Computing the Median

,36,5,21,8,13,11,20,5,4,1}?

Idea: first sort, then pick the element in the middle SIMPLEMEDIAN(A)

(105)

Computing the Median

,36,5,21,8,13,11,20,5,4,1}?

1 X ←MERGESORT(A) 2 returnX[⌊length(A)/2⌋]

(106)

Computing the Median

,36,5,21,8,13,11,20,5,4,1}?

(107)

Computing the Median

,36,5,21,8,13,11,20,5,4,1}?

Is it correct? Yes

(108)

Computing the Median

,36,5,21,8,13,11,20,5,4,1}?

Is it correct? Yes

(109)

Computing the Median

,36,5,21,8,13,11,20,5,4,1}?

Is it correct? Yes

How long does it take? T(N) =O(NlogN)

(110)

Computing the Median

,36,5,21,8,13,11,20,5,4,1}?

Is it correct? Yes

How long does it take? T(N) =O(NlogN)

(111)

Computing the Median (2)

(112)

Computing the Median (2)

(113)

Computing the Median (2)

Generalizating, thek-smallestelement of a sequenceAis a valuev ∈Asuch that |A| −k elements ofAare greater or equal tov

E.g.,

(114)

Computing the Median (2)

E.g.,

◮ fork =1, the minimum ofA

(115)

Computing the Median (2)

E.g.,

◮ fork =⌊|A|/2⌋, the median ofA ◮ what is the6th smallestelement of

(116)

Computing the Median (2)

E.g.,

◮ fork =⌊|A|/2⌋, the median ofA ◮ what is the6th smallestelement of

A={2,36,5,21,8,13,11,20,5,4,1}?

(117)

(118)

k-Smallest Element

Idea: we split the sequenceAin three parts based on achosen value v ∈A

◮ A

Lcontains the set of elements that areless than v

◮ A

vcontains the set of elements that areequal to v

◮ A

(119)

k-Smallest Element

◮ A

Rcontains the set of elements that aregreater then v

(120)

k-Smallest Element

◮ A

(121)

k-Smallest Element

◮ A

E.g.,A={2,36,5,21,8,13,11,20,5,4,1} and we must compute the 7th smallest value inA we pick a splitting value, sayv =5

(122)

k-Smallest Element

◮ A

(123)

k-Smallest Element

◮ A

(124)

k-Smallest Element

◮ A

AL={2,4,1} Av ={5,5} AR ={36,21,8,13,11,20}

(125)

k-Smallest Element

◮ A

AL={2,4,1} Av ={5,5} AR ={36,21,8,13,11,20}

(126)

k-Smallest Element (2)

We useselect(A,k)to denote the k-smallest element ofA

select(A,k) =



 

 

select(AL,k) ifk ≤ |AL|

(127)

k-Smallest Element (2)

select(A,k) =



 

 

v if|AL|<k ≤ |AL|+|Av| select(AR,k− |AL| − |Av|) ifk >|AL|+|Av|

(128)

k-Smallest Element (2)

select(A,k) =



 

 

v if|AL|<k ≤ |AL|+|Av| select(AR,k− |AL| − |Av|) ifk >|AL|+|Av|

ComputingAL,Av, andARtakesO(N)steps

(129)

k-Smallest Element (2)

(130)

k-Smallest Element (2)

select(A,k) =

(131)