Design and Analysis of Algorithms

(1)

Design and Analysis of Algorithms

http://www.cp.eng.chula.ac.th/faculty/spj

Introduction

Computational problems Algorithms

Algorithm Design Techniques Analysis of Algorithms Examples

Three Main Topics

Algorithm Design Algorithm Analysis Computation Complexity

Design and Analysis Steps

?

!" !

N

Y

N

Y N

Design and Analysis Steps

?

!" !

N

Y

N

Y N

Computational Problems

A computational (or algorithmic) problem is specified by a precise definition of

the legal inputs

the required outputs as a function of those inputs

(2)

Sorting Problem

Sorting problem instances : < 31, 41, 59, 26, 41 >

<1, 2, 5, 7, 6, 9, 2>

<2, 2, 2>

Input : a sequence of n numbers < a 1 , a 2 , ..., a n >

Output : a reordering < a 1 , a 2 , ..., a n > of the input such that a 1 a 2 ... a n

Problem Instance Input Instance Instance

Sorting

Input Output

Median and Selection

Input Output

k = 10

Minimum Spanning Tree

Input Output

Steiner Tree

Input Output

Bandwidth Minimization

1 2 3 4 5 6 7 7 1 6 2 5 3 4

Input Output

(3)

Traveling Salesman Problem

Input Output

Convex Hull

Input Output

Bin Packing

Input Output

Primality Testing

Input Output

8338169264555846052842102071 NO

Factoring

Input Output

8338169264555846052842102071

179424673 x 2038074743 x 22801763489

= 8338169264555846052842102071

String Matching

Input Output

You are the fairest of your sex, Let me be your hero;

I love you as one over x , As x approaches zero.

Positively.

you

You are the fairest of you r sex, Let me be you r hero;

Positively.

(4)

Approximate String Matching

Input Output

Positively.

heero

You are the fairest of your sex, Let me be your hero ; I love you as one over x , As x approaches zero . Positively.

Data Compression

Input Output

Positively.

I love you as one over x, As x approaches zero.

Positively.

Cryptography

Input Output

Positively.

)*#$(*KJSNpsld09LKDF0osk POS)s8sd,??<CNZisd(&sD(6%

(^%#&(dls28s8&AK)8dksF_8 OSD7slx.zz846(&%}{lps ska@

Satisfiability

Input Output

( x + y ) ( x + y ) x NO

Halting

Input Output

while x 1 do ?

if x is even then x = x/2 else x = 3x+1

x = 7

?

!" !

N

Y

N

Y N

!" !

(5)

Algorithms

A sequence of computational steps that

transform any legal input into the desired output A tool for solving a well-specified

computational problem

Idea behinds a computer program

Sequential Search

SeqSearch( D[1..n], x ) {

i = 1

while ( i <= n && D[i] != x ) i = i + 1

if ( i > n ) return 0 return i

}

Sequential Search

i = 1

while ( i <= n && D[i] != x ) i = i + 1

if ( i > n ) return 0 return i

}

Search forward

Sequential Search

i = n

while ( i >= 1 && D[i] != x ) i = i - 1

if ( i < 1 ) return 0 return i

}

Search backward

Sequential Search

i = n; D[0] = x while ( D[i] != x ) i = i - 1

return i }

Search backward with sentinel

Algorithms

An algorithms is correct if, for every input instance, it halts with the correct output.

We seek algorithms which are correct

efficient

(6)

Algorithm Design Techniques

Brute Force Incremental Divide and Conquer Dynamic Programming Greedy Algorithm Search and Traversal Probabilistic Algorithm Approximation Algorithm

Algorithm Design

?

!" !

N

Y

N

Y N Brute Force

Incremental Divide & Conquer Dynamic Prog.

Greedy Search & Traversal

Approximation Probabilistic !" !

?

!" !

N

Y

N

Y ? N

Analysis of Algorithms

ABCDEFGCDHAIJAKLMNONPQRNCAIS !

"!! memory $%&'(&! ! Worst case analysis

Average case analysis Amortized analysis

Algorithm Complexity

time

size of instance best avg.

worst

Algorithm Complexity

time

size of instance best avg.

worst

(7)

Sorting Algorithms

Bubble sort t(n) n 2 Insertion sort t(n) n 2 Shell sort t(n) n 1.xx Heap sort t(n) n log n Merge sort t(n) n log n

Quick sort t(n) n log n (average case)

?

!" !

N

Y

N

Y ? N

Computational Complexity

Problem reduction 0

0 A B

A B B A

0 0 2

=

A Small Problem

Input : a number n, k

Output : (the n th Fibonacci number ) mod k Input instance : 10, 21

Output : f 10 mod 21 = 55 mod 21 = 13

Fibonacci Number

n: 0 1 2 3 4 5 6 7 8 9 10 fn: 0 1 1 2 3 5 8 13 21 34 55

f n = f n-1 + f n-2 for n > 1 f 0 = 0, f 1 = 1

sss

n n

f

n

2 5 1 2

5 1 5 1

Fib1

Fib1( n, k ) {

t1 = (1 + sqrt(5))/2 t2 = (1 - sqrt(5))/2

f = int( (t1^n + t2^n)/sqrt(5) ) return f mod k

}

sss

n n

f

n

2 5 1 2

5

1

5

1

(8)

Fib1

Fib1( n, k ) {

t1 = (1 + sqrt(5))/2 t2 = (1 - sqrt(5))/2

f = int( (t1^n + t2^n)/sqrt(5) ) return f mod k

}

n : t ( n ) n n : t ( n ) ?

!"#$#!%&'(

Fib2

Fib2( n, k ) {

if ( n < 2 ) return n mod k return ( Fib2(n-1,k) + Fib2(n-2,k) ) mod k }

f n = f n-1 + f n-2 for n > 1 f 0 = 0, f 1 = 1

Fib2

Fib2( n, k ) {

if ( n < 2 ) return n mod k return ( Fib2(n-1,k) + Fib2(n-2,k) ) mod k }

t ( n ) = t ( n -1) + t ( n -2) + 1 t 0 = t 1 = 1

t ( n ) n

Fib2 on 1-MIPS Machine

n 10 20 30 50 100

Fib2 8ms 1s 2min 21days 10 9 yrs.

t(n) n , 1.618

Moores Law

MooreYs Law :

N[R 80 G\ ]D[ Pentium XXX 10 14 GHz.

(BC_R`BCabNOI[bIcQNO 10 17 BI`)

defBQ 10 9 yrs. BLabNE` Fn mod k BabN n = 181 ( t(n) n log 1.618 10 17 81 )

CN Pentium XXXX 10 1000 GHz !!!!!

Fib3

Fib3( n, k ) {

fn2 = 0 fn1 = 1 for i = 2 to n

fn = ( fn1 + fn2 ) mod k fn2 = fn1

fn1 = fn return fn }

f n = f n-1 + f n-2 for n > 1

f 0 = 0, f 1 = 1

(9)

Fib3

Fib3( n, k ) {

fn2 = 0 fn1 = 1 for i = 2 to n

fn = ( fn1 + fn2 ) mod k fn2 = fn1

fn1 = fn return fn

} t ( n ) n

Fib2 vs. Fib3

n 10 20 30 50 100

Fib2 8ms 1s 2min 21days 10 9 yrs.

Fib3 0.16ms 0.33ms 0.5ms 0.75ms 1.5ms

+ algorithm /+1 Fib 3 +

Conclusion

Chf]PRGijE

Chf]PRAJ[NNRkll

Chf]PRAJ[ABCDEF

mRSno Nph`I[bNPQRNCAIS