Design and Analysis Algorithm. Ahmad Afif Supianto, S.Si., M.Kom. Pertemuan 04

Design

and

Analysis

Algorithm

Ahmad Afif Supianto, S.Si., M.Kom

Contents

Brute Force Algorithm 1 2

Asymptotic Analysis 3

1

Asymptotic Notation



Think of n as the number of records we wish to sort with an algorithm that takes f(n) to run. How long will it take to sort n records?



What if n is big?



We are interested in the range of a function as n gets large.



Will f stay bounded?



Will f grow linearly?



Will f grow exponentially?



Our goal is to find out just how fast f grows with respect to n.

Asymptotic Notation

 Misalkan:

T(n) = 5n2 + 6n + 25

T(n) proporsional untuk ordo n2 _{untuk data yang sangat}

besar.

5

Memperkirakan formula untuk run-time

Indikasi kinerja algoritma

Asymptotic Notation



Indikator efisiensi algoritma bedasar pada OoG pada basic operation suatu algoritma.



Penggunaan notasi sebagai pembanding urutan OoG:

 O (big oh)

 Ω (big omega)

 Ө (big theta)



t(n) : algoritma running time (diindikasikan dengan basic operation count (C(n))



g(n) : simple function to compare the count 6

Classifying functions by their

Asymptotic Growth Rates (1/2)



asymptotic growth rate, asymptotic order, or order of functions

 Comparing and classifying functions that ignores

constant factors and small inputs.



O(g(n)), Big-Oh of g of n, the Asymptotic Upper Bound;



W(g(n)), Omega of g of n, the Asymptotic Lower Bound.



Q(g(n)), Theta of g of n, the Asymptotic Tight Bound; and

Example



Example: f(n) = n2 - 5n + 13.



The constant 13 doesn't change as n grows, so it is not crucial. The low order term, -5n, doesn't have much effect on f compared to the quadratic term, n2.

We will show that f(n) = Q(n2) .



Q: What does it mean to say f(n) = Q(g(n)) ?



A: Intuitively, it means that function f is the same order of magnitude as g.

Example (cont.)



Q: What does it mean to say f₁(n) = Q(1)?



A: f₁(n) = Q(1) means after a few n, f₁ is bounded above & below by a constant.



Q: What does it mean to say f₂(n) = Q(n log n)?



A: f₂(n) = Q(n log n) means that after a few n, f₂ is bounded above and below by a constant

times n log n. In other words, f₂ is the same order of magnitude as n log n.



More generally, f(n) = Q(g(n)) means that f(n) is a member of Q(g(n)) where Q(g(n)) is a set of functions of the same order of magnitude.

Big-Oh



The O symbol was introduced in 1927 to

indicate relative growth of two functions based on asymptotic behavior of the functions now used to classify functions and families of

Upper Bound Notation



We say Insertion Sort’s run time is O(n2₎

 Properly we should say run time is in O(n2₎

 Read O as “Big-O” (you’ll also hear it as “order”)



In general a function

 f(n) is O(g(n)) if  positive constants c and n₀ such

that f(n)  c  g(n)  n  n₀



e.g. if f(n)=1000n and g(n)=n2, n₀ > 1000 and c = 1 then f(n₀) < 1.g(n₀) and we say that f(n) = O(g(n))

Asymptotic Upper Bound

f(n) g(n) c g(n) • f(n)  c g(n) for all n  n₀ • g(n) is called an

asymptotic upper bound of f(n). • We write f(n)=O(g(n))

• It reads f(n) is big oh of g(n).

Big-Oh,

the Asymptotic Upper Bound



This is the most popular notation for run time

since we're usually looking for worst case time.



If Running Time of Algorithm X is O(n2) , then for any input the running time of algorithm X is at most a quadratic function, for sufficiently

large n.



e.g. 2n2 = O(n3) .



From the definition using c = 1 and n₀ = 2. O(n2) is tighter than O(n3).

6

g(n)

f(n) for all n>6, g(n) > 1 f(n).

Thus the function f is in the big-O of g.

that is, f(n) in O(g(n)).

g(n)

f(n)

5

There exists a n₀ s.t. for all n>n₀, f(n) < 1 g(n).

Thus, f(n) is in O(g(n)).

There exists a n₀=5, c=3.5, s.t. for all n>n₀, f(n) < c h(n). Thus, f(n) is in O(h(n)). 5 h(n) f(n) 3.5 h(n)

Example 3

Example of Asymptotic Upper Bound f(n)=3n2₊₅ g(n)=n2 4g(n)=4n2 4 g(n) = 4n2 = 3n2 _{+ n}2  3n2 _{+ 9 for all n}  ₃ > 3n2 + 5 = f(n) Thus, f(n)=O(g(n)). 3

Exercise on O-notation



Show that 3n2+2n+5 = O(n2)

10 n2 _{= 3n}2 _{+ 2n}2 _{+ 5n}2

 3n2 _{+ 2n + 5 for n}  ₁

Exercise on O-notation



f1(n) = 10 n + 25 n2



f2(n) = 20 n log n + 5 n



f3(n) = 12 n log n + 0.05 n2



f4(n) = n1/2 _{+ 3 n log n} • O(n2₎ • O(n log n) • O(n2₎ • O(n log n)

Classification of Function : BIG

O (1/2)

 A function f(n) is said to be of at most logarithmic growth if f(n) = O(log n)

 A function f(n) is said to be of at most quadratic growth if f(n) = O(n2)

 A function f(n) is said to be of at most polynomial

growth if f(n) = O(nk_{), for some natural number k > 1}

 A function f(n) is said to be of at most exponential growth if there is a constant c, such that f(n) = O(cn_),

and c > 1

 A function f(n) is said to be of at most factorial growth if f(n) = O(n!).

Classification of Function :

BIG O (2/2)



A function f(n) is said to have constant running time if the size of the input n has no effect on the running time of the algorithm (e.g.,

assignment of a value to a variable). The equation for this algorithm is f(n) = c



Other logarithmic classifications:

 f(n) = O(n log n)

Lower Bound Notation



We say InsertionSort’s run time is W(n)



In general a function

 f(n) is W(g(n)) if  positive constants c and n₀ such

that 0  cg(n)  f(n)  n  n₀



Proof:

 Suppose run time is an + b

• Assume a and b are positive (what if b is negative?)  an  an + b

Big

W

Asymptotic Lower Bound

f(n)

c g(n)

• f(n)  c g(n) for all n  n₀ • g(n) is called an

asymptotic lower bound of f(n). • We write f(n)=W(g(n))

• It reads f(n) is omega of g(n).

Example of Asymptotic Lower Bound f(n)=n2_/2-7 c g(n)=n2_/4 g(n)=n2 g(n)/4 = n2_/4 = n2/2 – n2_/4  n2/2 – 9 for all n  ₆ < n2/2 – 7 Thus, f(n)= W(g(n)). 6 g(n)=n2

Example: Big Omega



Example: n 1/2 = W( log n) .

Use the definition with c = 1 and n₀ = 16. Checks OK.

Let n > 16 : n 1/2 (1) log n if and only if n = ( log n )2 by squaring both sides.

Big Theta Notation



Definition: Two functions f and g are said to be of equal growth, f = Big Theta(g) if and only if

both

f=Q(g) and g = Q(f).



Definition: f(n) = O(g(n)) means  positive constants c₁, c₂, and n₀ such that

c₁ g(n)  f(n)  c₂ g(n)  n  n₀

 If f(n) = O(g(n)) and f(n) = W(g(n)) then f(n) = Q(g(n))

Theta, the Asymptotic Tight

Bound



Theta means that f is bounded above and below by g; BigTheta implies the "best fit".



f(n) does not have to be linear itself in order to be of linear growth; it just has to be between two linear functions,

Asymptotically Tight Bound

f(n)

c₁ g(n)

• f(n) = O(g(n)) and f(n) = W(g(n)) • g(n) is called an

asymptotically tight bound of f(n). • We write f(n)=Q(g(n))

• It reads f(n) is theta of g(n).

n₀

Other Asymptotic Notations



A function f(n) is o(g(n)) if  positive constants c and n₀ such that

f(n) < c g(n)  n  n₀



A function f(n) is (g(n)) if  positive constants c and n₀ such that

c g(n) < f(n)  n  n₀



Intuitively, – o() is like < – O() is like  – () is like > – W() is like  – Q() is like =

Examples

1. 2n3 _{+ 3n}2 _{+ n = 2n}3 _{+ 3n}2 _{+ O(n)} = 2n3 + O( n2 + n) = 2n3 + O( n2 ) = O(n3 ) = O(n4) 2. 2n3 _{+ 3n}2 _{+ n = 2n}3 _{+ 3n}2 _{+ O(n)} = 2n3 + Q(n2 + n) = 2n3 + Q(n2) = Q(n3)

Examples (cont.)

3. Suppose a program P is O(n3_{), and a program Q} is O(3n_{), and that currently both can solve}

problems of size 50 in 1 hour. If the programs are run on another system that executes exactly 729 times as fast as the original system, what size

Example (cont.)

n3 = 503 * 729 3n = 350 * 729 n = n = log₃ (729 * 350) n = n = log₃(729) + log₃ 350 n = 50 * 9 n = 6 + log₃ 350 n = 50 * 9 = 450 n = 6 + 50 = 56

 Improvement: problem size increased by 9 times for n3 algorithm but only a slight improvement in problem size (+6) for exponential algorithm.

3 3 729 * 50 3 3 3 729 50

More Examples

(a) 0.5n2 - 5n + 2 = W( n2). Let c = 0.25 and n₀ = 25. 0.5 n2 _{- 5n + 2 = 0.25( n}2_{) for all n = 25} (b) 0.5 n2 - 5n + 2 = O( n2). Let c = 0.5 and n₀ = 1. 0.5( n2_{) = 0.5 n}2 _{- 5n + 2 for all n = 1} (c) 0.5 n2 - 5n + 2 = Q( n2) from (a) and (b) above.

More Examples

(d) 6 * 2n _{+ n}2 _{= O(2}n).

Let c = 7 and n₀ = 4.

Note that 2n _{= n}2 _{for n = 4. Not a tight upper bound, but}

it's true.

(e) 10 n2 _{+ 2 = O(n}4_).

There's nothing wrong with this, but usually we try to get the closest g(n). Better is to use O(n2 ).

How to show a function f(n) is in/not

in O(g(n)).

f(n) is in O(g(n)):

find a constant c and large n₀ such that for all n > n₀, f(n) < c g(n).

f(n) is not in O(g(n)):

for any constant c and any large n₀, we can find a m such that

f(m) > c g(m).

Usually it is more difficult to proof that

a function f(n) is not in the big-O of another function g(n).

c,n₀ n>n₀ s.t. f(n)< c g(n)

Big O Again!!!!



O(1) The cost of applying the algorithm can be bounded independently of the value of n. This is called constant complexity.



O(log n) The cost of applying the algorithm to problems of sufficiently large size n can be bounded by a function of the form k log n, where k is a fixed constant. This is called

logarithmic complexity.



O(n) linear complexity



O(n log n) n lg n complexity

Big O Again!!!!



O(n3) cubic complexity



O(n4) quartic complexity



O(n32) polynomial complexity



O(cn_{) If constant c 1, then this is called}

exponential complexity



O(2n) _{exponential complexity}



O(en) exponential complexity



O(n!) factorial complexity

Practical Complexity t < 250

0 250 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

Practical Complexity t < 500

0 500 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

Practical Complexity t < 1000

0 1000 1 3 5 7 9 11 13 15 17 19 f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

Practical Complexity t < 5000

0 1000 2000 3000 4000 5000 1 3 5 7 9 11 13 15 17 19 f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n

Definisi Brute Force



Brute force : pendekatan straight forward untuk memecahkan suatu masalah



Algoritma brute force memecahkan masalah dengan sangat sederhana, langsung, danjelas (obvious way)

Contoh-contoh

(Berdasarkan pernyataan masalah)



Mencari elemen terbesar (terkecil)



Persoalan: Diberikan sebuah array yang

beranggotakan n buah bilangan bulat (a₁, a₂, ..., a_n).Carilah elemen terbesar di dalam array

tersebut.



Algoritma brute force: bandingkan setiap elemen array untuk menemukan elemen terbesar

Contoh-contoh

(Berdasarkan pernyataan masalah)

Contoh-contoh

(Berdasarkan pernyataan masalah)



Pencarian beruntun (Sequential Search)



Persoalan: Diberikan array yang berisi n buah bilangan bulat (a₁, a₂, ..., a_n). Carilah nilai x di dalam array tersebut. Jika x ditemukan, maka keluarannya adalah indeks elemen array, jika x tidak ditemukan, maka keluarannya adalah 0.



Algoritma brute force (sequential serach): setiap elemen array dibandingkan dengan x.

Pencarian selesai jika x ditemukan atau elemen array sudah habis diperiksa.

Contoh-contoh

(Berdasarkan pernyataan masalah)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)



Menghitung an (a > 0, n adalah bilangan bulat tak-negatif)



Definisi: an= a x a x ... x a (n kali) , jika n>0 = 1, jika n = 0



Algoritma brute force: kalikan 1 dengan a sebanyak n kali

Contoh-contoh (Berdasarkan definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)



Menghitung n! (n bilangan bulat tak-negatif) Definisi: n!=1 ×2×3×...×n, jika n>0

= 1, jika n = 0



Algoritma brute force: kalikan n buah bilangan, yaitu 1, 2, 3, ..., n, bersama-sama

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)



Mengalikan dua buah matriks, A dan B Definisi: Misalkan C = A × B dan

elemen-elemen matrik dinyatakan sebagai c_ij, a_ij, dan b_ij



Algoritma brute force: hitung setiap elemen hasil perkalian satu per satu, dengan cara

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)



Menemukan semua faktor dari bilangan bulat n (selain dari 1 dan n itu sendiri).

Definisi: Bilangan bulat a adalah faktor dari bilangan bulat b jika a habis membagi b.



Algoritma brute force: bagi n denga n setiap i = 2, 3, ..., n – 1. Jika n habis membagi i, maka I adalah faktor dari n.

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)



Uji keprimaan Persoalan: Diberikan sebuah bilangan bilangan bulat positif. Ujilah apakah bilangan tersebut merupakan bilangan prima atau bukan.

Definisi: bilangan prima adalah bilangan yang hanya habis dibagi oleh 1 dan dirinya sendiri. Algoritma brute force: bagi n dengan 2 sampai n–1. Jika semuanya tidak habis membagi n, maka n adalah bilangan prima



Perbaikan: cukup membagi dengan 2 sampai √n saja

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)



Algoritma Pengurutan Brute Force, Algoritma apa yang paling lempang dalam memecahkan masalah pengurutan?

Bubble sort dan selection sort!



Kedua algoritma ini memperlihatkan teknik brute force dengan jelas sekali.

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Bubble Sort

Mulai dari elemen ke-n:



1. Jika s_n< s_n-1, pertukarkan



2. Jika s_n-1 < s_n-2, pertukarkan ...



3. Jika s₂< s₁, pertukarkan 1 kali pass

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Selection Sort Pass ke –1:



1.Cari elemen terbesar mulai di dalam s[1..n]



2.Letakkan elemen terbesar pada posisi n (pertukaran)

Pass ke-2:



1.Cari elemen terbesar mulai di dalam s[1..n - 1]



2.Letakkan elemen terbesar pada posisi n - 1 (pertukaran)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Mengevaluasi polinom



Persoalan: Hitung nilai polinom

p(x)=a_nxn +a_n-1xn-1 +...+a₁x +a₀ untuk x = t.



Algoritma brute force: x_i dihitung secara brute force (seperti perhitungan a_n). Kalikan nilai xi dengan a_i, lalu jumlahkan dengan suku-suku lainnya.

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Contoh-contoh (Berdasarkan

definisi konsep yang terlibat)

Karakteristik Algoritma Brute Force



Algoritma brute force umumnya tidak “cerdas” dan tidak cepat, karena ia membutuhkan jumlah langkah yang besar dalam penyelesaiannya.



Kata “force” mengindikasikan “tenaga” ketimbang “otak”



Kadang-kadang algoritma brute force disebut juga algoritma naif (naïve algorithm).

Karakteristik Algoritma Brute Force



Algoritma brute force lebih cocok untuk masalah yang berukuran kecil.



Pertimbangannya:

 sederhana,

 Implementasinya mudah



Algoritma brute force sering digunakan sebagai basis pembanding dengan algoritma yang lebih cepat.

Karakteristik Algoritma Brute Force



Meskipun bukan metode yang cepat, hampir semua masalah dapat diselesaikan dengan algoritma brute force.



Sukar menunjukkan masalah yang tidak dapat diselesaikan dengan metode brute force.

Bahkan, ada masalah yang hanya dapat diselesaikan dengan metode brute force.



Contoh: mencari elemen terbesar di dalam array.

Tugas

Buatlah algoritma berikut dengan kompleksitasnya



Pencocokan String (String Matching)



Mencari pasangan titik yang jaraknya terdekat (Closest pairs)



Travelling Salesman Problem (TSP)