Cust. 1 : milk, bread, cereal. Cust. 2 : milk, bread, Sugar, eggs. Cust. 3 : milk, bread, butter

(1)

Mining Association Rules in

Large Databases

S1

S1 Teknik

Teknik Informatika

Informatika

Fakultas

Fakultas Teknologi

Teknologi Informasi

Informasi

Universitas

(2)

Agenda

Pendahuluan

Association Rule Mining

Market Basket Analysis

Basic Concept

Contoh

Algoritma Apriori

(3)

Pendahuluan

Asosiasi atau korelasi hubungan dalam

sekumpulan besar data.

Pendorong :

Jumlah data yang dikoleksi dan disimpan oleh

industri yang semakin besar.

Asosiasi hubungan yang ditemukan dapat

menolong dalam pembuatan keputusan bisnis

seperti desain katalog dan marketing

(4)

Market Basket Analysis

Cust. 1 : milk, bread, cereal

Cust. 2 : milk, bread, Cust. 2 : milk, bread,

Sugar, eggs

Cust. 3 : milk, bread, butter

(5)

Contoh Market Basket Analysis (MBA)

Sales manager, ingin mengetahui kebiasaan

membeli para customernya.

Pertanyaan : kelompok atau item mana saja

yang biasa dibeli oleh customer pada satu kali

waktu belanja ?

MBA : diterapkan pada data retail transaksi

customer.

Hasil dapat digunakan untuk merencanakan

strategi marketing atau advertising.

(6)

Contoh Penggunaan Hasil MBA

Untuk menentukan layout toko.

Item yang sering dibeli bersamaan

ditempatkan berdekatan.

Penggambaran pola dapat

direpresentasikan dengan

direpresentasikan dengan association

association

rule

rule..

(7)

Contoh Penggunaan Hasil MBA

Support dan confidence merupakan ukuran

tingkat kemenarikan.

Keduanya menunjukkan tingkat kegunaan dan

keyakinan dari rule yang ada.

Support 2% berarti 2% dari seluruh transaksi

yang dianalisis menunjukkan bahwa computer

Support 2% berarti 2% dari seluruh transaksi

yang dianalisis menunjukkan bahwa computer

dan software dibeli bersamaan.

Confidence 60% berarti 60% customer yang

membeli computer juga membeli software.

Pada umumnya, suatu association rule

dianggap menarik jika memenuhi minimum

support dan confidence threshold tertentu yang

diset oleh user / domain expert.

(8)

Basic concept : Istilah (1)

Support({A,B})

Support({A,B}) : banyaknya transaksi yang

: banyaknya transaksi yang

mengandung item A dan item B

Minimum Support :

Minimum Support : digunakan untuk

digunakan untuk

membatasi variasi itemset

(9)

Basic concept : Istilah(2)

Confidence

Confidence (A=>B)

(A=>B): probabilitas (B|A),

: probabilitas (B|A),

dihitung sebagai

}) sup({ }) , sup({ ) ( A B A B A confidence => =

Jika rule ini mempunyai confidence 0.33,

berarti jika A dan B terjadi dalam satu

transaksi, terdapat 33% kemungkinan

bahwa B juga terjadi.

(10)

Basic concept : Istilah (3)

K

K--itemset : suatu itemset yang berisi k

itemset : suatu itemset yang berisi k

item.

Ex. {computer,software} adalah 2

Ex. {computer,software} adalah 2--itemset.

itemset.

Frekuensi kejadian / occurrence frequency

of itemset : banyaknya transaksi yang

berisi itemset. Sering disebut juga sebagai

frequency, support count, atau count.

Rule disebut strong jika memenuhi min

support dan confidence threshold.

(11)

Basic Concept : Contoh

Total transaksi : 1000Total transaksi : 1000 Hammer : 50Hammer : 50 Nails : 80Nails : 80 Lumber : 20Lumber : 20

Hammer + nails : 15Hammer + nails : 15

Support u/ hammer, nails : Support u/ hammer, nails :

1,5% (15/1000) 1,5% (15/1000)

Support u/ hammer, nails, Support u/ hammer, nails,

lumber : 0,5% (5/1000) lumber : 0,5% (5/1000)

Confidence hammer Confidence hammer nails : nails :

Hammer + nails : 15Hammer + nails : 15

Nails + lumber : 10Nails + lumber : 10

Hammer + lumber : 10Hammer + lumber : 10

Hammer, nails, lumber : 5Hammer, nails, lumber : 5

Confidence hammer

Confidence hammer nails : nails : 30% (15/50)

30% (15/50)

Confidence nails Confidence nails hammer : hammer :

19% (15/80) 19% (15/80)

Confidence hammer, nails Confidence hammer, nails

lumber : 33% (5/15) lumber : 33% (5/15)

Confidence lumber Confidence lumber

hammer, nails : 25% (5/20) hammer, nails : 25% (5/20)

(12)

Step association rule mining

Temukan seluruh frequent itemset

Harus memenuhi minimum support yang telah

ditentukan sebelumnya.

Generate strong association rules dari

frequent itemset

(13)

Frequent Patterns &

Association Rules

Itemset X = {xItemset X = {x₁₁, …, x, …, x_k_k}}

Find all the rules Find all the rules X X YY with minimum with minimum

support and confidence support and confidence

supportsupport, , ss, , probabilityprobability that a that a

transaction contains X

transaction contains X ∪∪ _Y_Y

confidenceconfidence, , c,c, conditional conditional

Transaction

Transaction--idid Items boughtItems bought

10 10 A, B, DA, B, D 20 20 A, C, DA, C, D 30 30 A, D, EA, D, E 40 40 B, E, FB, E, F 50

50 B, C, D, E, FB, C, D, E, F confidenceconfidence, , c,c, conditional conditional

probability

probability that a transaction having that a transaction having X also contains

X also contains YY

Let sup_min = 50%, conf_min = 50% Jumlah Transaksi : 5

Freq. Pat.: {A:3, B:3, D:4, E:3, AD:3 } Association rules (support, confidence):

A D (60%, 100%) D A (60%, 75%) Customer buys A Customer buys both Customer buys D 50 50 B, C, D, E, FB, C, D, E, F }) , sup({ ) (A B A B confidence => =

(14)

Kekuatan dan Kelemahan AR

Kekuatan :

Result yang mudah dipahami dengan jelasResult yang mudah dipahami dengan jelas

Mendukung undirected DMMendukung undirected DM

Dapat dijalankan pada data yang variable lengthDapat dijalankan pada data yang variable length

Kelemahan :

Membutuhkan resource komputasi yang besarnya Membutuhkan resource komputasi yang besarnya

meningkat secara eksponensial sesuai problem yang meningkat secara eksponensial sesuai problem yang ada

ada

Cukup sulit untuk menentukan itemset yang benar Cukup sulit untuk menentukan itemset yang benar

untuk dianalisis untuk dianalisis

(15)

Algoritma Apriori & FP Growth

Algoritma apriori :

algoritma untuk menemukan frequent itemset

dengan candidate generation.

FP Growth :

metode mining frequent itemset tanpa

candidate generation.

(16)

Apriori: A Candidate Generation

Apriori: A Candidate Generation--and

and--Test Approach

Test Approach

Apriori pruning principleApriori pruning principle: : If there is If there is anyany itemset which is itemset which is

infrequent, its superset should not be generated/tested! infrequent, its superset should not be generated/tested! ((Agrawal & Srikant @VLDB’94, Mannila, et al. @ KDD’ 94)Agrawal & Srikant @VLDB’94, Mannila, et al. @ KDD’ 94)

Method: Method:

Initially,Initially, scan DB once to get frequent 1scan DB once to get frequent 1--itemsetitemset

Generate Generate length (k+1) candidate itemsets from length k length (k+1) candidate itemsets from length k

frequent itemsets frequent itemsets

(17)

The Apriori Algorithm

The Apriori Algorithm—

—An

An

Example

Database TDB 1st _scan C₁ L1 C Tid

Tid ItemsItems

10 10 A, C, DA, C, D 20 20 B, C, EB, C, E 30 30 A, B, C, EA, B, C, E 40 40 B, EB, E Itemset

Itemset supsup

{A} {A} 22 {B} {B} 33 {C} {C} 33 {D} {D} 11 {E} {E} 33 Itemset

{A} {A} 22 {B} {B} 33 {C} {C} 33 {E} {E} 33 Sup_min = 2 L₂ C₂ _C 2 2nd scan C₃ ₃rd _scan L₃ 40 40 B, EB, E Itemset Itemset {A, B} {A, B} {A, C} {A, C} {A, E} {A, E} {B, C} {B, C} {B, E} {B, E} {C, E} {C, E} Itemset

{A, B} {A, B} 11 {A, C} {A, C} 22 {A, E} {A, E} 11 {B, C} {B, C} 22 {B, E} {B, E} 33 {C, E} {C, E} 22 Itemset

{A, C} {A, C} 22 {B, C} {B, C} 22 {B, E} {B, E} 33 {C, E} {C, E} 22 Itemset

(18)

The Apriori Algorithm

Pseudo

Pseudo--code

code::

C

C_k_k: Candidate itemset of size k: Candidate itemset of size k

L

L_k_k : frequent itemset of size k: frequent itemset of size k

L

L₁₁ = {frequent items};= {frequent items};

for

for ((kk = 1; = 1; LL !=!=∅∅_;_{; k}_k++)₊₊₎_{do begin}_{do begin}

for

for ((kk = 1; = 1; LL_k_k !=!=∅∅_;_{; k}_k++)₊₊₎_{do begin}_{do begin}

C

C_k+1_k+1 = candidates generated from = candidates generated from LL_k_k;;

for each

for each transaction transaction tt in database doin database do

increment the count of all candidates in

increment the count of all candidates in CC_k+1_k+1

that are contained in that are contained in tt

L

(19)

Important Details of Apriori

How to generate candidates?How to generate candidates?

Step 1: selfStep 1: self--joining joining LL_k_k

Step 2: pruningStep 2: pruning

How to count supports of candidates?How to count supports of candidates?

Example of CandidateExample of Candidate--generationgeneration

LL₃₃=={{abc, abd, acd, ace, bcdabc, abd, acd, ace, bcd}}

SelfSelf--joining: joining: LL₃₃*L*L₃₃

•• abcd abcd from from abcabc and and abdabd

•• acdeacde from from acdacd and and aceace

Pruning:Pruning:

•• acdeacde is removed because is removed because adeade is not in is not in LL₃₃

(20)

Mining Frequent Patterns

Without Candidate Generation

Grow long patterns from short ones using

local frequent items

“abc” is a frequent pattern

Get all transactions having “abc”: DB|abc

“d” is a local frequent item in DB|abc

abcd is

a frequent pattern

(21)

Construct FP

Construct FP--tree from a Transaction

tree from a Transaction

Database

{}

Header Table

min_support = 3 TID Items bought (ordered) frequent items

100 {f, a, c, d, g, i, m, p} {f, c, a, m, p}

200 {a, b, c, f, l, m, o} {f, c, a, b, m}

300 {b, f, h, j, o, w} {f, b}

400 {b, c, k, s, p} {c, b, p}

500 {a, f, c, e, l, p, m, n} {f, c, a, m, p}

1. Scan DB once, find

f:4 c:1 b:1 p:1 b:1 c:3 a:3 b:1 m:2 Header Table

Item frequency head

f 4 c 4 a 3 b 3 m 3 p 3

1. Scan DB once, find frequent 1-itemset (single item pattern) 2. Sort frequent items in

frequency descending order, f-list

3. Scan DB again, construct FP-tree

(22)

Find Patterns Having P From

P

P--conditional Database

conditional Database

Starting at the frequent item header table in the FP

Starting at the frequent item header table in the

FP--tree

tree

Traverse the FP

Traverse the FP--tree by following the link of each

tree by following the link of each

frequent item

frequent item p

p

Accumulate all of

Accumulate all of transformed prefix paths

transformed prefix paths

of item

of item p

p to

to

form

form p’

p’s conditional pattern base

s conditional pattern base

{}

Conditional pattern bases item cond. pattern base

c f:3 a fc:3 b fca:1, f:1, c:1 {} f:4 c:1 b:1 p:1 b:1 c:3 a:3 Header Table

f 4

c 4

a 3

b 3

(23)

From Conditional Pattern

From Conditional Pattern--bases to

bases to

Conditional FP

Conditional FP--trees

trees

For each pattern

For each pattern--base

base

Accumulate the count for each item in the baseAccumulate the count for each item in the base

Construct the FPConstruct the FP--tree for the frequent items of the tree for the frequent items of the

pattern base pattern base

m-conditional pattern base: fca:2, fcab:1 {} f:3 c:3 a:3 All frequent patterns relate to m m, fm, cm, am, fcm, fam, cam, fcam {} f:4 c:1 b:1 p:1 b:1 c:3 a:3 b:1 m:2 p:2 m:1 Header Table

f 4 c 4 a 3 b 3 m 3 p 3

(24)

Benefits of the FP

Benefits of the FP--tree Structure

tree Structure

Completeness

Preserve complete information for frequent pattern Preserve complete information for frequent pattern

mining mining

Never break a long pattern of any transactionNever break a long pattern of any transaction

Compactness

Reduce irrelevant infoReduce irrelevant info——infrequent items are goneinfrequent items are gone

Items in frequency descending order: the more Items in frequency descending order: the more

frequently occurring, the more likely to be shared frequently occurring, the more likely to be shared

Never be larger than the original database (not count Never be larger than the original database (not count

node

(25)

Latihan

Diberikan :

Min_sup = 60% Min_sup = 60% Min_conf = 80% Min_conf = 80% Transaction

Transaction--idid Items boughtItems bought

T100 T100 K, A, B, DK, A, B, D T200 T200 A, C, D, B, EA, C, D, B, E T300 T300 C, A, B, EC, A, B, E T400 T400 B, EB, E

Carilah semua itemset frequent dengan Apriori dan FPCarilah semua itemset frequent dengan Apriori dan

FP--growth, bandingkan efisiensi keduanya growth, bandingkan efisiensi keduanya

Daftarkan semua strong association rules yang Daftarkan semua strong association rules yang