CSC662 Data Mining, Data Warehouse and Visualization

(1)

7 ก ก

. . ก !"

! #$ $ %&ก$ '

CSC662 Data Mining, Data Warehouse and

Visualization

มิถุนายน

18 2550 การทําเหมืองขอมูล As s ocia tion rule 2

ก /กก

ก()*+ก,# (Association rule)

' ')ก,#

-!)ก,#

)' .ก()*+ก,#

ก,#'#'

'กก progressive refinement

/

0 .ก%ก) *ก = * ก%ก1' 2*ก%

34กก+ก/ก' 5-

*ก )*ก/) ก3ก' 5/ก 3ก'

ก 'ก/ *ก

34ก)ก)5/63%%ก'*ก 2 0

'ก1$) !'$78*ก/)*/ก'

'ก1$ก)+ก/ก'

3 ( + 'ก1$ก( .ก%%ก *5

ก /กก

*0#)*/ 3

/ก34ก)*ก (5/0#/*ก 2 +/*ก' 5-5/%%)

)0%'ก1$34ก (0#-# 5

)0%!'$7

)9 ก')#

5- ก'

%5-# %กก ' ก)

-ก33)

ก '!0

ก ( +ก +%ก +8

ก /กก

(2)

ก กกก/) 0#0 ก 'ก/

#/ ก,# ก0#)*%ก point-of-sale (p-o-s) )* (0# */0 *- transaction

'ก 209%

)*'ก1$*ก (%)

-$

)*-!) )

%(

ก กก

-ก)*ก-: transaction 8 (

%( 09%6/*ก 2

%( 6/*ก 2

%( 6/ 209%

)-)ก 2#

ก-9 )*ก

%( 6)09%/*ก 2 'ก/

%( 68*ก/ 209%

ก-'ก!3 . 2#/ 'ก1$ */)2

Transaction Data

'ก1$))*

'ก1$)ก)

8 ก9 0#/ 8 ก'/)5/ ?

*-;.'ก3) /'ก1$5/ ?

0-ก,/ ก' /0 209% ?

0-ก,/ ก' /0 ก/ ก( ?

5 3 /8*ก)/0ก'*ก'ก( < ? 'ก1$ก)) ก- 5- 5/ ?

'ก1$ก)) - 5-*/)*ก5/ /5 ?

+ <

'ก1$(8('"ก ก'ก)

(3)

ก()*+ก,#ก*-+ก/

(frequent pattern) #ก)2 (association) '3' .

(correlation) )ก/ item %ก)**/0 *- transaction

(5-0#ก'

ก กก (Market basket analysis) ก(ก

)!'$7 (cross-marketing) ก *-++ก++9 9ก (catalog design) -:

'3.*/0 *-+ก,: “Body Head [support, confidence]”

buys(x, “diapers”) buys(x, “beers”) [0.5%, 60%]

major(x,“CS”) ^ takes(x,“DB”) grade(x,“A”) [1%, 75%]

ก()*+ก,#

ก ก,' 2+ '3' . / item #/

*ก'ก %8 98% +-ก$+/83 + ก (5-0#

* Maintenance Agreement ก,'ก/ #/ ก'ก1$)*ก 9 '""ก('ก1 ก'

Home Electronics * ก,'ก/ ก 8*ก5-3ก' -ก$5==>0#0 81'ก)-ก$'ก/ + )*/ก' %กก9 0 9ก033

Attached mailing *ก 0%ก'?1$

+ 'ก)ก,#

ก()* 0%ก-ก;5/-ก;) (Boolean

association rule) vs. ก()*#/ ' ))ก )

(Quantitative association rule)

buys(x, “SQLServer”) ^ buys(x, “DMBook”) → buys(x, “DBMiner”) [0.2%, 60%]

age(x, “30..39”) ^ income(x, “42..48K”) → buys(x, “PC”) [1%, 75%]

ก()* 0% (Single dimensional association rule) #/

0%ก)*ก3/ vs. ก()* 0%

(Multiple dimensional association rule) #/ 0%ก)*ก+

+ < 3ก'

-!ก()*+ก,#

ก()* 0%'#' ก' (Single level association rule) vs.

ก()* 0%'#' /ก' (Multiple level association rule) buys(x, “SQLServer”) ^ buys(x, “DMBook”) → buys(x, “DBMiner”) buys(x, DB books) ^ buys(x, “Buy.com”) → buys(x, PC)

'ก1$ <

'3' .ก'ก 8)ก (

Maxpatterns + closed itemsets ก3 5)''

#/ small sales (sum < 100) trigger big buys (sum > 1,000)?

-!ก()*+ก,#

(4)

' ' support ก' confidence

ก( buy(x, “Beer”) + ก

+ buy(x, “Pamper”) + ก

) / B = buy(x, “Beer”)

+ P = buy(x, “Pamper”)

*ก

*ก' /

*ก

/ ' (support) )ก, Beer → Pamper /%-: %3ก

' Beer + Pamper

/ #' (confidence) )ก, Beer → Pamper /%-: + 5) /ก Beer + %ก Pamper

' /ก( $/ ' +/ #'

ก,-: 5-5) #

A B, B A, C A, D A, E A A C, B C, C B, D B, E B A D, B D, C D, D C, E C A E, B E, C E, D E, E D A F, B F, C F, D F, E F, F A, F B, F C, F D, F E

ก, P → P -: ก,5/ / 0% 3-: ก,%('ก/ P

0#/()/ ' ก'/ #' ก( / 0% ก,/

' > min support +/ #' > min confidence -: ก, / 0%

ก( min support = 50% + min confidence 50%

A → C (50%, 66.67) ก' C → A (50%, 100%)

Transaction ID

2000 A, B, C 1000 A, C

4000 A, D

5000 B, E, F

)ก/ / ' > min support ก frequent itemset

3%$ก, A → C

support = support({A, C}) = 50%

confidence = support({A, C})/support({A}) = 66.67%

' Apriori:

ก') frequent itemset -: frequent itemset Min. support = 50%

Min. confidence = 50%

' /กก,#

Transaction ID

2000 A, B, C 1000 A, C 4000 A, D 5000 B, E, F

Support

{A} 75.00%

{B} 50.00%

{C} 50.00%

{A, C} 50.00%

Frequent Itemset

frequent itemset ก/) item / ' กก //ก'/

minimum support

') frequent itemset -: frequent itemset 8 {A, B}

-: frequent itemset + {A} + {B} -: frequent itemset 6 ' ก frequent itemset k ' (5%กก3%$ k – 1 '

ก/ frequent itemset %ก 1 5-82 k (k-itemset)

'%ก3%$ก frequent itemset %( ก+ ก,

#%ก frequent itemset '5

ก frequent itemset

(5)

)' : C

_k

) candidate 5%กก ก' ) L

_k-1

ก' L

_k-1

)' : 8 (k-1)-itemset ) k-itemset 5/0#/ frequent itemset + k-

itemset %ก'ก/ 5/0#/ frequent itemset

)' .: C

_k

: ก/) Candidate ) k, L

_k

: ก/) frequent itemset ) k L

₁

= ) frequent items #ก 2' ;

for (k = 1; L

_k

≠Ø; k++) do begin C

_k+1

= candidate 5%ก L

_k

;

for each transaction t in database do '%( )ก candidate 0 C

_k+1

*/0 t L

_k+1

= candidates in C

_k+1

with min_support end

return ∪

k

L

_k

;

)' . Apriori

TID รายการ 100 1 3 4 200 2 3 5 300 1 2 3 5 400 2 5

Database D itemset sup.

{1} 2

{2} 3

{3} 3

{4} 1

{5} 3

itemset sup.

{1} 2

{2} 3

{3} 3

{5} 3

Scan D

C

₁

L

₁

itemset {1 2}

{1 3}

{1 5}

{2 3}

{2 5}

{3 5}

itemset sup {1 2} 1 {1 3} 2 {1 5} 1 {2 3} 2 {2 5} 3 {3 5} 2 itemset sup

{1 3} 2 {2 3} 2 {2 5} 3 {3 5} 2 L

₂

C

₂

C

₂

Scan D

C

₃

itemset L

₃

{2 3 5} Scan D itemset sup

{2 3 5} 2

ก( ))' . Apriori

%'(' item ' + 0#(''ก/ ก'ก L

_k

= ) frequent itemset #ก k '

)' : L

_k-1

3 C

_k

select p.item

₁

, p.item

₂

, …, p.item

_k-1

, q.item

_k-1

from L

_k-1

p, L

_k-1

q

where p.item

₁

=q.item

₁

,…,p.item

_k-2

=q.item

_k-2

,p.item

_k-1

< q.item

_k-1

)' : itemset 0 L

_k

forall itemsets c in C

_k

do

forall (k-1)-subsets s of c do

if (s is not in L

_k-1

) then delete c from C

_k

'กก frequent itemset

(5ก( $/ ' ) itemset %5/-.!3 ? %( itemset 3%$%%( ก

0 2 transaction % itemset %( ก- */

.ก ' :

Candidate ก90 hash-tree

Leaf node ) hash-tree -ก ก itemset + count

Interior node -ก hash table

Subset function: itemset '0 +/ transaction

0#0 ก '/ '

(6)

L

₃

={abc, abd, acd, ace, bcd}

)' : L

₃

*L

₃

abcd 5%ก abc + abd

acde 5%ก acd + ace

)' :

acde ก3 ade 5/*/0 L

₃

C

₄

= {abcd}

' /ก candidate

Hash-based itemset counting: k-itemset 2%( bucket count

(ก /))ก( 0ก(%'ก

Transaction reduction: ) transaction 5/ frequent k-

itemset 5/%(-: / ก (ก%กF )*

Partitioning: ก itemset %-: frequent itemset 0 DB -:

frequent itemset 20 / +/ก' (partition)

Sampling: %0#ก/' /%ก transaction + 0#/ min support ('3. %ก ' %2( $/ ' %ก'F )*!'

-.!3))' . Apriori

'ก))' . Apriori:

0# frequent (k – 1) itemsets candidate frequent k-itemsets / database +*-+3 '+ candidate itemset -G") ))' . Apriori: ก candidate

-$ candidate กก 5- :

10

⁴

frequent 1-itemset 5 10

⁷

candidate 2-itemsets

ก frequent pattern ) 0"/ #/ {a

₁

, a

₂

, …, a

₁₀₀

}, 2

¹⁰⁰

≈ 10

³⁰

candidates

ก/ database :

ก (n +1) scans n ) pattern

-G"))' . Apriori

.กก/F )*09ก+ Frequent-Pattern tree (FP-tree)

FP-tree -: ก/)*'-ก' /)*0# 5/*"

กก/ database

'กก FP-tree-based frequent pattern mining

0#'ก divide-and-conquer +//0()*9ก < + ก'

กก Candidate

กก,#5/ Candidate

(7)

ก+'3.)ก,#0#

PMML visualization 0 *-

ก+'3.)ก,#0#

PMML visualization 0

(8)

ก+'3.)ก,#0#ก=

PMML visualization 0 *-%0

PMML visualization 0 *-ก=

-ก item ก ' ก' -ก -: '#' hierarchy

'/ item %/ ' (

' ' ก,#ก )ก'

itemset 0 '%

-# 0 ก (5-0#

ก%ก )* transaction %8*ก

*-+'#'

ก()*+ multi-level mining %2 / 0%

Food

bread milk

skim

Sunset Fraser

2% wheat white

TID Items

T1 {111, 121, 211, 221}

T2 {111, 211, 222, 323}

T3 {112, 122, 221, 411}

T4 {111, 121}

T5 {111, 122, 211, 221, 413}

ก,#ก''#'

(9)

.ก top_down (progressive deepening approach):

)' +ก ก,0 '#' * / 0%

milk → bread [20%, 60%]

+ ก,ก)2 0 '(

2% milk → wheat bread [6%, 50%].

'ก1$ 0#0 ก()*+ก,#'#'

Level-crossed association rules: ก,#)'0 '#'

2% milk → Wonder wheat bread

Association rules with multiple, alternative hierarchies: ก,#

)'0 '#' +ก/ก'

2% milk → Wonder bread

ก,#'#'

Uniform Support: 0#/ minimum support 2/ก'ก'#' + )/ minimum support 2/(05/3%$ itemset

+/ item ancestor 5// minimum support

– ) item 0 '#' (%5/ก-ก;//ก' item

*/0 '#' * 8ก( / '

*ก 5- + %5/8 low level associations

(ก 5- + %5 high level associations %( ก

Reduced Support: / minimum support 0 '#' (ก /

. .0# 4 'ก1$: Level-by-level independent, Level- cross filtering by k-itemset, Level-cross filtering by single item, Controlled level-cross filtering by single item

ก,#'#' ก'/ '

ก()*'#' ก' uniform support

Milk [support = 10%]

2% Milk [support = 6%]

Skim Milk [support = 4%]

Level 1

min_sup = 5%

Level 2

min_sup = 5%

Uniform support

ก()*'#' ก' reduced support

2% Milk [support = 6%]

Skim Milk [support = 4%]

Level 1

min_sup = 5%

Level 2

min_sup = 3%

Milk [support = 10%]

Reduced support

(10)

ก,ก,%5/%(-: 0# '3' . “ancestor” ) item

#/

milk → wheat bread [support = 8%, confidence = 70%]

2% milk → wheat bread [support = 2%, confidence = 72%]

%ก' /ก,+ก-: ancestor )ก,

ก,5-: ก,ก %(-: (redundant) 8/ ' )ก,/0กก' / %ก ancestor )ก,

กกก,ก %(-: (Redundancy filtering)

.ก progressive deepening 0#'กก+ top-down:

)' +ก high-level frequent item:

milk (15%), bread (10%)

+ lower-level “weaker” frequent itemsets:

2% milk (5%), wheat bread (4%)

กก( / min support +ก/ก' 0'3./ก'

80#'ก) min support ก' 0 ก'#' + ก(%' t

8 t’s ancestor ' 0/ ' (

80#'กก min support 0 '#' ( + % % 63 descendent / ' ) ancestor

กก 0 + 2ก (progressive deepending)

(5 ( progressive refinement?

ก()*%0#ก')*ก 5-

vs. $!3)'3.: step-by-step refinement ' Superset coverage:

'ก1'3.8*ก (positive) 0 false positive +/5/0 ก false negative

ก- (กก / 2'

'+ก0#' ( ก/9 rough/cheap (superset coverage) + 0#' ( ก3 candidate set

$!3)ก( progressive refinement

'3' .0 'ก1$ spatial #/ “g_close_to”: near_by, touch, intersect, contain

ก()* spatial association +)'

)' 1: rough spatial computation ( กก )

0# MBR R-tree 0 ก-$+

)' 2: Detailed spatial algorithm ( )

0#ก'/ rough spatial association

ก()* Spatial ก' Association

(11)

ก()*+ก,# (Mining association rules) -: .กก%ก ก()*-: 'ก

-ก;0 KDD +ก ( %( ก

'3.5ก, 2% 0%ก5 .ก/ < +'ก1$)ก, ) %' / 0%

ก Association ก')*-! #/ spatial data, multimedia data, time series data -:

-

R. Agarwal, C. Aggarwal, and V. V. V. Prasad. A tree projection algorithm for generation of frequent itemsets. In Journal of Parallel and Distributed Computing (Special Issue on High Performance Data Mining), 2000.

R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. SIGMOD'93, 207-216, Washington, D.C.

R. Agrawal and R. Srikant. Fast algorithms for mining association rules. VLDB'94 487-499, Santiago, Chile.

R. Agrawal and R. Srikant. Mining sequential patterns. ICDE'95, 3-14, Taipei, Taiwan.

R. J. Bayardo. Efficiently mining long patterns from databases. SIGMOD'98, 85-93, Seattle, Washington.

S. Brin, R. Motwani, and C. Silverstein. Beyond market basket: Generalizing association rules to correlations. SIGMOD'97, 265-276, Tucson, Arizona.

S. Brin, R. Motwani, J. D. Ullman, and S. Tsur. Dynamic itemset counting and implication rules for market basket analysis. SIGMOD'97, 255-264, Tucson, Arizona, May 1997.

K. Beyer and R. Ramakrishnan. Bottom-up computation of sparse and iceberg cubes. SIGMOD'99, 359- 370, Philadelphia, PA, June 1999.

D.W. Cheung, J. Han, V. Ng, and C.Y. Wong. Maintenance of discovered association rules in large databases: An incremental updating technique. ICDE'96, 106-114, New Orleans, LA.

M. Fang, N. Shivakumar, H. Garcia-Molina, R. Motwani, and J. D. Ullman. Computing iceberg queries efficiently. VLDB'98, 299-310, New York, NY, Aug. 1998.

ก (H)

G. Grahne, L. Lakshmanan, and X. Wang. Efficient mining of constrained correlated sets. ICDE'00, 512- 521, San Diego, CA, Feb. 2000.

Y. Fu and J. Han. Meta-rule-guided mining of association rules in relational databases. KDOOD'95, 39- 46, Singapore, Dec. 1995.

T. Fukuda, Y. Morimoto, S. Morishita, and T. Tokuyama. Data mining using two-dimensional optimized association rules: Scheme, algorithms, and visualization. SIGMOD'96, 13-23, Montreal, Canada.

E.-H. Han, G. Karypis, and V. Kumar. Scalable parallel data mining for association rules. SIGMOD'97, 277-288, Tucson, Arizona.

J. Han, G. Dong, and Y. Yin. Efficient mining of partial periodic patterns in time series database.

ICDE'99, Sydney, Australia.

J. Han and Y. Fu. Discovery of multiple-level association rules from large databases. VLDB'95, 420-431, Zurich, Switzerland.

J. Han, J. Pei, and Y. Yin. Mining frequent patterns without candidate generation. SIGMOD'00, 1-12, Dallas, TX, May 2000.

T. Imielinski and H. Mannila. A database perspective on knowledge discovery. Communications of ACM, 39:58-64, 1996.

M. Kamber, J. Han, and J. Y. Chiang. Metarule-guided mining of multi-dimensional association rules using data cubes. KDD'97, 207-210, Newport Beach, California.

M. Klemettinen, H. Mannila, P. Ronkainen, H. Toivonen, and A.I. Verkamo. Finding interesting rules from large sets of discovered association rules. CIKM'94, 401-408, Gaithersburg, Maryland.

ก (I)

F. Korn, A. Labrinidis, Y. Kotidis, and C. Faloutsos. Ratio rules: A new paradigm for fast, quantifiable data mining. VLDB'98, 582-593, New York, NY.

B. Lent, A. Swami, and J. Widom. Clustering association rules. ICDE'97, 220-231, Birmingham, England.

H. Lu, J. Han, and L. Feng. Stock movement and n-dimensional inter-transaction association rules.

SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery (DMKD'98), 12:1- 12:7, Seattle, Washington.

H. Mannila, H. Toivonen, and A. I. Verkamo. Efficient algorithms for discovering association rules.

KDD'94, 181-192, Seattle, WA, July 1994.

H. Mannila, H Toivonen, and A. I. Verkamo. Discovery of frequent episodes in event sequences. Data Mining and Knowledge Discovery, 1:259-289, 1997.

R. Meo, G. Psaila, and S. Ceri. A new SQL-like operator for mining association rules. VLDB'96, 122- 133, Bombay, India.

R.J. Miller and Y. Yang. Association rules over interval data. SIGMOD'97, 452-461, Tucson, Arizona.

R. Ng, L. V. S. Lakshmanan, J. Han, and A. Pang. Exploratory mining and pruning optimizations of constrained associations rules. SIGMOD'98, 13-24, Seattle, Washington.

N. Pasquier, Y. Bastide, R. Taouil, and L. Lakhal. Discovering frequent closed itemsets for association rules. ICDT'99, 398-416, Jerusalem, Israel, Jan. 1999.

ก (J)