CSC662 Data Mining, Data Warehouse and Visualization

(1)

5 ก

. . ก !"# $%

#& '& $ & $ ()ก$*

CSC662 Data Mining, Data Warehouse and

Visualization

18 .. 2550 Concept description 2

ก1*2"34""5

"! ' ก(1

ก!1!" 5$

ก &*$&ก !3" ก(1

"6!7 ก! " 8!ก! &ก1 " '!(!

ก9"5 '

:

!2;"*

ก1*2"34""5 (Descriptive mining)<!ก=

&4!4!ก"5 !& * ก ;!ก' '8! ก ก3!

>!34>(8 >'4"* ก>*%8>!ก"5 4ก8

ก"5 ก (Characterization) ก"5 ก (Comparison)

ก1*2"34"1! (Predictive mining)<!ก=&

4!4!ก4 (model) !1>'41!34">!"!&8 ก8"!

&ก8*8 Descriptive mining ก Predictive mining

ก1*2"34""5

34">'48(1<!4""8>!3"Multidimensional data model 8((ก 34"<! *2"!"

กก&4!<!" ! กก1*! "$2"ก&4!*

&44(ก"

>'4ก Multidimensional data modelก! ' (! >'4ก

sum, count, max, min

>'4ก4"ก 4>'42" (&4(ก8" 4>'44"&

34>(>! 4"ก ก3"5ก ( (

ก1*2"34""5 ก &4(ก"

(2)

*2"!ก !

5$4<!ก34" ก(( <!

>'4 1! !ก กก ! drilling, pivoting, slicing dicing

ก8ก !

<!ก( ก" ! ก ก( ก4>'4

<!ก &*$2" ก ก2"ก (") 4>'4

( กก '! ? ?4"!ก ก( กก 8? ?4"!

&*$กก(3"34"ก 8ก (กก(3"34"

&ก8*8 กก "

ก!34">*4"8>!

34"">! 8 ("ก) >'4ก"&&

8 3=;!

5ก

1* "1

1* ก1*2"34" >'4 5ก"! ' ก(1 (Attribute- oriented induction)

* กก!34">*4"8>!

ก"&&

:!

" ก "" %@!

ก '>*8

' ;!

( *

34"กก$ (data cube)

>'4ก count( ), sum( ), average( ), max( ), min(), product() ( (Generalization) >'4roll-up

( ก>! 8"(Specialization)>'4 drill-down

34"(1ก

8:>'4ก ก(18>'8(1!! *2" "2!

>'4!$&4!*&4?=4"" ก$ >'4ก 1! !ก"

roll-up, drill-down, pivot, slice and dice

ก>'4กก$2"3!

5ก"! ' ก(1 Attribute-oriented induction (AOI) >'44ก ก(1 ;<!(1!!8>'8(1!!

3 ;!"! 5

(ก 34"' !5$ 4! (initial relation)

ก1( ก(1ก!ก(ก88" >!

กก ก?;1?4"!3"!34""( >*4ก !>!83"&:

(! *!=(ก8!) *2"ก !>!8(1!! (!1&8 <!

(1!!กก !) *2" ;!?;1ก ! ก8

!?;1(!2"!3ก( '8! ((1!! ก(18กก !)

"! ' ก(1 (AOI)

(3)

Data focusing: & 2"ก ก(1ก6>! initial relation Generalization:

Attribute removal:ก1( ก(1 :4

ก(1 ( A &8ก8ก !ก

8: *2" A :ก"5 4 ก(1"2!

Attribute generalization:&8>! ก(1 :4

ก(1&8ก8ก !ก

ก83" 1! !ก! ก(1>*4"8>! 4

ก( ก ก(13" AOI

Attribute generalization threshold control: "(ก1*!*&8 1* 8 ก(1*2"ก1*!&8>!A!34"' !5$

&8 ก8"ก(1!!&88ก !" 43" ก(1

(distinct values)'8!&8 distinct value = 1 <! ก(18!8

!>( *2"&8ก8ก !ก(distinct values > 8)<! ก(1

"กก !>*41ก3=;!>!"3" ก(1

Generalized relation threshold control: ก1*!ก1! ; A!34"' !5$ & (1!!&88ก !3"ก ก(18

*8( >*4"8>! *2"8 ก3=;!(1>*4(1!!

ก(1!4" ( ) *2"ก((1>*4(1!! ก (1ก3=;! ( )

&&ก1>*4"8>! 3" AOI

InitialRel: 3 ;!ก4 initial relation

PreGen: 3 ;! &*$ก1>*4<! '8!กก1( *2"ก ก(1>*4"8>!

PrimeGen: 3 ;!2"ก!(ก PreGen 2">'4ก initial relation

1>*4ก “prime generalized relation” ( กก 34"?;1?4"!

ก4 ก(1ก63"!?;1?4"!

Presentation: (1) >*4* (2) ก*! (3) ก4ก9

!1!"4#!5

3 ;!"! 5"! ' ก(1

Name Gender Major Birth-Place Birth_date Residence Phone # GPA Jim

Woodman

M CS Vancouver,BC, Canada

8-12-76 3511 Main St., Richmond

687-4598 3.67 Scott

Lachance

M CS Montreal, Que, Canada

28-7-75 345 1st Ave., Richmond

253-9106 3.70 Laura Lee

…

F

…

Physics

…

Seattle, WA, USA

…

25-8-70

…

125 Austin Ave., Burnaby

…

420-5232

…

3.83

… Removed Retained Sci,Eng,

Bus

Country Age range City Removed Excl,

VG,..

Gender Major Birth_region Age_range Residence GPA Count

M Science Canada 20-25 Richmond Very-good 16 F Science Foreign 25-30 Burnaby Excellent 22

… … … … … … …

Birth_Region Gender

Canada Foreign Total

M 16 14 30 F 10 22 32 Total 26 36 62 Prime

Generalize d Relation Initial Relation

"8ก>'4 AOI

(4)

& !5$ (Generalized relation)

& !5$3" ก(1

34 (Cross tabulation)

3"& !5$&8 (&4ก contingency tables)

!#!5 (Visualization)

! # !# ก !# 8 !# 4!

ก9 ก(1!! (Quantitative characteristic rules) <!ก94"&8ก34" '8!

grad(x) ∧ male(x) ⇒ birth(x)=“Thai” [t:95%] ∨ birth(x)=“foreign” [t:5%]

ก!1!" 5$

ก!1!"& !5$

15 300

12 250

" ก 28 450

&" "$ 120 1,000 &" "$ 150 1,200

" ก &" "$ 200 1,800 (

) ()

"?

5000 525

4000 470

1000

45

1300 1450 2250 135

162 228 1000 1200 1800 120

150 200 300

250 450 15

12 28

"?

" ก

(1!!

"3 (1!!

"3

;&8

&" "$

:!\ !&4

ก!1!"34

ก

&*3" :ก2"ก (&>! ' ;!)

&<!" ! 4>'4:&&8! "$

5$4&88"&34>(3"4>'4 ก &*$&ก !3" ก(1

<! 5: >'4>!3 ;!ก34"

2"ก ก(18ก34"*2"ก!4"""ก

1 &1& %3" ก(1 42"ก(กก!4"

&ก ! ((ก &>! ' ;!

ก &*$&ก !3" ก(1

(5)

3 ;!"! 5

34" (Data Collection)

&1!*&8'82"ก 5ก>*4"8>!

>'4ก&1! information gain 2"* &ก !

&*$*&ก ! Relevant Analysis

&ก !

Attribute-oriented induction

2"ก&ก !(ก ก(14(ก AOI >'4" ('8! drilling, slicing) ก ก(12"ก

3 ;!"!ก &*$&ก !3" ก(1

&ก ! <!&8&1!(ก34">! ก(1 &84

"ก&1& %3" ก(1 ก8ก ;34"

&ก !>'4

Information gain

Gain ratio

Gini index

χ² contingency table statistics

"2! E

&ก !

4!4ก !>( (Decision tree) ก"4

("#>!!ก" ก(12"ก

ก !&8<!4(กก" ก(1

>! 5$3"ก8F* = ก(1&

:4ก ก(1<! Categorical >'43 ;!"! 5 ID3

>'4ก834">'44 train 34""&:ก4" test >'4&8 information gain (ID3) 3"ก ก(1*2""8 ก 4!4ก !>(48&&=กกก ! ! ;!(!1>'4

ก 34"8&*6!48! ก (Overfitting problem)

ก>'4G934"1* &ก !

?3" ก(1 = {Outlook, Temperature, Humidity, Wind}

Outlook

Humidity Wind

sunny overcast rain

yes

no yes

high normal

no

strong weak

yes

ก(1F* PlayTennis = {yes, no}

"84!4ก !>(

(6)

ก1*!?3"34" S ?=8<! m ?8"&83"& c_i i = {1,

…, m} 8?8"' ก s_i ! &83"34"&&2"

"6!73" ก(1 ^A &88ก ! a₁,a₂,…,a_v 8""ก<!?

8ก ;!&8 a_j 8?(34"?8"&83"& c_i (1!! s_ij

&8ก!34"4(ก ก(1 A &1!4

InfoGain(S, A) = H(S) – H(S | A)

"6!7&8ก!

HS=

∑

i=1 m s_i

Slog₂s_i S

HS∣A=

∑

j=1

v s1j⋯sm j

S Is1 j,, sm j

&8" 8!ก! <! &ก !3" ก(1"ก#*!=>'4

* กก&1!&4ก &8ก! 8(ก ก&83"34">! ก (1!>(

&8" 8!ก! ก(1 A &1!4(ก

&ก1 " <! &ก !>'4: &ก1 "&F*

4"ก

&8" 8!ก!&ก1 "

GainRatioS , A=HSHS∣A

HA

18 .. 2550 Classification: Decision tree 23

ก1*!?3"34" S ;* n &, &8'!(! gini !

2" p_j &2"&8&: 5$3"& j >! S

(? S 8 ก(1 A ""ก<!&88ก ! a₁,a₂,…,

a_v 8""ก<!?8ก ;! A_j 1 ?(34"?8"&8 3"& c_i (1!! s_ij

'!(!6ก(:ก2"ก>'4 giniS=1

∑

j=1 n

p²_j

&8'!(! (Gini index)

GiniIndexS∣A=

∑

j=1

v s_1j⋯s_{m j}

S giniA_j

(*&1"5 ก3"! H =ก ;!ก'

ก1*!

ก(1name, gender, major, gpa birth_place, birth_date

phone#

Gen(a_i) =ก"&& ' ;!3" ก(1 a_i

U_i =&8ก8ก !ก>! ก(1 a_i

T_i =&8&&(1!!ก9""ก3" ก(1 a_i

R = 3"3&8&ก ! ("$?6!$3"(1!! ก(1 ก9>! 5$)

($ "8ก1!3" AOI

(7)

1. 3 ;!34": กก8F* (! H =ก) ก ก8 (! %%)

2. 3 ;!ก &*$*

1. ก1( ก(18*ก ก &*$: ก1( namephone#

&834"ก8ก !ก (>'4 U_i)

2. !&834">*4ก43=;! ( "8 ' ;!3=;!): !&83"

major, birth_place, birth_date ^gpaก^count (>'4 T_i)

3. !4!ก"5 ก(1*2": gender, major, birth_country, age_rangegpa (>'4 R)

ก!ก I

gender major birth_country age_range gpa count M Science Canada 20-25 Very_good 16 F Science Foreign 25-30 Excellent 22 M Engineering Foreign 25-30 Excellent 18 F Science Foreign 25-30 Excellent 25 M Science Canada 20-25 Excellent 21 F Engineering Canada 20-25 Excellent 18

Candidate relation for Target class: Graduate students (ΣΣΣΣ₌₁₂₀₎ gender major birth_country age_range gpa count M Science Foreign <20 Very_good 18

F Business Canada <20 Fair 20

M Business Canada <20 Fair 22

F Science Canada 20-25 Fair 24

M Engineering Foreign 20-25 Very_good 22 F Engineering Canada <20 Excellent 24 Candidate relation for Contrasting class: Undergraduate students (ΣΣΣΣ₌₁₃₀₎

4(กก!ก I

ก &*$&ก ! (Relevant analysis)

&1!&8&&!3"information gain >'4>!ก8

&1!"6!73" ก(1: major

(1!!! H >!

IScience”

(1!!! %%

>!IScience”

ก!ก II

1* major=“Science”:

1* major=“Engineering”:

1* major=“Business”:

s₁₁= 84 s₂₁= 42 I(s₁₁, s₂₁) = 0.9183 s₁₂= 36 s₂₂= 46 I(s₁₂, s₂₂) = 0.9892 s₁₃= 0 s₂₃= 42 I(s₁₃, s₂₃) = 0

Is_1,s₂=I120, 130=120

250log₂120 250130

250log₂130

250=0.9988

&1!&8&&!3" information &

&1!&8ก!34"3" major

Gain(major) = I(s₁, s₂) – E(major) = 0.2115 &8ก!34"3" ก(1*2"

5$(กก!ก II

Gain(gender) = 0.0003

Gain(major) = 0.2115

Gain(birth_country) = 0.0407

Gain(gpa) = 0.4490

Gain(age_range) = 0.5971 Emajor=126

250Is_{1 1}, s₂₁ 82

250Is_{1 2}, s_{2 2} 42

250Is_{1 3}, s_{2 3}=0.7873

(8)

4 Initial working relation (W₀) ก1*!R = 0.1

ก1( ก(18ก34" (genderbirth_country)

8!1"&>'4>!ก4relation

>'4 attribute-oriented induction! ^W04"&8^Ti

major age_range gpa count

Science 20-25 Very_good 16 Science 25-30 Excellent 47 Science 20-25 Excellent 21 Engineering 20-25 Excellent 18 Engineering 25-30 Excellent 18

Initial target class working relation W₀: Graduate students

"8ก1*2"34""5

ก ก(*8&>! ก !

3 ;!"! 5:

8ก ;!?34"&83"&F* &

2"ก ' ;!>!8!3"&F**2"& >*4 ' ;!3"&"ก&4

!1!" 5$ 4&1"5 3" ก(14" " &2"

support – กก(3"&2"ก ;*

comparison – กก(3"&2"ก &"2!

!4! 5$>*4&8 ก8(ก&8"2! E

ก12"34"&ก88&

ก1*!

ก(1 name, gender, major, birth_place, birth_date, residence, phone# gpa

Gen(a_i) = ก"&& ' ;!3" ก(1 a_i

U_i =&8ก8ก !ก>! ก(1 a_i

T_i = &8&&(1!!ก9""ก3" ก(1 a_i

R = 3"3&8&ก !3" 5$

"8ก1*2"34"

1. ก634"8&

8<!34">!ก8F* 34">!ก8 2. &*$&ก !3" ก(1

ก1( ก(1 name, gender, major, phone#

3. !>*4"8>! 3" ;"&

4>'4ก1*!3"33"(1!! 4"ก

5$3" ;&F* &>! ก

& !5$*2"*ก (relations/cuboids)

ก!ก I ก &*$ก

(9)

Birth_country Age_range Gpa Count%

Canada 20-25 Good 5.53%

Canada Over_30 Very_good 5.86%

… … … …

Other Over_30 Excellent 4.68%

Prime generalized relation for the target class: Graduate students Birth_country Age_range Gpa Count%

Canada 15-20 Fair 5.53%

… … … …

Other Over_30 Excellent 0.68%

Prime generalized relation for the contrasting class: Undergraduate students

(กก!ก I

>*4 A ! ก(14"ก' A_i &2"34"3"ก8 i >! ก(14"ก

t-weight &8>!'8 [0, 1] ! &2"

ก9 ก' (quantitative characteristic)

∀x, target_class(x) ⇒ condition(A₁) [t:t_weight] ∨ ... ∨ condition(A_n) [t:t_weight]

ก9 ก'

tweight= countAi

∑

i=1 n

countA_i

>*4 C_j !ก8F*

q_a &2"34" >*4"8>! &ก8F*

"(&"&34"8!3"ก8

d-weight &8>!'8 [0, 1] ! &2"

ก9&ก8' (quantitative discriminant)

∀x, target_class(x) ⇐ condition(x) [d:d_weight]

ก9&ก8'

dweight= countq_a∈C_j

∑

i=1 m

countq_a∈C_i

ก9&ก8' (Quantitative discriminant rules)&2"

∀x, graduate_student(x) ⇐ birth_country(x) = “Canada” ∧ age_range(x) = “25-30” ∧ gpa(x) = “good” [d:30%]

90/(90+210) = 30%

Status Birth_country Age_range Gpa Count Graduate Canada 25-30 Good 90 Undergraduate Canada 25-30 Good 210

! กก(*8! H =กก ! %% H

"8ก9&ก8'

(10)

ก9 ก' (Quantitative characteristic rules)

∀x, target(x) ⇒ condition(x) [t:t_weight]

(1<! (necessary) ก!34""8>!& ก8 ก9&ก8' (Quantitative discriminant rules)

∀x, target(x) ⇐ condition(x) [d:d_weight]

" (sufficient) :434" 4(1!!! ก8& 8!

8>1* &!>(

ก9"5 ' (Quantitative description)

∀x, target(x) ⇔ condition₁(x) [t:w₁, d:v₁] ∨ ... ∨condition_n(x) [t:w_n, d:v_n] (1<!"(necessary and sufficient)

ก91* "5 &

ก9"5 ' 1* ก8F* Europe

∀x, Europe(x) ⇔ (item(x) = “TV”) [t:25%, d:40%] ∨ (item(x) =

“computer”) [t:75%, d:30%]

Location/item TV Computer Both_items

Count t-wt d-wt Count t-wt d-wt Count t-wt d-wt

Europe 80 25% 40% 240 75% 30% 320 100% 32%

N_Am 120 17.65% 60% 560 82.35% 70% 680 100% 68%

Both_

regions

200 20% 100% 800 80% 100% 1000 100% 100%

34&8 t-weight, d-weight ก (*!8<! !) 3"&" "$

"8ก9"5 '

1: :=:ก!1>'4>!ก1*2"34"

: <!$=กกก 34" "5 34">! ก8 E '8!

ก"5 4 !34" กก(34"<!4!

5ก"กกก(34" '8! &8 5A! (median), &8 (max), &81

(min), &"$$ (quartiles), &! ^(variance) (4"<!(1!!2">'4&1!4

&*$กก(34" >'4 boxplot*2"quantile analysis ก &*$กก(! ก! <!(1!! >'4boxplot

*2"quantile analysis

ก1*2"34"4 5:

&83& (Mean)

&83& :8!;1*! ก 5A! (Median)

&8ก=ก34":4(1!!34"<!&*2"&83"&8ก"&8:4 (1!!34"<!&8

A!! (Mode)

&834"ก98"

mean – mode ≈ 3 ×(mean - median)

&8ก

x=

∑

i=1 n

x_i

n

x=

∑

i=1 n

w_ix_i

∑

i=1 n

w_i

(11)

กกก(""ก<!8!>'4*4&8:

&81(Q₀), Q₁, 5A! (Median), Q₃,&8 (Q₄) กกJก8" Boxplot

'34":ก!4ก8"

(3"ก8"&2" &"$$ก&"$$

&3"ก8"&2"IQR (Interquartile range) 5A!&2"4!>!ก8"

32!""ก(Whiskers) '2" &81&88ก ! 1.5 IQR

ก &*$>'4 boxplot

"8 boxplot

Whisker

Median 25%

25%

Q₃

Q₁

กJ3" boxplot >!

&!3" "8

8!!A!: ก"<!ก3"&!

กก("&83&

<!!$ ก68"2"34"&8*2"!ก !*

'& (algebraic property)

กก(

s²= 1

n1

∑

i=1 n

x_ix²= 1 n1

 ∑i=1

n

x_i²1 n

∑

i=1 n

x_i²



(12)

ก &*$>'4Kก

กJ8กก(3"*!= ก(1 & = &:

กJ&"!$ (Quantile plot)

กJ&"!$ กก(3"34" ;* 34"ก !

!4"ก8*2"8ก x_i

Quantile-Quantile (Q-Q) plot

กJ&"!$-&"!$ *2" Normal quantile plot :=กก (34"2"ก ก(ก(ก

กJ3"" (Scatter plot)

กJ3"" (Scatter plot/Bivariate plot) (34">'4

" ก1*!>!8ก! := กกกก8ก!

3"34"3" *!=ก *2"

(13)

ก1*2"34""5 (Mining concept description) &2"ก

&4!*&1"5 3"ก834" "( ก 3" ก (1*2"กก ก8ก8ก !

5ก4?=&1"5

OLAP-based

attribute-oriented induction

ก &*$&ก !'8>!ก2"ก ก(11& %>!ก"5 !"ก(ก!;: :ก!1>'4ก1*2"34""5

Y. Cai, N. Cercone, and J. Han. Attribute-oriented induction in relational databases. In G.

Piatetsky-Shapiro and W. J. Frawley, editors, Knowledge Discovery in Databases, pages 213-228. AAAI/MIT Press, 1991.

S. Chaudhuri and U. Dayal. An overview of data warehousing and OLAP technology. ACM SIGMOD Record, 26:65-74, 1997

C. Carter and H. Hamilton. Efficient attribute-oriented generalization for knowledge discovery from large databases. IEEE Trans. Knowledge and Data Engineering, 10:193-208, 1998.

W. Cleveland. Visualizing Data. Hobart Press, Summit NJ, 1993.

J. L. Devore. Probability and Statistics for Engineering and the Science, 4th ed. Duxbury Press, 1995.

T. G. Dietterich and R. S. Michalski. A comparative review of selected methods for learning from examples. In Michalski et al., editor, Machine Learning: An Artificial Intelligence Approach, Vol. 1, pages 41-82. Morgan Kaufmann, 1983.

J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Reichart, M. Venkatrao, F. Pellow, and H. Pirahesh. Data cube: A relational aggregation operator generalizing group-by, cross-tab and sub-totals. Data Mining and Knowledge Discovery, 1:29-54, 1997.

J. Han, Y. Cai, and N. Cercone. Data-driven discovery of quantitative rules in relational databases. IEEE Trans. Knowledge and Data Engineering, 5:29-40, 1993.

"ก"4" L

J. Han and Y. Fu. Exploration of the power of attribute-oriented induction in data mining.

In U.M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, editors, Advances in Knowledge Discovery and Data Mining, pages 399-421. AAAI/MIT Press, 1996.

R. A. Johnson and D. A. Wichern. Applied Multivariate Statistical Analysis, 3rd ed.

Prentice Hall, 1992.

E. Knorr and R. Ng. Algorithms for mining distance-based outliers in large datasets.

VLDB'98, New York, NY, Aug. 1998.

H. Liu and H. Motoda. Feature Selection for Knowledge Discovery and Data Mining.

Kluwer Academic Publishers, 1998.

R. S. Michalski. A theory and methodology of inductive learning. In Michalski et al., editor, Machine Learning: An Artificial Intelligence Approach, Vol. 1, Morgan Kaufmann, 1983.

T. M. Mitchell. Version spaces: A candidate elimination approach to rule learning.

IJCAI'97, Cambridge, MA.

T. M. Mitchell. Generalization as search. Artificial Intelligence, 18:203-226, 1982.

T. M. Mitchell. Machine Learning. McGraw Hill, 1997.

J. R. Quinlan. Induction of decision trees. Machine Learning, 1:81-106, 1986.

D. Subramanian and J. Feigenbaum. Factorization in experiment generation. AAAI'86, Philadelphia, PA, Aug. 1986.

"ก"4" M

CSC662 Data Mining, Data Warehouse and Visualization

CSC662 Data Mining, Data Warehouse and

&ก8*8 กก "

ก( ก ก(13" AOI

"8ก>'4 AOI

3 ;!"!ก &*$&ก !3" ก(1

&8" 8!ก!&ก1 "

ก1*2"34"&ก8*8&

ก9&ก8'

ก &*$>'4 boxplot

ก12"34"&ก88&