Lecture 22: 21 April, 2022

(1)

Lecture 22: 21 April, 2022

Madhavan Mukund

https://www.cmi.ac.in/~madhavan

Data Mining and Machine Learning January–May 2022

(2)

Probabilistic graphical models

Underlying DAG, no cyclic dependencies

Each node has a local (conditional) probability table

(3)

Conditional independence

x ?y — x and y are independent P(x^y) =P(x)·P(y)

x ?y |z

x andy are independent givenz

Is JohnCallsindependent of MaryCalls (j ?m)?

No — value ofj tells us something about value ofmand vice versa

Is JohnCallsindependent of MaryCalls given Alarm (j ?m|a)?

Yes — by semantics of network, local independence

Madhavan Mukund Lecture 22: 21 April, 2022 DMML Jan–May 2022 3 / 10

(4)

Conditional independence

x ?y |z

x andy are independent given z

(5)

x ?y |z

(6)

Conditional independence

x ?y |z

(7)

x ?y |z

(8)

Conditional independence

x ?y |z

(9)

Probabilistic graphical models

Fundamental assumption

A node is conditionally independent of non-descendants, given its parents

(10)

Student example

SAT ?Grade |Difficulty ?

No

Can we calculate

conditional independence from the graph?

In general, check if X ?Y |Z for setsof variables X,Y,Z

¥

(11)

Student example

SAT ?Grade |Difficulty ? No

Can we calculate

(12)

Student example

Can we calculate

(13)

Student example

Can we calculate

~

(14)

Conditional independence

How does dependence “flow” through a network?

For neighbouring nodes, dependence flows both ways

Ifx !y, knowingx tells us about y and vice versa

Examinetrails between nodes Paths in the underlying undirected graph

Basic trails— (undirected) paths of length 2

Four basic trails

(15)

Conditional independence

Ifx!y, knowingx tells us about y and vice versa

Four basic trails

✗

¥ FE

(16)

Conditional independence

Four basic trails

✗t

0

✗

↓ ↑

t

(17)

Conditional independence

Four basic trails

FETE

✓ ✓

A

✓

(18)

Basic trails

(a), (b) and (c): Z blocks flow between X andY, by semantics of Bayesian networks

In (d), knowingZ allows influence to flow

Z: Car does not start X: Low Battery,Y: No Fuel Z: Grass is wet

X: Overnight rain,Y: Sprinkler ran Simplest form of V-structure

E

✗ ✗

J

- ^-

É→→→o→E

(19)

Basic trails

I

^✗¹⁷

/

^2-

(20)

Basic trails

(21)

Basic trails

Z: Car does not start X: Low Battery,Y: No Fuel

Z: Grass is wet

(22)

Basic trails

X: Overnight rain,Y: Sprinkler ran

Simplest form of V-structure

☐

(23)

Basic trails

(24)

D-Separation

Check if X ?Y |Z

Dependence should be blocked on every trail from X toY

Each undirected path from X toY is a sequence of basic trails For (a), (b), (c), needZ present For (d), need Z absent

In general, V-structure includes descendants of the bottom node

x andy are D-separatedgiven z if all trails are blocked

Variation of breadth first search (BFS)to check if y is reachable fromx through some trail

Extends to sets — eachx 2X is D-separated from eachy 2Y

(25)

D-Separation

Check if X ?Y |Z

Each undirected path from X toY is a sequence of basic trails For (a), (b), (c), needZ present For (d), need Z absent

(26)

D-Separation

Check if X ?Y |Z

Each undirected path fromX toY is a sequence of basic trails

For (a), (b), (c), needZ present For (d), need Z absent

(27)

D-Separation

Check if X ?Y |Z

Each undirected path fromX toY is a sequence of basic trails For (a), (b), (c), needZ present

For (d), need Z absent

⊕

(28)

D-Separation

Check if X ?Y |Z

Each undirected path fromX toY is a sequence of basic trails For (a), (b), (c), needZ present For (d), need Z absent

(29)

D-Separation

Check if X ?Y |Z

④

^→

④

^←

⑦

②

^→ ^✓ ^W ^is ^known

(30)

D-Separation

Check if X ?Y |Z

(31)

D-Separation

Check if X ?Y |Z

(32)

D-Separation

Check if X ?Y |Z

(33)

Conditional independence, example

Is SATindependent of Difficulty given Intelligence?

Yes,Difficulty – Grade –

Intelligence – SAT trail is blocked atGrade(V-structure)

Is SATindependent of Difficulty given Grade?

No,Difficulty – Grade – Intelligence – SATtrail is open becauseGrade is known (V-structure)

-

_

-

00*0

(34)

Conditional independence, example

Is SATindependent of Difficulty given Intelligence?

Yes,Difficulty – Grade –

Intelligence – SAT trail is blocked atGrade(V-structure)

Is SATindependent of Difficulty given Grade?

No,Difficulty – Grade – Intelligence – SATtrail is open becauseGrade is known (V-structure)

0*0