Marginal Probability Mass Function

Joint Distribution

randomly draw 3 balls

X = no. of red balls Y = no. of white balls

{0,1,2,3} {0,1,2,3}

Prob.

3 2 1 0 X

Prob.

3 2 1 0 Y

(

0

)

0.3818 Pr

3 12

3 9 ₌

= =

C C X

0.3818

(

1

)

0.4909

Pr

3 12

2 9 1 3 × ₌

= =

C C C X

0.4909

(

2

)

0.1227

Pr

3 12

1 9 2 3 × ₌ = =

C C C X

0.1227

(

3

)

0.004545

Pr

3 12

3 3 ₌ = =

C C X

0.004545

0.2545 0.5091 0.2182 0.01818

Joint Distribution

randomly draw 3 balls

X = no. of red balls Y = no. of white balls

{0,1,2,3} {0,1,2,3}

(

0, 0

)

0.04545 Pr

3 12

3 5 ₌ = = =

C C Y X

3 2 1 0

3 2 1 0 X Y

0.04545

(

1, 2

)

0.08182 Pr

3 12

2 4 1

3 × ₌

= = =

C C C Y X

0.08182

(

2, 0

)

0.06818 Pr

3 12

1 5 2

3 × ₌

= = =

C C C Y X

0.06818

0.1818 0.1364 0.01818

0.1364 0.2727 0

0.05455 0 0

0 0 0 0.004545

Joint Probability Mass Function

( )

x y

(

X xY y

)

p , =Pr = , =

( ), ,0 3

3 12

3 5 4

3 × × _{≤ + ≤}

= −−

y x C

C C C y x

p x y xy

Joint pmf

Example :

0 0 0 0.04545 3

0 0 0.05455 0.06818 2

0 0.08182 0.2727 0.1364 1

0.01818 0.1364 0.1818 0.04505 0

3 2 1 0 X Y

0.04545 0.1227 0.4909 0.3818 Total

1 0.01818 0.2182 0.5091 0.2545 Total

( 0)

PrY= Pr(Y=1) Pr(Y=2) Pr(Y=3)

( 0) PrX=

( 1) PrX=

( 2) PrX=

( 3) PrX=

Marginal Probability Mass Function

( )

=

(

=

)

=

∑

( )

y

X x X x pxy

p Pr ,

( )

=

(

=

)

=

∑

( )

x

Y y Y y pxy

p Pr ,

Example :

( )

, 0,1,2,3

3 12

3 9

3 × ₌

= − _x

C C C x

p x x

X

( )

, 0,1,2,3 3

12 3 9

4 × ₌

= − _y

C C C y

p y y

Y

I ndependence

( )

x y p

( ) ( )

xp y

p , = X Y

X and Y are independent if

for all x,y

0 0 0 0.04545 3

0 0 0.05455 0.06818 2

0 0.08182 0.2727 0.1364 1

0.01818 0.1364 0.1818 0.04505 0

3 2 1 0 X Y

0.04545 0.1227 0.4909 0.3818 Total

1 0.01818 0.2182 0.5091 0.2545 Total Example :

X and Y are related (not independent)._{0.04505≠0.3818×0.2545} 0 0 0 1 3

0 0 0.4444 0.5556 2

0 0.1667 0.5556 0.2778 1

0.0476 0.3571 0.4762 0.1190 0

3 2 1 0 X Y

1 1 1 1 Total

1 0.01818 0.2182 0.5091 0.2545 Total Example :

Conditional pmf

(

)

( )

( )

x p

y x p x y p

X ,

| =

I ndependence

Example :

0.15 0.24 0.24 0.12 40

0.05 0.08 0.08 0.04 20

80 40 20 10 X Y

1 0.20 0.32 0.32 0.16

pY(y)

0.75 0.25

pX(x)

04

.

0

25

.

0

16

.

0

.

32

×

0

.

25

=

0

.

08

0

.

32

×

0

.

25

=

0

.

08

0

.

20

×

0

.

25

=

0

.

05

0

.

16

×

0

.

75

=

0

.

12

0

.

32

×

0

.

75

=

0

.

24

0

.

32

×

0

.

75

=

0

.

24

0

.

20

×

0

.

75

=

0

.

15

0

×

=

Mathematical Expectation

Example : (X, Y) = height and weight of a random man

1 0.18 0.28 0.33 0.21

pY(y)

0.1 0.05 0.03 90

0.24 0.07

0.05 0.02 1.9

0.49 0.15

0.2 0.09 1.8

0.27 0.06

0.08 0.1 1.7

pX(x) 85

80 75 X Y

Joint distribution

( ) ( )(

X =1.7 0.27

) ( )(

+1.8 0.49

) ( )(

+1.9 0.24

)

=1.797 E

( ) ( )( ) ( )( ) ( )(

Y = 75 0.21+80 0.33+85 0.28

) ( )(

+90 0.18

)

=82.15 E

( )X =1.797 E

( )Y =82.15 E

?

2_=

    X

Y

E ( ) ( ) ( ) ( )

5191 . 25

1 . 0 9 . 1 90 07 . 0 9 . 1 85 08 . 0 7 . 1 80 1 . 0 7 . 1 75

2 2

2 2

2 =

     

+

     

+ +

     

+

     

=

     

X Y E

( ) ( )

_{Y EX} 2

E

≠

(

36X+0.8Y

) (

=36×1.7+0.8×75

)( )

0.1+

(

36×1.9+0.8×90

)( )

0.1 E

412 . 130

=

( )

X E

( )

Y E 0.8

36 +

=

Mathematical Expectation

(

X Y

)

E

( )

X E

( )

Y E3 +4 =3 +4

(

X Y XY

)

E

(

X

)

E

( )

Y E

( )

XY

E7log ₋2 2₊5 ₌7 log ₊₋2 2 ₊5

( ) ( ) ( )

XY E XEY

E ≠

In general, E

(

X Y

) ( ) ( )

≠EX EY

If X and Y are independent, then

( ) ( ) ( )

XY EXEY

E =

(

X Y

)

E

( )

X E

( )

Y

E 2 ₌ 2

(

XY

) ( ) ( )

EXE Y

E = 1

Covariance

Example : (X, Y) = height and weight of a random man

1 0.18 0.28 0.33 0.21

pY(y)

0.1 0.05 0.03 90

0.24 0.07

0.05 0.02 1.9

0.49 0.15

0.2 0.09 1.8

0.27 0.06

0.08 0.1 1.7

pX(x) 85

80 75 X Y

Joint distribution _=1.797

x

µ

15 . 82

=

y

µ

x

µ

y

µ (X−µx)(Y−µy)>0

(X−µx)(Y−µy)>0 (X−µx)(Y−µy)<0

(X−µx)(Y−µy)<0

(

)

(

)

[

X

−

x

Y

−

y

]

>

0

E

µ

µ

=

xy

σ

Covariance

Example : (X, Y) = height and weight of a random man

1 0.18 0.28 0.33 0.21

pY(y)

0.1 0.05 0.03 90

0.24 0.07

0.05 0.02 1.9

0.49 0.15

0.2 0.09 1.8

0.27 0.06

0.08 0.1 1.7

pX(x) 85

80 75 X Y

Joint distribution

( )X =1.797 E

( )Y =82.15 E

(

)

[

(

x

)

(

y

)

]

xy CovXY E X µ Y µ

σ = , = E

(

XY−

)

− µ−xµy

( ) ( )( )( ) ( )( )(XY =1.7 75 0.1+1.780 0.08)++( )( )( )1.990 0.1=147.745 E

(X,Y)=147.745−(1.797)(82.15)=0.12145

Cov +ve correlated

Covariance

0

>

xy

σ σxy<0

0

=

xy

σ σxy=0

Dependent Independent

Covariance

X and Y independent ⇒ CovE

( ) ( ) ( )

XY

(

X,=YE

)

=X0EY

⇐ ?

Example :

1/ 3 0 1/ 3 0 2

1/ 3 0 1/ 3 30

1 1/ 3

1/ 3 Marginal

0 0

0 1

2/ 3 0

1/ 3 0

Marginal 20

10 No. of dates

Income ($1000)

( ) ( )

3 40 20 2 30 0 10 0 3

1 _{× + × + × =} =

XY E

( ) ₃2 2 3 1 0 3 2_{× + × =} =

X E

( ) (10 20 30) 20 3

1 _{+ + =}

=

Y E

( ) 20 0 3 2 3 40 ,Y= − × =

X Cov

( ) ( ) Pr( 0, 10)

9 2 10 Pr 0

PrX= Y= = ≠ X= Y=

X and Y are not independent!

Correlation Coefficient

Magnitude of σxydepends on the scale of X and Y.

(

aX b cY d

)

acCov

(

XY

)

Cov + , + = ,

Standardized by standard deviations of X and Y :

(

)

( ) ( )

XVarY Var

Y X

Cov ,

(

)

= =CorrX,Y

ρ Correlation coefficient

1

1≤ ≤

− ρ

(

aX b cY d

)

sign

( )

acCorr

(

XY

)

Corr + , + = ,

X and Y independent ⇐⇒ρ= 0

Correlation Coefficient

Example : (X, Y) = height and weight of a random man

1 0.18 0.28 0.33 0.21

pY(y)

0.1 0.05 0.03 90

0.24 0.07

0.05 0.02 1.9

0.49 0.15

0.2 0.09 1.8

0.27 0.06

0.08 0.1 1.7

pX(x) 85

80 75 X Y

Joint distribution

12145 . 0

=

xy

σ

( )X =1.797 E

( )Y =82.15 E

( ) ( )

_X2 ₌1.72(0.27)₊

( )

1.82(0.49)₊

( )

1.92(0.24)₌3.2343 E ( )_{X =}3.2343₋1.7972₌0.005091

Var

( ) ( )

_Y2 ₌ 752( )0.21₊

( )

802(0.33)₊

( )

852(0.28)₊

( )

902(0.18)₌6774.25 E ( )_{Y =}6774.25₋82.152₌25.6275

Var

( ) ( ) ( )XVarY Var

Y X

Cov ,

=

ρ _{( )( )} 0.3362

6275 . 25 005091 . 0

12145 .

0 ₌

Slightly +ve correlated

Linear Combination of Random Variables

(

aX bY

)

aE

( )

X bE

( )

Y

E + = +

(

aX bY

)

aVar

( )

X bVar

( )

Y abCov

(

XY

)

Var _{+ =} 2 ₊ 2 ₊2 ,

Example : X = husband’s income Y = wife’s income

( )

X =20

E E

( )

Y =16

( )

X =60

Var Var

( )

Y =70 Cov

(

X,Y

)

=49

Total income : S = X + Y

( ) ( ) ( )

S =EX +EY =20+16=36 E

( )

S Var

( )

X Var

( )

Y Cov

(

XY

)

Var

( )

S ==60+70++2×49=+2282 ,

Var σS= 228=15.1

Linear Combination of Random Variables

Example: investment

0.4 10%

0.3 0.2

0.1 Probability

20% 0%

-10% Return on Stock

0.2 10% 0.6 0.2 Probability

8% 6% Return on Bonds

( )

S =9% E

( )

B=8% E

( ) 2

% 89

=

S Var

( ) 2

% 6 . 1

=

B Var

1 0.3 0.4 0.2 0.1 Margin

0.2 0 0 0.1 0.1 10%

0.6 0.2 0.3 0.1 0 8%

0.2 0.1 0.1 0 0 6%

Margin 20% 10% 0% -10% B S

( ) (

SB =−10

)( )( ) ( )( )( )

6 0+0 6 0++

( )( )( )

20 10 0=64 E

( )

S,B =64−9×8=−8

Cov

( )

2

% 8 ,B=− S Cov

(

p

)

B pS

R= +1− Portfolio

( )

R pE

( ) (

S p

) ( )

EB E

( )

R= p

(

+1p−

)

E

( )

R=9 +p81− E =8+

( )R pVar( ) (S p)Var( )B p( p) ( )CovSB

Var( )₌ 2 ₊(1₋)2 ( ₊2)1₋ ,

p p p p R

Var

( )

=89 2+1.61− 2−16 1− 6 . 1 2 . 19 6 .

106 2_{− +}

= p p

R Var

( )

8.09 0.8576

09 .

0 = R=

= ER σ

p

Population and Sample

Population

Sample

2

,σ

µ Population parameters

2 ,S

X Sample Statistics 2 ,S

X 2

,σ

µ

Inference

Sample

Population and Sample

Population

6

= µ

8

2₌

σ

Sample (with replacement)

Sample Mean Sample Variance

{

X1,X2

}

₂

2 1 X

X

X= +

{

(

) (

2

)

2

}

2

1 2

1 2

1

X X X X

S − + −

−

=( )

2

2 2 1

2 X X

S = −

……….

2 0 Prob = 1/25

3 2 Prob = 2/25

Population and Sample

0.08

9 10

8 7 6 5 4 3 2

0.04 0.12

0.16 0.2 0.16 0.12 0.08 0.04 Prob

X

Sampling distributions

0.08 0.16 0.24 0.32 0.2 Prob

32 18 8 2 0 2 S

( )

X =2×0.04+3×0.08++10×0.04 E

( )

X =6=µ

E

( )

2 ₌0_×0.2₊2_×0.32_{+ +}32_×0.08 S

E

( )

2 _=8=σ2 S

E Unbiased

( )

X=4⇒ Var

( )

X =2

Var Var

( )

S2 =86.4⇒ Var

( )

S2 ₌9.295

Very Simple Random Sample (VSRS)

{

X1,X2,...,Xn

}

(very simple) random sample

Each X drawn from same population (distribution)

X’s are independent

( )

X =µ

E

( )

n X Var

2

σ

= SE= Var

( )

X =σ_n standard _error

Example : Final examination scores µ=71.8 _σ2₌195.2

For a VSRS of 4 students : E( )X =µ=71.8 7.0 4

2 . 195 ₌

= =

n SE σ

For a VSRS of 16 students : E( )X =µ=71.8 3.5 16

2 . 195 ₌

= =

n SE σ

Sampling Distribution

Population

{

X1,X2

}

0.08

9 10

8 7 6 5 4 3 2

0.04 0.12

0.16 0.2 0.16 0.12 0.08 0.04 Prob

X

Sampling distributions

0.08 0.16 0.24 0.32 0.2 Prob

32 18 8 2 0 2 S

VSRS

Sampling Distribution

Population

{X1,X2}

VSRS

Dist. of 2

S

X _S2

Dist. of X

Normal Population

Population ( 2)

,σ µ

N

µ

{X1,X2,...,Xn}

VSRS

Dist. of ( 2) 2

1

σ

S n−

2 1

−

n

χ

Dist. of X      

n N

2

,σ

µ

µ

X 2

S

Normal Population

Example : measurement X , true value µ

X ~ N(µ, 0.01)

(

)



  

 

> − = > −

1 . 0 05 . 0 1 . 0 Pr 05 . 0

Pr X µ X µ =2(1−Φ( )0.5)=0.617

Take 10 measurements independently. (VSRS with n = 10)

     

10 01 . 0 , ~N µ X

(

)



  

  

> − = > −

10 1 . 0

05 . 0 10 1 . 0 Pr 05 . 0

Simple Random Sample (SRS)

{

X1,X2,...,Xn

}

simple random sample

Sampling without replacement from population (size N)

Equal probability for each possible sample

( )

X =µ

E

( )

n N

n N X Var

2

1 σ

     

− −

= ( )

n N

n N X Var

SE σ

1

− − =

= standard

error

1 1≤

− −

N n N

Finite population correction factor

Simple Random Sample

Population

6

= µ

8

2₌

σ

Sample (withoutreplacement)

Sample Mean Sample Variance

{

X1,X2

}

₂

2 1 X

X

X= + ( )

2

2 2 1

2 X X

S = −

……….

6 8 Prob = 1/10

3 2 Prob = 1/10

4 8 Prob = 1/10

Simple Random Sample

9 8 7 6 5 4 3

0.1 0.1 0.2 0.2 0.2 0.1 0.1 Prob

X

Sampling distributions

0.1 0.2 0.3 0.4 Prob

32 18 8 2 2 S

( )

X =3×0.1+4×0.1++9×0.1 E

( )

X =6=µ

E

( )

2 ₌2_×0.4₊8_×0.3_{+ +}32_×0.1

S E

( )

_S2_=10≠σ2 E

Unbiased

( )

X=3⇒ Var

( )

X =1.732

Var _Var

( )

_S2 ₌88_{⇒ Var}

( )

_S2 ₌9.38

Biased

Sampling Distribution (SRS)

Population

{X1,X2}

SRS

Dist. of 2

S

X _S2

Dist. of X

small n

X

moderate n Very large n

Central Limit Theorem

Arbitrarypopulation

(mean µµµµ, variance σσσσ2₎

{

X1,X2,...,Xn

}

VSRS

Central Limit Theorem

Arbitrarypopulation

(mean µµµµ, variance σσσσ2₎

{

X1,X2,...,Xn

}

VSRS

( )

→∞

→  −

n N

n

X L

as 1 , 0

σ µ

( ) 

     − Φ ≈ ≤

n c c X

σ µ Pr

For large n

( ) 

     − Φ = ≤c c_σµ

X ?

Pr

Normal Approximation

Example : Monthly income

0.05 20000

0.03 30000

0.01 40000

0.01 0.1

0.15 0.25 0.4 Prob

60000 15000

10000 7000 4000 Income

( ) (

=4000

)( ) (

0.4+7000

)(

0.25

)

+ +

(

60000

)(

0.01

)

=EX

µ=E

( )

X =9250

µ

( ) (

2 ₌40002

)

( )

0.4 ₊

(

70002

)

(

0.25

)

_{+ +}

(

600002

)

(

0.01

)

X

E

( )

2 ₌155150000 X

E2

( )

2

9250 155150000−

= =VarX

σ2₌

( )

₌69587500 X

Var

σ

( )

=8342

= VarX

σ

For a VSRS with size 100 

  

 

100 8342 , 9250 ~

2 .

N X

(

)

1 (0.90) 0.1841

10 8342

9250 10000 1 10000

Pr = −Φ =

  

  − Φ − ≈ >

X

( ) 1 (0.09) 0.4641 8342

9250 10000 1 10000

Pr = −Φ =

  



 −

Φ − ≈ >

X

(

10000

)

0.1 0.05 0.03 0.01 0.01 0.2

PrX> = + + + + =

Normal Approximation

Normal approximate binomial

( ),π

~bn

Y nlarge

(1 ) ( )~ 0,1

. N n

n Y

π π

π

− −

Example : Y~b(35,0.25)

( )

Y =35×0.25=8.75

E Var

( )

Y =35×0.25×0.75=6.5625

(

)



     − Φ −

      − Φ ≈ ≤ ≤

5625 . 6

75 . 8 7 5625 . 6

75 . 8 15 15 7

Pr

(

7 Y 15

) (

2.440

) (

0.683

)

Pr

(

7≤Y≤15

)

≈Φ0.9927

(

−1Φ0−.7527

)

0.7454

Pr ≤Y≤ ≈ − − =

By exact calculation,

(

7 15

)

(

0.25

) (

0.75

)

0.8018 Pr

15

7

35

35 =

= ≤

≤

∑

=

−

y

y y

y

C Y

Normal Approximation

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16

0 5 10 15 20 25 30 35

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16

0 5

7

10 15 20 25 30 35

(

)

(

7 35,0.25 15

)

PrPr(7≤≤Nb(8.75,6.5625≤)≤15)

6.5 Continuity correction 15.5

Continuity Correction

Approximate discrete distribution by continuous distribution

(

X≤c

)

Pr Pr

(

X≤c+0.5

)

(

X<c

)

Pr Pr

(

X≤c−0.5

)

(

X≥c

)

Pr Pr

(

X≥c−0.5

)

(

X>c

)

Pr Pr

(

X≥c+0.5

)

Example : Y~b(35,0.25) E( )X =8.75 Var( )X =6.5625

(

)



   

 −

Φ −

   



 −

Φ ≈ ≤ ≤

5625 . 6

75 . 8 5 . 6 5625 . 6

75 . 8 5 . 15 15 7

PrPrPr

(

(

77≤≤YYY≤≤1515

) (

)

≈≈Φ0.99582.635−

) (

(

−1−Φ0−.81000.878

)

=

)

0.8058

By exact calculation, Pr(7≤Y≤15)=0.8018

Normal Approximation

Normal approximate Poisson

( )θ ℘

~

Y θlarge Y _~._N( )_0,1

θ θ −

Example : Y~℘( )30

(

)



   

 −

Φ −

    

 −

Φ ≈ ≤ ≤

30 30 5 . 23 30

30 5 . 39 39 24

Pr

(

24 Y 39

) (

1.734

) (

1.187

)

PrPr

(

24≤≤YY≤≤39

)

≈≈Φ0.9586−−

(

1−Φ0−.8824

)

=0.841

By exact calculation,

(

)

0.8391

! 30 39

24 Pr

39

24 30

= =

≤

≤

∑

= −

y y

y e Y

By normal approximation with continuity correction,

Normal Approximation

Normal approximate Chi-Square

2

~ r

Y χ rlarge _~ ( )_0,1 2

. N r

r Y−

Example : 2 80

~χ Y

(

)



  



 −

Φ −

   



 −

Φ ≈ ≤ ≤

160 80 39 . 60 160

80 58 . 96 58 . 96 39 . 60

Pr

(

60.39 Y 96.58

) (

1.311

) (

1.550

)

Pr

(

60.39≤Y≤96.58

)

≈Φ0.9051

(

−1Φ0−.9394

)

0.8445

Pr ≤Y≤ ≈ − − =

From Chi-Square distribution table,

(

60.39 96.58

)

0.9 0.05 0.85

Pr ≤Y≤ = − =