Two Types of Errors

(1)

Testing Hypothesis

  

≠ =

0 :

0 : 1 0

µ µ

H H

Statistical Hypothesis A statement about population

Null hypothesis : H0 Alternative hypothesis : H1

  

≠ =

1 :

2 1

2 0

σ σ

H H

  

≠ =

2 1 1

2 1 0

2 :

π π

H H

    

≠ =

2 2 1

2 2 0

: :

y x

H H

σ σ

  

on. distributi normal a not is Population :

on. distributi normal a is Population :

1 0

H H

Testing Hypothesis

Test Procedure (based on sample) leads to rejection or non-rejection of the hypothesis

Example : Test H0:µ=1 vs H1:µ≠1.

" . 0 or 2 if Reject

" H0 X> X< is a reasonable test.

Draw a sample _X ₌₂_.₃ Reject H0 .

Conclude that µ ≠1 .

But we may be wrong !

Two Types of Errors

0

H

1

H

0

H

0

H

Correct Type I error

Reject H0

Type I I error Correct

Do not reject H0

H1True

H0True

(

TypeIerror

)

Pr

(

Reject | true

)

Pr = H0 H0

= α

(

TypeIIerror

)

Pr

(

Donot reject | true

)

Pr = H0 H1

=

β

Two Types of Errors

Example : life time of light bulbs with σ= 300 hours

1240 : vs 1200

: 1

0 µ= H µ=

H

Test "Reject H0ifX>1249."      

100 300 , ~

2 µ N X

Draw a sample with size 100.

(

Reject | true

)

Pr H0 H0

=

αα=Pr

(

X>1249|µ=1200

)

   

 

 ₋

Φ − =

100 300

1200 1249 1

(

1.633

)

1−Φ

=0.0513 =

(

Donot reject | true

)

Pr H0 H1

=

β=Pr

(

≤1249|µ=1240

)

β X _

  

 

 ₋

Φ =

100 300

1240 1249

( )

0.3

Φ =0.6179 =

6179 . 0

= β

Two Types of Errors

1200 1240

H₀ H₁

1249 0513 . 0

= α

increased

β

reduced

α

X of on distributi Sampling

Type II error Jail an innocence.

Choice of

H

0

and

H

1

α

β

α

β

α

β

α

β

ββ

For fix n, we can only control one of the errors.

Convention : Control α.

Criterion 1 : Make Type I error a more serious error.

Example : Want to know if a man is guilty.

guilty. is He :

0 H

guilty. not is He :

1 H

Type I error Release a criminal.

More serious error

Type II error Jail an innocence.

guilty. not is He :

0 H

guilty. is He :

1 H

Type I error Jail an innocence.

Type II error Release a criminal.

(2)

Choice of

H

₀

and

H

₁

Convention : Control αto be small.

Criterion 2 : Put what you want to prove in H1.

Pr (false rejection of H0) = Pr (reject H0| H0true) = α

Pr (false acceptance of H0) = Pr (accept H0| H1true) = β

May be large

Reject H0 Strong conclusion

Accept H0 Weak conclusion

Do not reject

Choice of

H

₀

and

H

₁

Criterion 2 : Put what you want to prove in H1.

Example : Want to show that Brand A is more popular than Brand B.

H0: B is more popular than A.

H₁: A is more popular than B.

If reject H0 Data shows strong evidence to against H0 ,

i.e., high confidence that H₁is true.

If do not reject H0 Not enough evidence to against H0 ,

but may not have high confidence on H0.

Choice of

H

0

and

H

1

Example : Clinical trials on drug design.

Standard medicine : cure rate π0= 0.6 New medicine : believe that cure rate π > 0.6

6 . 0 :

0 π≤

H

6 . 0 :

1 π>

H

(new medicine is not better)

(new medicine is better)

Criterion 1 : Type I error (switch to worse medicine)_{Type II error (abandon better medicine)} more serious

Criterion 2 : What we want to prove is H1.

Construction of Test Procedure

1240 : vs 1200

: 1

0 µ= H µ=

H

     

100 300 , ~

2 µ N X

Preset α= 0.05.

Reasonable test ""Reject Reject H0Hif0Xifis tooX large> c."."

(Reject | true)

Pr 05 .

0.05=α=Pr

(

|H0 1200H0

)

0 =α= X>c µ= 

    

 ₋

Φ − = =

100 300

1200 1

05 .

0 α c 1.645

30 1200

05 . 0 = = − ⇒c=1249.35Z

⇒c

" . 35 . 1249 if Reject

" H0 X> Constructed before observing data.

Construction of Test Procedure

1240 : vs 1200

: 1

0 µ= H µ=

H

     

100 300 , ~

2 µ N X

Preset α= 0.05.

(Donot reject | true)

Pr H0 H1

=

ββ=Pr

(

X≤1249.35|µ_=1240

)

  

   ₋

Φ =

100 300

1240 35 . 1249

β ==Φ0(.06225.312)

" . 35 . 1249 if Reject " H0 X>

(

Reject | true

)

1 0.3775

Pr H0 H1 = −β= Power of the test.

Construction of Test Procedure

1240 : vs 1200

: 1

0 µ= H µ=

H

     

400 300 , ~

2 µ N X

Preset α= 0.05.

" . if Reject

" H0 X>c

(Reject | true)

Pr 05 .

0.05=α=Pr

(

|H0 1200H0

)

0 =α= X>c µ= _

      ₋

Φ − = =

400 300

1200 1

05 .

0 α c 1.645

15 1200

05 . 0 = = −

⇒⇒cc=1224.675Z

(Donot reject | true)

Pr H0 H1

=

β=Pr

(

≤1224.675|µ=1240

)

β X _

  

 

 ₋

Φ =

400 300

1240 675 . 1224

(3)

Construction of Test Procedure

1240 : vs 1200

: 1

0 µ= H µ=

H

Preset α= 0.05. Preset power = 0.9 , i.e. β= 0.1

05 . 0 1200 | 300

1200 300

1200

Pr =

  

  

= − >

− _µ

n c n X

1 . 0 1240 | 300

1240 300

1240

Pr _=

  

  

= − ≤

− _µ

n c n X

645 . 1 300

1200₌

−

n c

282 . 1 300

1240₌₋

−

n c

48 . 1222 ,

482 =

= c

n

Significance Probability

1240 : vs 1200

: 1

0 µ= H µ=

H

482

=

n

"Reject H0ifX>1222.48."

1223

=

X Reject H0.

1237

=

X Reject H0.

Stronger evidence

Data strongly disagree with H0.

Measure of agreement between data and H0

significance probability / p-value

1200 1223

0462 . 0 = −value p

(

1223

)

of =

−value X p

0462 . 0 482 300

1200 -1223

-1 =

      Φ =

Significance Probability

smallest αfor which it leads to the rejection of H0

significance probability of an observation

1240 : vs 1200

: 1

0 µ= H µ=

H n=482

1237

00342 . 0 = −value p

(

1237

)

of =

−value X p

00342 . 0 482 300

1200 -1237

-1 _=

  

  

Φ =

Significance Probability

αααα

critical value compute

c

p-value

compute

p-value

data

X

compare

c X>

compare

α < −value p

Reject H0if

data fall in rejection regionp-value< α ⇔

Reject H0

0

Large Sample, Known Variance

0 1 0

0:µ=µ vs H:µ≠µ

H σ2 _known

n X Z

σ µ0 − =

Test statistic

. if level ce significan at

Reject H0 α Zobs>Zα2

Zobs

If Z_obs> 0 ,

p-value= 2(1-Φ(Z_obs))

Zobs

If Z_obs< 0 ,

p-value= 2Φ(Z_obs)

0

Large Sample, Known Variance

0 1 0

0:µ≤µ vs H :µ>µ

H σ2 _known

n X Z

σ µ0 − =

Test statistic

Reject H0 α Zobs>Zα

Zobs

p-value= 1-Φ(Z_obs)

Zobs

p-value= Φ(Z_obs) 0

1 0

0:µ≥µ vs H:µ<µ

H

(4)

Large Sample, Known Variance

100 5

28

− =X Z Test statistic

. 1.645

if 05 . 0 level ce significan at

Reject H0 α= Zobs>Z0.05=

Example : package delivery time with known σ= 5 mins

28 : vs 28

: 1

0 µ≤ H µ>

H α=0.05

Sample with n = 100 _X₌31.5 , _S2₌28.9

645 . 1 7 100 5

28 5 . 31

> = − =

obs

Z Reject H0at α= 0.05.

( )

7 0

1−Φ ≈

= −value p

Small Sample, Normal Population,

Unknown Variance

0 1 0

0:µ=µ vs H:µ≠µ

H

n S X T₌ −µ0

Test statistic

.

if level ce significan at

Reject H0 α Tobs>tn−1,α2 σ2 _unknown

Normal population

0 1 0

0:µ≤µ vs H :µ>µ

H

Reject H0 α Tobs>tn−1,α

0 1 0

0:µ≥µ vs H:µ<µ

H

.

Reject H0 α Tobs<−tn−1,α

Small Sample, Unknown Variance

10 28

S X T= − Test statistic

. 1.833

if 05 . 0 level ce significan at

Reject H0 α= Tobs>t9,0.05= Example : package delivery time _X_~_N

(

µ_,σ2

)

28 : vs 28

: 1

0 µ≤ H µ>

H α=0.05

Sample with n = 10 _X₌32.3 , _S2₌46.3

833 . 1 998 . 1 10 3 . 46

28 3 . 32

> = − =

obs

T Reject H0at α= 0.05.

(

1.998

)

0.0384 Pr 9> = =

−value t

p

Two Samples Problem

(Large Samples)

y x y

x H

H0:µ =µ vs 1:µ ≠µ

n m

Y X Z

y x

2

2 _σ

σ + − =

Test statistic

2 , x xσ

µ

2 , y yσ

µ

Sample with size m

Sample with size n Independent

y x y

x H

H0:µ ≤µ vs 1:µ >µ

y x y

x H

H0:µ ≥µ vs 1:µ <µ

Reject H0 α Zobs<−Zα

n S m S

Y X Z

y x

2

2 ₊

− =

Two Samples Problem

(Small Samples)

y x y

x H

n m S

Y X T

pool 1 +1 − = Test statistic

.

Reject H0 α Tobs>tm+n−2,α2

(

_, 2

)

x x Nµ σ

( _, 2)

y y

Nµ σ

Sample with size m

Sample with size n Independent 2

2 y x σ

σ =

y x y

x H

H0:µ ≤µ vs 1:µ >µ

.

Reject H0 α Tobs>tm+n−2,α

y x y

x H

H0:µ ≥µ vs 1:µ <µ

.

Reject H0 α Tobs<−tm+n−2,α

Two Samples Problem

(Small Samples)

Example : 4 scores from class A : 64, 66, 89, 77 3 scores from class B : 56, 71, 53

Assumptions :

(i) Normal populations. (ii) Independent samples. (iii) Equal variances.

3 1 4

1 +

− =

pool S

Y X

T _RejectH₀_atα=₀_.₀₅_ifT_obs>t₅_,₀_.₀₂₅=₂_.₅₇₀₆_.

y x y

x H

H0:µ =µ vs 1:µ ≠µ α=0.05

93 , 60 , 667 . 132 ,

74 2= = 2=

= Sx Y Sy

X ( )( ) ( )( ) 117

2 3

93 2 667 . 132 3

2 ₌

+ + =

pool S

( )(11714 13) 1.695 2.5706

60 74

< = + − = obs

(5)

Two Samples Problem

(Small Samples)

y x y

x H

n S m S

Y X T

y x

2

2 ₊

− =

Reject H0 α Tobs>tv,α2

(

_, 2

)

x x Nµ σ

( 2)

, y y

Nµ σ

Sample with size m

Sample with size n Independent 2

2 y

x σ

σ ≠

(

)

( )

(

2 1

)

4

(

2( 1)

)

4

2 2 2

− + −

+ =

n n S m m S

n S m S v

y x

Paired Data

Example : effect of stimulus on blood pressure

135 129 132 118 137 141 125 132 127 131 131 128 After (Y)

127 126 130 126 135 140 128 140 118 130 124 120 Before (X)

12 11 10 9 8 7 6 5 4 3 2 1 Man

. 0.05 at : vs :

Test H0 µx=µy H1 µx≠µy α=

X and Y are dependent Two sample T-Test

X

Y

D

=

−

µD=E

( )

D =µy−µx 8 3 2 -8 2 1 -3 -8 9 1 7 8 D = Y - X

. 0.05 at 0 : vs 0 :

Test H0

µ

D= H1

µ

D≠

α

=

Paired Data

Example : effect of stimulus on blood pressure

12 11 10 9 8 7 6 5 4 3 2 1 Man

8 3 2 -8 2 1 -3 -8 9 1 7 8 D = Y - X

. 0.05 at 0 : vs 0 :

Test H0 µD= H1 µD≠ α=

12 0

D

S D T= −

Test statistic Reject H0 ifTobs >t11,0.025=1.796

83 . 5 , 833 .

1 =

= SD

D 1.09 1.796

12 83 . 5

833 .

1 ₌ _<

= obs

T

Do not reject H0at α= 0.05.

95% C.I. for µD= µy-µx : 11,0.025 ₁₂ D

S t D±

12 83 . 5 796 . 1 833 .

1 ± =1.833±3.023 0

Population Proportion

0 1 0

0:π=π vs H:π≠π

H

Zobs

If Zobs> 0 ,

p-value= 2(1-Φ(Zobs))

Zobs

If Zobs< 0 ,

p-value= 2Φ(Zobs)

( )n

Z

π π

− − =

1 0

Test statistic

n X = π

0

Population Proportion

0 1 0

0:π≤π vs H:π>π

H

Zobs

p-value= 1-Φ(Z_obs)

Zobs

p-value= Φ(Z_obs) 0

1 0

0:π≥π vs H:π<π

H

( )n

Z

π π

− − =

1 0

Test statistic

n X =

π

Population Proportion

(1 )224 75 . 0

π π

π

− − =

Z

Test statistic

Example : Crosses of peas π= Pr(Yellow pea)

75 . 0 : vs 75 . 0

: 1

0 π= H π≠

H α=0.05

(

0.7857

)(

0.2143

)

224 1.3021 1.96 75

. 0 7857 . 0

025 .

0 =

< = −

= Z

Zobs

Do not reject H0at α= 0.05.

(

)

(

1 1.0321

)

0.1928

2 −Φ =

= −value p

Experiment : 176 Yellow, 48 Green

7857 . 0 224

176₌

=

(6)

Population Proportions

Independent binomial random variables

(

1, 1

)

~bnπ

X Y~b

(

n2,π2

)

. : vs :

Test H0 π1=π2 H1 π1≠π2

Test statistic

( )      

+ −

− =

2 1 2 1

1 1 1

n n Z

π π

I I

1 1

n X =

πI

2 2

n Y =

πI

2 1 n

n Y X

++ =

πI

Reject H0 α Zobs >Zα2

. : vs :

Test H0 π1≤π2 H1 π1>π2

. : vs :

Test H0 π1≥π2 H1 π1<π2

(

)

(

)

2 2 2

1 1 1

2 1

1 1

n n

Z

π π π π

π π

I I I I

I I

− + −

− =

Population Proportions

Example : Graduation rate

210 198 Tot al

52 9 Not Graduat ed

158 189 Graduated

Non- I SP I SP

9545 . 0 198 189₌ = ISP πI

7524 . 0 210 158₌ = N πI

N ISP N

ISP H

H0:π ≤π vs 1:π >π α=0.05

8505 . 0 210 198

158

189 ₌

+ + =

πI

(0.8505)(0.1495)(1198 1210) 5.7216 1.645

7524 . 0 9545 . 0

05 . 0 =

> = + −

= Z

Zobs

Reject H0at α= 0.05.

(

0.9545

)(

0.0455

)

198

(

0.7524

)(

0.2476

)

210 6.0757 1.645 7524

. 0 9545 . 0

05 . 0 = > = +

−

= Z