Directory UMM :wiley:Public:college:hill:

(1)

Copyright 1996 Lawrence C. Marsh

PowerPoint Slides

for

Undergraduate Econometrics

by

Lawrence C. Marsh

To accompany: Undergraduate Econometrics

(2)

The Role of

Econometrics

in Economic Analysis

Chapter 1

Copyright © 1997 John Wiley & Sons, Inc. All rights reserved. Reproduction or translation of this work beyond that permitted in Section 117 of the 1976 United States Copyright Act without the express written permission of the copyright owner is unlawful. Request for further information should be addressed to the Permissions Department, John Wiley & Sons, Inc. The purchaser may make back-up copies for his/her own use only and not for distribution or resale. The Publisher assumes no responsibility for errors, omissions, or damages, caused by the use of these programs or from the use of the information contained herein.

(3)

Copyright 1996 Lawrence C. Marsh

Using Information:

1. Information from

economic theory.

2. Information from

economic data.

The Role of Econometrics

(4)

Copyright 1996 Lawrence C. Marsh

Understanding Economic Relationships:

federal budget

Dow-Jones Stock Index

trade

deficit _{Federal Reserve}

Discount Rate capital gains tax

(5)

economic theory

economic data

}

economic decisions

To use information effectively:

*Econometrics* helps us combine

economic theory and economic data .

(6)

Copyright 1996 Lawrence C. Marsh

Consumption, c, is some function of income, i :

c = f(i)

For applied econometric analysis this consumption function must be specified more precisely.

(7)

demand, qd, for an individual commodity: qd_{= f( p, p}c_{, p}s_{, i )}

supply, qs, of an individual commodity: qs_{= f( p, p}c_{, p}f₎

p = own price; pc_{= price of complements;}

ps_{= price of substitutes; i = income}

p = own price; pc_{= price of competitive products;}

ps_{= price of substitutes; p}f_{= price of factor inputs}

demand

supply

(8)

Copyright 1996 Lawrence C. Marsh

Listing the variables in an economic relationship is not enough. For effective policy we must know the amount of change

needed for a policy instrument to bring about the desired effect:

How much ?

• By how much should the Federal Reserve raise interest rates to prevent inflation?

• By how much can the price of football tickets be increased and still fill the stadium?

(9)

Copyright 1996 Lawrence C. Marsh

Answering the How Much? question

Need to estimate parameters that are both:

1. unknown

and

2. unobservable

(10)

Average or systematic behavior

over many individuals or many firms. Not a single individual or single firm. Economists are concerned with the

unemployment rate and not whether a particular individual gets a job.

The Statistical Model

(11)

Copyright 1996 Lawrence C. Marsh

The Statistical Model

Actual vs. Predicted Consumption:

Actual = systematic part + random error

Systematic part provides prediction, f(i), but actual will miss by random error, e.

Consumption, c, is function, f, of income, i, with error, e:

c = f(i) + e

(12)

Copyright 1996 Lawrence C. Marsh

c = f(i) + e

Need to define f(i) in some way. To make consumption, c,

a linear function of income, i : f(i) = β₁ + β₂ i

The statistical model then becomes: c = β₁ + β₂ i + e

(13)

• Dependent variable, y, is focus of study

(predict or explain changes in dependent variable).

• Explanatory variables, X₂ and X₃, help us explain observed changes in the dependent variable.

y = β₁ + β₂ X₂+ β₃ X₃+ e

The Econometric Model

(14)

Copyright 1996 Lawrence C. Marsh Statistical Models

Controlled (experimental) vs.

Uncontrolled (observational)

Uncontrolled experiment (econometrics) explaining consump-tion, y : price, X₂, and income, X₃, vary at the same time. Controlled experiment (“pure” science) explaining mass, y : pressure, X₂, held constant when varying temperature, X₃, and vice versa.

(15)

Econometric

model

• economic model

economic variables and parameters. • statistical model

sampling process with its parameters. • data

observed values of the variables.

(16)

Copyright 1996 Lawrence C. Marsh

• Uncertainty regarding an outcome.

• Relationships suggested by economic theory. • Assumptions and hypotheses to be specified. • Sampling process including functional form. • Obtaining data for the analysis.

• Estimation rule with good statistical properties. • Fit and test model using software package.

• Analyze and evaluate implications of the results. • Problems suggest approaches for further research.

The Practice of Econometrics

(17)

Copyright 1996 Lawrence C. Marsh

Note: the textbook uses the following symbol to mark sections with advanced material

:

“Skippy”

(18)

Some Basic

Probability

Concepts

Chapter 2

(19)

Copyright 1996 Lawrence C. Marsh

random variable:

A variable whose value is unknown until it is observed. The value of a random variable results from an experiment. The term random variable implies the existence of some known or unknown probability distribution defined over the set of all possible values of that variable.

In contrast, an arbitrary variable does not have a probability distribution associated with its values.

(20)

Controlled experiment values

of explanatory variables are chosen with great care in accordance with an appropriate experimental design.

Uncontrolled experiment values

of explanatory variables consist of nonexperimental observations over which the analyst has no control.

(21)

Copyright 1996 Lawrence C. Marsh

discrete random variable:

A discrete random variable can take only a finite number of values, that can be counted by using the positive integers.

Example: Prize money from the following lottery is a discrete random variable:

first prize: $1,000 second prize: $50 third prize: $5.75

since it has only four (a finite number) (count: 1,2,3,4) of possible outcomes:

$0.00; $5.75; $50.00; $1,000.00

(22)

Copyright 1996 Lawrence C. Marsh

continuous random variable:

A continuous random variable can take any real value (not just whole numbers) in at least one interval on the real line.

Examples:

Gross national product (GNP) money supply

interest rates price of eggs

household income

expenditure on clothing

(23)

A discrete random variable that is restricted to two possible values (usually 0 and 1) is called a dummy variable (also, binary or indicator variable).

Dummy variables account for qualitative differences: gender (0=male, 1=female),

race (0=white, 1=nonwhite),

citizenship (0=U.S., 1=not U.S.), income class (0=poor, 1=rich).

(24)

Copyright 1996 Lawrence C. Marsh

A list of all of the possible values taken by a discrete random variable along with

their chances of occurring is called a probability function or probability density function (pdf).

die x f(x)

one dot 1 1/6 two dots 2 1/6 three dots 3 1/6 four dots 4 1/6 five dots 5 1/6 six dots 6 1/6

(25)

A discrete random variable X

has pdf, f(x), which is the probability

that X takes on the value x.

f(x) = P(X=x)

0 < f(x) < 1

If X takes on the n values: x₁, x₂, . . . , x_n, then f(x₁) + f(x₂)+. . .+f(x_n) = 1.

Therefore,

(26)

Copyright 1996 Lawrence C. Marsh

Probability, f(x), for a discrete random

variable, X, can be represented by height:

0 1 2 3 X

number, X, on Dean’s List of three roommates f(x)

0.2

0.4

0.1

0.3

(27)

A continuous random variable uses

area under a curve rather than the

height, f(x), to represent probability: f(x)

X $34,000 $55,000

. .

per capita income, X, in the United States

0.1324 0.8676

red area green area

(28)

Copyright 1996 Lawrence C. Marsh

Since a continuous random variable has an

uncountably infinite number of values, the probability of one occurring is zero.

P [ X = a ] = P [ a < X < a ] = 0

Probability is represented by area. Height alone has no area.

An interval for X is needed to get an area under the curve.

(29)

P [ a < X < b ] =

∫

f(x) dx

b a

The area under a curve is the integral of the equation that generates the curve:

For continuous random variables it is the

integral of f(x), and not f(x) itself, which

defines the area and, therefore, the probability.

(30)

Copyright 1996 Lawrence C. Marsh

n

Rule 2: Σ ax_i = a Σ x_i

i = 1 i = 1

n

Rule 1: Σ x_i = x₁ + x₂ + . . . + x_n

i = 1

n

Rule 3: Σ (x_i + y_i) = Σ x_i + Σ y_i

i = 1 _{i = 1} _{i = 1}

n n n

Note that summation is a linear operator which means it operates term by term.

(31)

Rule 4: Σ (ax_i + by_i) = a Σ x_i + b Σ y_i

i = 1 _{i = 1} _{i = 1}

n n n

Rules of Summation (continued)

Rule 5: x = Σ x_i =

i = 1

n n

1 x₁ + x₂ + . . . + x_n

n

The definition of x as given in Rule 5 implies the following important fact:

Σ (x_i − x) = 0

i = 1

n

(32)

Copyright 1996 Lawrence C. Marsh

Rule 6: Σ f(x_i) = f(x₁) + f(x₂) + . . . + f(x_n)

i = 1

n

Notation: Σ f(x_i) = Σ f(x_i)= nΣ f(x_i)

x i i = 1

n

Rule 7: Σ Σ f(x_i,y_j) = Σ [ f(x_i,y₁) + f(x_i,y₂)+. . .+ f(x_i,y_m)]

i = 1 i = 1

n m

j = 1

The order of summation does not matter :

Σ Σ f(x_i,y_j) = Σ Σ f(x_i,y_j)

i = 1

n m

j = 1 j = 1

m n

i = 1

Rules of Summation (continued)

(33)

Copyright 1996 Lawrence C. Marsh

The

mean

or arithmetic average of a

random variable is its mathematical

expectation or expected value, EX.

The Mean of a Random Variable

(34)

Copyright 1996 Lawrence C. Marsh

Expected Value

There are two entirely different, but mathematically equivalent, ways of determining the expected value:

1. Empirically:

The expected value of a random variable, X, is the average value of the random variable in an infinite number of repetitions of the experiment.

In other words, draw an infinite number of samples, and average the values of X that you get.

(35)

Copyright 1996 Lawrence C. Marsh

Expected Value

2. Analytically:

The expected value of a discrete random variable, X, is determined by weighting all the possible values of X by the corresponding probability density function values, f(x), and summing them up.

E[X] = x₁f(x₁) + x₂f(x₂) + . . . + x_nf(x_n)

In other words:

(36)

Copyright 1996 Lawrence C. Marsh

In the empirical case when the

sample goes to infinity the values

of X occur with a frequency

equal to the corresponding f(x)

in the analytical expression.

As sample size goes to infinity, the

empirical and analytical methods

will produce the same value.

(37)

Copyright 1996 Lawrence C. Marsh

x =

Σ

n x_i

i = 1

where n is the number of sample observations.

Empirical (sample) mean:

E[X] =

Σ

x_i f(x_i)

i = 1

n

where n is the number of possible values of x_i.

Analytical mean:

Notice how the meaning of n changes.

(38)

E X =

Σ

x_if(x_i)

i=1

n

The expected value of X-squared:

E X =

Σ

x_if(x_i)

i=1

n

2 ₂

It is important to notice that f(x_i) does not change!

The expected value of X-cubed:

E X =

Σ

x_if(x_i)

i=1

n

3 ₃

(39)

EX = 0 (.1) + 1 (.3) + 2 (.3) + 3 (.2) + 4 (.1)

2

EX = 0 (.1) + 1 (.3) + 2 (.3) + 3 (.2) + 4 (.1)2 2 2 2 2

= 1.9

= 0 + .3 + 1.2 + 1.8 + 1.6

= 4.9

3

EX = 0 (.1) + 1 (.3) + 2 (.3) + 3 (.2) +4 (.1)3 3 3 3 3 = 0 + .3 + 2.4 + 5.4 + 6.4

= 14.5

(40)

E[g(X)]

=

Σ

g

(

x_i

)

f(x_i)

n

i = 1

g(X) = g₁(X) + g₂(X)

E[g(X)]

=

Σ [

g₁(x_i) + g₂(x_i)] f(x_i)

n

i = 1

E[g(X)]

=

Σ

g₁(x_i) f(x_i) +

Σ

g₂(x_i) f(x_i)

n

i = 1

n

i = 1

E[g(X)]

=

E[g₁(X)]+ E[g₂(X)]

(41)

Adding

and

Subtracting

Random Variables

E(X-Y) = E(X) - E(Y)

E(X+Y) = E(X) + E(Y)

(42)

Copyright 1996 Lawrence C. Marsh

E(X+a) = E(X) + a

Adding

a

constant

to a variable will

add a constant to its expected value:

Multiplying by

constant

will multiply

its expected value by that constant:

E(bX) = b E(X)

(43)

Copyright 1996 Lawrence C. Marsh

var(X) = average squared deviations around the mean of X.

var(X) = expected value of the squared deviations around the expected value of X.

var(X) = E [(X - EX) ] 2

(44)

var(X) = E [(X - EX) ]

= E [X - 2XEX + (EX) ]

2

= E(X ) - 2 EX EX + E (EX)

2

= E(X ) - 2 (EX) + (EX) 2 2 2 = E(X ) - (EX) 2 2

var(X) = E [(X - EX) ] 2

var(X) = E(X ) - (EX) 2 2

(45)

variance

of a discrete

random variable, X:

standard deviation is square root of variance

var ( X ) =

(x

_i

- EX )

2

f(x

_i

)

i = 1 n

∑

(46)

Copyright 1996 Lawrence C. Marsh

x_i f(x_i) (x_i - EX) (x_i - EX) f(x_i)

2 .1 2 - 4.3 = -2.3 5.29 (.1) = .529

3 .3 3 - 4.3 = -1.3 1.69 (.3) = .507

4 .1 4 - 4.3 = - .3 .09 (.1) = .009

5 .2 5 - 4.3 = .7 .49 (.2) = .098

6 .3 6 - 4.3 = 1.7 2.89 (.3) = .867

Σ x_i f(x_i) = .2 + .9 + .4 + 1.0 + 1.8 = 4.3

Σ (x_i - EX) f(x_i) = .529 + .507 + .009 + .098 + .867 = 2.01

2

calculate the variance for a discrete random variable, X:

i = 1

n n

i = 1

(47)

Z = a + cX

var(Z) = var(a + cX)

= E [(a+cX) - E(a+cX)]

= c var(X)

2 2

var(a + cX) = c var(X)

2

(48)

Copyright 1996 Lawrence C. Marsh

A

joint

probability density function,

f(x,y), provides the probabilities

associated with the joint occurrence

of all of the possible pairs of X and Y.

(49)

.15

.05

.45

.35

joint pdf

f(x,y)

Y = 1 Y = 2

vacation homes

owned

X = 0

X = 1

Survey of College City, NY

f

(0,1)

f

(0,2)

f

(1,1)

f

_(1,2)

(50)

Copyright 1996 Lawrence C. Marsh

E[g(X,Y)] =

Σ

g(x

_i

,y

_j

) f(x

_i

,y

_j

)

i _j

E(XY) = (0)(1)(.45)+(0)(2)(.15)+(1)(1)(.05)+(1)(2)(.35)=.75

E(XY) =

Σ

x

_i

y

_j

f(x

_i

,y

_j

)

i _j

Calculating the expected value of

functions of two random variables.

(51)

Copyright 1996 Lawrence C. Marsh

The

marginal

probability density functions,

f(x) and f(y), for discrete random variables,

can be obtained by summing over the f(x,y)

with respect to the values of Y to obtain f(x)

with respect to the values of X to obtain f(y).

f(x

_i

) =

Σ

f(x

_i

,y

_j

)

f(y

_j

) =

Σ

f(x

_i

,y

_j

)

i j

(52)

Copyright 1996 Lawrence C. Marsh

.15

.05

.45

.35

marginal

Y = 1 Y = 2

X = 0

X = 1

.60

.40

.50

f

(X = 1)

f

(X = 0)

f

(Y = 1)

f

(Y = 2)

marginal pdf for Y:

marginal pdf for X:

(53)

Copyright 1996 Lawrence C. Marsh

The

conditional

probability density

functions of X given Y=y , f(x

|

y),

and of Y given X=x , f(y

|

x),

are obtained by dividing f(x,y) by f(y)

to get f(x

|

y) and by f(x) to get f(y

|

x).

f(x

|

y) =

f(x,y)

f(y

|

x) =

f(x,y)

f(y)

f(x)

(54)

Copyright 1996 Lawrence C. Marsh

.15

.05

.45

.35

conditonal

Y = 1 _{Y = 2}

X = 0

X = 1

.60

.40

.50

.25 .75 .875 .125 .90 .10 .70 .30

f(Y=2_|X= 0)=.25

f(Y=1_|X = 0)=.75

f(Y=2_|X = 1)=.875

f(X=0_|Y=2)=.30

f(X=1_|Y=2)=.70

f(X=0_|Y=1)=.90

f(X=1_|Y=1)=.10

f(Y=1_|X = 1)=.125

(55)

Copyright 1996 Lawrence C. Marsh

X and Y are

independent

random

variables if their joint pdf, f(x,y),

is the product of their respective

marginal pdfs, f(x) and f(y) .

f(x

_i

,y

_j

) = f(x

_i

) f(y

_j

)

for independence this must hold for all pairs of i and j

(56)

Copyright 1996 Lawrence C. Marsh

.15

.05

.45

.35

not independent

Y = 1 Y = 2

X = 0

X = 1

.60

.40

.50

f

(X = 1)

f

(X = 0)

f

(Y = 1)

f

(Y = 2)

marginal pdf for Y:

marginal pdf for X: .50x.60=.30 .50x.60=.30

.50x.40=.20 .50x.40=.20

The calculations in the boxes show the numbers

required to have

independence.

(57)

Copyright 1996 Lawrence C. Marsh

The

covariance

between two random

variables, X and Y, measures the

linear association between them.

cov(X,Y) = E[(X - EX)(Y-EY)]

Note that variance is a special case of covariance.

cov(X,X) = var(X) = E[(X - EX) ]

2

(58)

Copyright 1996 Lawrence C. Marsh

cov(X,Y) = E [(X - EX)(Y-EY)]

= E [XY - X EY - Y EX + EX EY]

= E(XY) - 2 EX EY + EX EY = E(XY) - EX EY

cov(X,Y) = E [(X - EX)(Y-EY)]

cov(X,Y) = E(XY) - EX EY

= E(XY) - EX EY - EY EX + EX EY

(59)

.15

.05

.45

.35

Y = 1 Y = 2

X = 0

X = 1

.60

.40

.50

EX=0(.60)+1(.40)=.40 EY=1(.50)+2(.50)=1.50

E(XY) = (0)(1)(.45)+(0)(2)(.15)+(1)(1)(.05)+(1)(2)(.35)=.75

EX EY = (.40)(1.50) = .60

cov(X,Y) = E(XY) - EX EY

= .75 - (.40)(1.50) = .75 - .60

= .15

covariance

(60)

Copyright 1996 Lawrence C. Marsh

The

correlation

between two random

variables X and Y is their covariance

divided by the square roots of their

respective variances.

Correlation is a pure number falling between -1 and 1.

cov(X,Y)

ρ

(X,Y) =

var(X) var(Y)

(61)

Copyright 1996 Lawrence C. Marsh

.15

.05

.45

.35

Y = 1 Y = 2

X = 0

X = 1

.60

.40

.50

EX=.40 EY=1.50

cov(X,Y) = .15

correlation

EX=0(.60)+1(.40)=2 2 2 .40

var(X) = E(X ) - (EX)

= .40 - (.40) = .24

2 2 2

EY=1(.50)+2(.50) = .50 + 2.0

= 2.50

2 2 2

var(Y) = E(Y ) - (EY)

= 2.50 - (1.50) = .25

2 2 2

ρ(X,Y) = cov(X,Y)

var(X) var(Y)

ρ(X,Y) = .61

(62)

Copyright 1996 Lawrence C. Marsh

Independent random variables

have zero covariance and,

therefore, zero correlation.

The converse is not true.

Zero Covariance & Correlation

(63)

Copyright 1996 Lawrence C. Marsh

The expected value of the weighted sum

of random variables is the sum of the

expectations of the individual terms.

Since expectation is a linear operator, it can be applied term by term.

E[c

₁

X + c

₂

Y] = c

₁

EX + c

₂

EY

E[c

₁

X

₁

+...+ c

_n

X

_n

] = c

₁

EX

₁

+...+ c

_n

EX

_n

In general, for random variables X₁, . . . , X_n :

(64)

Copyright 1996 Lawrence C. Marsh

The variance of a weighted sum of random

variables is the sum of the variances, each times

the square of the weight, plus twice the covariances of all the random variables times the products of

their weights.

var(c₁X + c₂Y)=c₁2var(X)+c₂2var(Y) + 2c₁c₂cov(X,Y)

var(c₁X − c₂Y) = c2₁var(X)+c₂2var(Y) − 2c₁c₂cov(X,Y)

Weighted sum of random variables:

Weighted difference of random variables:

(65)

Copyright 1996 Lawrence C. Marsh

The Normal Distribution

Y ~ N(

β

,

σ

2

)

f(y) =

2

π

σ

2 1

exp

β

y

f(y)

2

σ

2

(y -

β

)

2

(66)

Copyright 1996 Lawrence C. Marsh

The Standardized Normal

Z ~ N(

0

,

1

)

f(z) =

2

π

1

exp

-

2

z

2

Z = (y -

β

)/

σ

(67)

Copyright 1996 Lawrence C. Marsh

P [ Y > a ] = P Y - β > = a - β P Z > a - β

σ σ σ

β

y

f(y)

a

(68)

P [ a < Y < b ] = P < <

= P < Z < a - β Y - β

σ

σ b - σβ

a - β

σ

b - β

σ

β

y

f(y)

a

Y ~ N(

β

,

σ

2

)

b

(69)

Y₁ ~ N(β₁,σ₁2), Y₂ ~ N(β₂,σ₂2), . . . , Y_n ~ N(β_n,σ_n2)

W = c

₁

Y

₁

+ c

₂

Y

₂

+ . . . + c

_n

Y

_n

Linear combinations of jointly

normally distributed random variables

are themselves normally distributed.

W ~ N

[

E(W), var(W)

]

(70)

Copyright 1996 Lawrence C. Marsh

mean: E[V] = E[

χ

_(m)] = m

If Z₁, Z₂, . . . , Z_m denote m independent N(0,1) random variables, and

V = Z2₁+ Z2₂ + . . . + Z2_m, then V ~

χ

_(m)2

V is chi-square with m degrees of freedom.

Chi-Square

variance: var[V] = var[

χ

_(m)] = 2m

If Z₁, Z₂, . . . , Z_m denote m independent N(0,1) random variables, and

V = Z2₁+ Z2₂ + . . . + Z2_m, then V ~

χ

_(m)2

V is chi-square with m degrees of freedom.

2

(71)

Copyright 1996 Lawrence C. Marsh

mean: E[

t

] = E[

t

_(m)] = 0 symmetric about zero variance: var[t] = var[t_(m)] = m / (m

−

2)

If Z ~ N(0,1) and V ~

χ

_(m)and if Z and V are independent then,

~ t

_(m)

t

is student-t with m degrees of freedom.

2

t =

Z V _m

(72)

If V₁ ~

χ

_(m

1) and V2 ~

χ

(m2) and if V1 and V2

are independent, then

~ F

_(m

1,m2)

F

is an F statistic with m₁ numerator degrees of freedom and m₂ denominator degrees of freedom.

2

F =

V_{1 m}

1

V₂

m₂

2

(73)

The Simple Linear

Regression

Model

Chapter 3

(74)

Copyright 1996 Lawrence C. Marsh

1. Estimate a relationship among economic variables, such as y = f(x).

2. Forecast or predict the value of one variable, y, based on the value of

another variable, x.

(75)

Weekly Food Expenditures

y = dollars spent each week on food items. x = consumer’s weekly income.

The relationship between x and the expected value of y , given x, might be linear:

E(y|x) = β₁+ β₂x

(76)

Copyright 1996 Lawrence C. Marsh

f(y|x=480)

y

µ_y|x=480

Figure 3.1a Probability Distribution f(y|x=480) of Food Expenditures if given income x=$480.

(77)

f(y|x) _f(y|x=480) _f(y|x=800)

y

µ_y|x=480 µ_y|x=800

Figure 3.1b Probability Distribution of Food

Expenditures if given income x=$480 and x=$800.

(78)

{

β₁

∆x

∆E(y|x) E(y|x)

Average

Expenditure

x (income) E(y|x)=β₁+β₂x

β₂= ∆E(y|x)

∆x

Figure 3.2 The Economic Model: a linear relationship between avearage expenditure on food and income.

(79)

Copyright 1996 Lawrence C. Marsh

.

x_t x₁=480 x₂=800

y t

f(y_t)

Figure 3.3. The probability density function for y_t at two levels of household income, x_t

expenditure

Homoskedastic Case

income

(80)

.

x_t x₁ x₂

y t

f(y_t)

Figure 3.3+. The variance of y_t increases as household income, x_t, increases.

expenditure

Heteroskedastic Case

x₃

.

income

(81)

Assumptions of the Simple Linear

Regression Model - I

1. The average value of y, given x, is given by the linear regression:

E(y) = β₁+ β₂x

2. For each value of x, the values of y are

distributed around their mean with variance:

var(y) = σ2

3. The values of y are uncorrelated, having zero covariance and thus no linear relationship:

cov(y_i,y_j) = 0

4. The variable x must take at least two different values, so that x ≠ c, where c is a constant.

(82)

5. (optional) The values of y are normally distributed about their mean for each

value of x:

y ~ N [(β₁+β₂x), σ2 _]

One more assumption that is often used in

(83)

The Error Term

y is a random variable composed of two parts:

I. Systematic component: E(y) = β₁+ β₂x

This is the mean of y.

II. Random component: e = y - E(y) = y - β₁- β₂x

This is called the random error.

Together E(y) and e form the model:

y = β₁+ β₂x + e

(84)

Copyright 1996 Lawrence C. Marsh

Figure 3.5 The relationship among y, e and the true regression line.

.

y₄ y₁ y₂ y₃

x₁ x₂ _x₃ x₄

}

{

e₁ e₂ e₃

e₄ _{E(y) =}_β

1 + β2x

x

(85)

}

.

}

.

y₄ y₁

y₂ _y

3

x₁ x₂ _x₃ x₄

{

e₁ e₂ e₃ e₄ x y

Figure 3.7a The relationship among y, e and the fitted regression line.

^

y = b₁+ b₂x ^

.

y₁ y₂

y₃ y4

(86)

{

.

y₄ y₁

y₂ _y

3

x₁ x₂ _x₃ x₄ _x

y

Figure 3.7b The sum of squared residuals from any other line will be larger.

y = b₁+ b₂x ^

.

y^₁

y₃ ^

y₄

^ y = b^* *1 + b*2x *

e₁ ^*

e₂ ^*

y^*₂

e₃ ^*

* ^e4 *

*

{

(87)

Copyright 1996 Lawrence C. Marsh

f(

.

) _f(e) _f(y)

Figure 3.4 Probability density function for e and y

0 _β

1+β2x

(88)

The Error Term Assumptions

1. The value of y, for each value of x, is y = β₁+ β₂x + e

2. The average value of the random error e is:

E(e) = 0

3. The variance of the random error e is:

var(e) = σ2 = var(y)

4. The covariance between any pair of e’s is:

cov(e_i,e_j) = cov(y_i,y_j) = 0

5. x must take at least two different values so that

x ≠ c, where c is a constant.

6. e is normally distributed with mean 0, var(e)=σ2

(optional) e ~ N(0,σ2₎

(89)

Unobservable Nature

of the Error Term

1. Unspecified factors / explanatory variables, not in the model, may be in the error term. 2. Approximation error is in the error term if

relationship between y and x is not exactly a perfectly linear relationship.

3. Strictly unpredictable random behavior that may be unique to that observation is in error.

(90)

Copyright 1996 Lawrence C. Marsh

Population

regression values:

y

_t

=

β

₁

+

β

₂

x

_t

+ e

_t

Population

regression line:

E(y

_t

|x

_t

) =

β

₁

+

β

₂

x

_t

Sample

regression values:

y

_t

= b

₁

+ b

₂

x

_t

+ e

_t

Sample

regression line:

y

_t

= b

₁

+ b

₂

x

_t

^

(91)

Copyright 1996 Lawrence C. Marsh

y

_t

=

β

₁

+

β

₂

x

_t

+ e

_t

Minimize error sum of squared deviations:

S(

β

₁

,

β

₂) =

Σ

(

y

_t

-

β

₁

-

β

₂

x

_t )

2_(3.3.4)

t=1 T

e

_t

= y

_t

-

β

₁

-

β

₂

x

_t

(92)

Copyright 1996 Lawrence C. Marsh

Minimize w.

r.

t.

β

₁

and

β

₂

:

S(

β

₁

,

β

₂) =

Σ

(

y

_t

-

β

₁

-

β

₂

x

_t )

2_(3.3.4)

t =1 T

= - 2

Σ

(

y

_t

-

β

₁

-

β

₂

x

_t )

= - 2

Σ

x

_t(

y

_t

-

β

₁

-

β

₂

x

_t )

∂S(

.

)

∂β

₁

∂S(

.

)

∂β

₂

Set each of these two derivatives equal to zero and

solve these two equations for the two unknowns:

β

₁

β

₂

(93)

.

Minimize w.

r.

t.

β

₁

and

β

₂

:

S(

.

) =

Σ

(

y

_t

-

β

₁

-

β

₂

x

_t )

2

t =1 T

∂S(.)

∂β_i < 0

∂S(.)

∂β_i > 0 ∂S(.)

∂β_i = 0

(94)

Copyright 1996 Lawrence C. Marsh

To minimize S(.), you set the two derivatives equal to zero to get:

= - 2

Σ

(

y

_t

-

b

₁

-

b

₂

x

_t ) = 0

= - 2

Σ

x

_t(

y

_t

-

b

₁

-

b

₂

x

_t ) = 0

∂S(

.

)

∂β

₁

∂S(

.

)

∂β

₂

When these two terms are set to zero,

β

₁ and

β

₂ become

b

₁ and

b

₂ because they no longer represent just any value of

β

₁ and

β

₂ but the special values that correspond to the minimum of S(

.

) .

(95)

- 2

Σ

(

y

_t

-

b

₁

-

b

₂

x

_t ) = 0

- 2

Σ

x

_t(

y

_t

-

b

₁

-

b

₂

x

_t ) = 0

Σ

y

_t -

T

b

₁ -

b

₂

Σ

x

_t = 0

Σ

x

_t

y

_t -

b

₁

Σ

x

_t-

b

₂

Σ

x

_t2 = 0

T

b

₁ +

b

₂

Σ

x

_t =

Σ

y

t

b

₁

Σ

x

_t+

b

₂

Σ

x

2_t =

Σ

x

_t

y

_t

(96)

Solve for

b

₁ and

b

₂ using definitions of

x

and

y

T

b

₁ +

b

₂

Σ

x

_t =

Σ

y

t

b

₁

Σ

x

_t+

b

₂

Σ

x

2_t =

Σ

x

_t

y

_t

T

Σ

x

_t

y

_t-

Σ

x

_t

Σ

y

_t

T

Σ

x

_t2

- (

Σ

x

_t)2

b

₂=

b

₁=

y

-

b

₂

x

(97)

elasticities

percentage change in y percentage change in x

η = ₌

∆x/x

∆y/y

= _∆∆y x

x y

Using calculus, we can get the elasticity at a point:

η = lim ∆y x ₌

∆x y

∂y x

∂x y

∆x→0

(98)

Copyright 1996 Lawrence C. Marsh

E(y) =

β

₁

+

β

₂

x

∂

E(y)

∂

x

=

β

2

applying elasticities

∂

E(y)

∂

x

=

β

2

η

=

E(y)

x

E(y)

x

(99)

Copyright 1996 Lawrence C. Marsh

estimating elasticities

∂

y

∂

x

=

b

2

η

=

y

x

y

x

^

y

^_t

=

b

₁

+ b

₂

x

_t

= 4 + 1.5 x

_t

x

= 8 = average number of years of experience

y

= $10 = average wage rate

= 1.5 = 1.2

8

10

=

b

₂

η

y

x

^

(100)

Copyright 1996 Lawrence C. Marsh

Prediction

y

^_t

= 4 + 1.5 x

_t

Estimated regression equation:

x

_t= years of experience

y

^_t = predicted wage rate

If

x

_t= 2 years, then

y

^ _t = $7.00 per hour. If

x

_t= 3 years, then

y

^_t =

$8.50

per hour

.

(101)

Copyright 1996 Lawrence C. Marsh

log-log models

ln(y) =

β

₁

+

β

₂

ln(x)

∂

ln(y)

∂

x

∂

ln(x)

∂

x

=

β

₂

∂

y

∂

x

=

β

2

1

y

∂

x

∂

x

1

x

(102)

Copyright 1996 Lawrence C. Marsh

∂

y

∂

x

=

β

2

1

y

∂

x

∂

x

1

x

=

β

₂

∂

y

∂

x

y

elasticity of y with respect to x:

=

β

₂

(103)

Copyright 1996 Lawrence C. Marsh

Properties of

Least

Squares

Estimators

Chapter 4

(104)

Copyright 1996 Lawrence C. Marsh

y

_t

=

household weekly food expenditures

Simple Linear Regression Model

y

_t

=

β

₁

+

β

₂

x

_t

+

ε

_t

x

_t

=

household weekly income

For a given level of

x

_t, the expected level of food expenditures will be:

E(

y

_t

|

x

_t)

=

β

₁

+

β

₂

x

_t

(105)

Copyright 1996 Lawrence C. Marsh

1.

y

_t

=

β

₁

+

β

₂

x

_t

+

ε

_t

2.

E(

ε

_t

)

=

0 <=> E(

y

_t

)

=

β

₁

+

β

₂

x

_t

3.

var(

ε

_t

)

=

σ

2

=

var(

y

_t

)

4.

cov(

ε

_i

,

ε

_j

)

=

cov(

y

_i

,

y

_j

)

=

0

5.

x

_t

≠

c

for every observation

6.

ε

_t

~N(0,

σ

2

) <=>

y

_t

~

N(

β

₁

+

β

₂

x

_t

,

σ

2

)

Assumptions of the Simple

Linear Regression Model

(106)

The population parameters

β

₁ and

β

₂ are unknown population constants

.

The formulas that produce the

sample estimates b₁ and b₂ are

called the estimators of

β

₁ and

β

₂

.

When b₀ and b₁ are used to represent

the formulas rather than specific values, they are called estimators of

β

₁ and

β

₂ which are random variables because

they are different from sample to sample.

(107)

• If the least squares estimators b₀ and b₁ are random variables, then what are their their means, variances, covariances and probability distributions?

• Compare the properties of alternative estimators to the properties of the

least squares estimators.

Estimators are Random Variables

( estimates are not )

(108)

The Expected Values of b1 and b2

The least squares formulas (estimators) in the simple regression case:

b

₂

=

T

Σ

x

t

y

t

-

Σ

x

t

Σ

y

t

T

Σ

x

_t2

-(

Σ

x

_t

)

2

b

₁

= y - b

₂

x

where

y =

Σ

y

_t

/ T

and

x =

Σ

x

_t

/ T

(3.3.8a)

(3.3.8b)

(109)

Substitute in

y

_t

=

β

₁

+

β

₂

x

_t

+

ε

_t to get:

b

₂

=

β

₂

+

T

Σ

x

t

ε

t

-

Σ

x

t

Σε

t

T

Σ

x

_t2

-(

Σ

x

_t

)

2

The mean of

b

₂ is:

Eb

₂

=

β

₂

+

T

Σ

x

t

E

ε

t

-

Σ

x

t

Σ

E

ε

t

T

Σ

x

_t2

-(

Σ

x

_t

)

2

Since

E

ε

_t

= 0

, then

Eb

₂

=

β

₂.

(110)

The result

Eb

₂

=

β

₂ means that

the distribution of

b

₂ is centered at

β

₂.

Since the distribution of

b

₂

is centered at

β

₂ ,we say that

b

₂is an unbiased estimator of

β

₂.

(111)

The unbiasedness result on the

previous slide assumes that we

are using the

correct model

.

If the model is of the wrong form

or is missing important variables,

then

E

ε

_t

≠

0

, then

Eb

₂

≠

β

₂

.

(112)

Unbiased Estimator of the Intercept

In a similar manner, the estimator b

₁

of the intercept or constant term can be

shown to be an

unbiased

estimator of

β

₁

when the model is correctly specified.

Eb

₁

=

β

₁

(113)

b

₂

=

T

Σ

x

t

y

t

−

Σ

x

t

Σ

y

t

T

Σ

x

_t2

−

(

Σ

x

_t

)

2 (3.3.8a)

(4.2.6)

Equivalent expressions for b

₂

:

Expand and multiply top and bottom by T:

b

₂

=

Σ

(x

Σ(

t

−

x )

(

y

t

−

y )

x

_t

−

x

)

2

(114)

Variance of b

₂

Given that both y_t and

ε

_t have variance

σ

2, the variance of the estimator b₂ is:

b₂ is a function of the y_t values but var(b₂) does not involve y_t directly.

Σ(

x

_t

−

x

)

σ

2

var(b

₂

) =

(115)

Variance of b

₁

Τ Σ(

x

_t

−

x

)

2

var(b

₁

) =

σ

2

Σ

x

_t2

the variance of the estimator b₁ is:

b

₁

= y

−

b

₂

x

Given

(116)

Covariance of b

₁

and b

₂

Σ(

x

_t

−

x

)

2

cov(b

₁

,b

₂

) =

σ

2

−

x

If x = 0, slope can change without affecting the variance.

(117)

What factors determine

variance and covariance ?

1. σ 2: uncertainty about yt values uncertainty about

b₁, b₂ and their relationship.

2. The more spread out the x_t values are then the more

confidence we have in b₁, b₂, etc.

3. The larger the sample size, T, the smaller the

variances and covariances.

4. The variance b₁ is large when the (squared) x_t values

are far from zero (in either direction).

5. Changing the slope, b₂, has no effect on the intercept,

b1, when the sample mean is zero. But if sample

mean is positive, the covariance between b₁ and

b₂ will be negative, and vice versa.

(118)

Gauss-Markov Theorm

Under the first five assumptions of the simple, linear regression model, the

ordinary least squares estimators b₁ and b₂ have the smallest variance of all linear and unbiased estimators of

β₁ and β₂. This means that b₁and b₂

are the Best Linear Unbiased Estimators (BLUE) of β₁ and β₂.

(119)

implications of Gauss-Markov

1. b₁and b₂ are “best” within the class of linear and unbiased estimators.

2. “Best” means smallest variance

within the class of linear/unbiased.

3. All of the first five assumptions must hold to satisfy Gauss-Markov.

4. Gauss-Markov does not require assumption six: normality.

5. G-Markov is not based on the least squares principle but on b₁and b₂.

(120)

G-Markov implications (continued)

6. If we are not satisfied with restricting our estimation to the class of linear and unbiased estimators, we should ignore the Gauss-Markov Theorem and use

some nonlinear and/or biased estimator instead. (Note: a biased or nonlinear

estimator could have smaller variance

than those satisfying Gauss-Markov.) 7. Gauss-Markov applies to the b₁and b₂

estimators and not to particular sample values (estimates) of b₁and b₂.

(121)

Probability Distribution

of Least Squares Estimators

b

₂

~ N

β

₂

,

Σ(

x

_t

−

x

)

σ

2

b

₁

~ N

β

₁

,

Τ Σ(

x

_t

−

x

)

2

σ

2

Σ

x

t

2

(122)

y

_t

and

ε

_t

normally distributed

The least squares estimator of β₂ can be expressed as a linear combination of y_t’s:

b

₂

=

Σ

w

_t

y

_t

b

₁

= y

−

b

₂

x

Σ(

x

_t

−

x

)

2

where w

_t

=

(

x

t

−

x

)

This means that b₁and b₂ are normal since linear combinations of normals are normal.

(123)

normally distributed under

The Central Limit Theorem

If the first five Gauss-Markov assumptions

hold, and sample size, T, is sufficiently large, then the least squares estimators, b₁and b₂, have a distribution that approximates the

normal distribution with greater accuracy the larger the value of sample size, T.

(124)

Consistency

We would like our estimators, b₁ and b₂, to collapse

onto the true population values, β₁ and β₂, as sample size, T, goes to infinity.

One way to achieve this consistency property is

for the variances of b₁ and b₂ to go to zero as T

goes to infinity.

Since the formulas for the variances of the least

squares estimators b₁ and b₂ show that their

variances do, in fact, go to zero, then b₁ and b₂,

are consistent estimators of β₁ and β₂.

(125)

Estimating the variance

of the error term,

σ

2

e

_t

= y

_t

−

b

₁

−

b

₂

x

_t

^

Σ

e

^

_t

t =1 T

2

T− 2

σ

2

=

σ

2

is an unbiased estimator of

σ

2

^

(126)

The Least Squares

Predictor, y

^

_o

G iven a value of the explanatory

variable, X_o, w e w ould like to predict a value of the dependent variable, y_o.

The least squares predictor is:

y

_o

= b

₁

+ b

₂

x

_o(4.7.2)

^

(127)

Inference

in the Simple

Regression Model

Chapter 5

(128)

1.

y

_t

=

β

₁

+

β

₂

x

_t

+

ε

_t

2. E(

ε

_t

)

=

0 <=> E(

y

_t

)

=

β

₁

+

β

₂

x

_t

3. var(

ε

_t

)

=

σ

2

=

var(

y

_t

)

4. cov(

ε

_i

,

ε

_j

)

=

cov(

y

_i

,

y

_j

)

=

0

5.

x

_t

≠

c

for every observation

6.

ε

_t

~N(0,

σ

2

) <=>

y

_t

~

N(

β

₁

+

β

₂

x

_t

,

σ

2

)

Assumptions of the Simple

Linear Regression Model

(129)

Probability D istribution

of Least Squares Estim ators

b

₁

~ N

β

₁

,

Τ Σ(

x

t

−

x

)

2

σ

2

Σ

x

t

2

b

₂

~ N

β

₂

,

Σ(

x

_t

−

x

)

σ

2

(130)

σ

^

2

=

Τ − 2

e

^_t2

Σ

Unbiased estimator of the error variance

:

σ

2

σ

2

^

(Τ − 2)

∼

Τ − 2

χ

Transform to a chi-square distribution

:

(131)

We make a correct decision if:

• The null hypothesis is false and we decide to reject it.

• The null hypothesis is true and we decide not to reject it.

Our decision is incorrect if:

• The null hypothesis is true and we decide to reject it. This is a type I error.

• The null hypothesis is false and we decide not to reject it. This is a type II error.

(132)

b

₂

~ N

β

₂

,

Σ(

x

_t

−

x

)

σ

2

Create a standardized normal random variable, Z, by subtracting the mean of b₂ and dividing by its standard deviation:

b

₂

− β

₂

var(b

₂

)

Ζ

=

∼ Ν(0,1)

(133)

Simple Linear Regression

y

_t

=

β

₁

+

β

₂

x

_t

+

ε

_t

where E

ε

_t = 0

y

_t

~

N(

β

₁

+

β

₂

x

_t

,

σ

2

)

since