Week 9 - Heteroskedasticity and Generalised Least Squares (no Week 8 due to the midsem)
Heteroskedasticity - variance of error term is not constant, but varies across observations
(violating an MLR assumption):
. E.g. variance in wages may ar(u |x , ..x )
V i 1 . k = σi2
depend on education.
With heteroskedasticity, the OLS estimators are still unbiased and consistent, but the OLS standard errors of the estimators are biased.
This invalidates t and F significance tests, and
means that OLS is not the “best” (i.e. most efficient) estimator.
But, we can adjust t and F stats so they are robust to heteroskedasticity. The t-test is valid asymptotically. F-test doesn’t work, but heteroskedasticity robust versions are available in econometric software (regress y x1 x2, robust).
SLR with heteroskedasticity
● β︿1= β1+ SST , remembering that
x2
Σ(xi−x)ui
ST (x )
S x2 = Σ i−x2
● Var(β )︿1 = SST , remembering that
x2
Σ(xi−x) σ2 i2
ar(u) σ2=V i
● Consistent estimator (i.e. approaches the true estimator as approaches infinity) forn under heteroskedasticity : (replace with )
ar(β )
V ︿1
︿
V ar(β )1
︿
SSTx2
Σ(xi−x)2︿ui2
σi2 u︿2i
○ This estimator applies in homoskedasticity as well
○ If σi2= σ2(constant), then formula simplifies to the usual:Var(β )1 = σ2/SSTx MLR with heteroskedasticity
Consistent estimator in MLR for Var(β )︿j under heteroskedasticity: V ar
︿
(β ) , wherej
︿ =
SSRj2 Σi ijr u︿2︿i2
× n−k−1n
is the i-th residual and is the sum of squared residuals from regressing on all other
r︿ij SSRj2 xj
independent variables.
●
√
V ar(β )︿
︿j = “heteroskedasticity robust standard error for ” (a.k.a.βj White/Huber/Eicker standard errors)● Used for inference
Considerations for heteroskedasticity
● All formulas are only valid in large samples.
● Heteroskedasticity robust standard errors may be larger or smaller than their non-robust counterparts. The differences are often small in practice.
● We can get better estimates if the form of heteroskedasticity is known
○ Use prior research (e.g.
studies suggest that high wage & education
individuals may also have more variability in wages compared to individuals with less wages & education)
○ Plot the residuals
● Using the logarithmic transformation for the dependent variable often reduces heteroskedasticity (below L, log wages; below R, level wages)
vs
Formal tests for heteroskedasticity
Test for a null of homoskedasticity: H0:Var(u|X)= σ2, equivalent to H0:Var(u|X) (u |X) E(u|X)]
=E 2 − [ 2
[since , by zero conditional mean assumption (MLR Assumption 4)]
(u |X)
=E 2 E(u|X)= 0
. Essentially, we’re testing whether is related to any of the ’s.
(u )
=E 2 = σ2 u2 x
If we are prepared to assume the relationship between u2and each is linear, we can test forx linear heteroskedasticity using the Breusch-Pagan test: Given u2= δ0+ δ1 1x + . + δ.. k kx +v, test
. We can’t observe the true errors, so we substitute the squared
H0: δ1= δ2= . = δ.. k = 0
residuals, ︿2u for u2.
Breusch-Pagan test
1. Outline the test: ︿u2= δ0+ δ1 1x + . + δ.. k kx +v H; 0 : δ1= δ2 = . = δ.. k= 0.
2. Regress y on the x’s and obtain the residuals, (︿u predict residuals; predict [variable name for residuals],r).
3. Regress ︿u2on all ’s to get xj R2︿u2(i.e. how well our model, ︿u2 = δ0+ δ1 1x + . + δ.. k kx +v , explains variation in the squared residuals).
4(a). Obtain the F-statistic: (R )/k (a large test statistic / a high R-squared is evidence
2 uhat2
(1−R2 )/(n−k−1)
uhat2
against the null hypothesis). OR:
4(b). Obtain the Lagrange multiplier statistic: LM =n·R2︿u2 ~ χk2(“chi-squared” distribution).
5(a). Reject the null hypothesis if the F-statistic is greater than the critical F-value with (k, n-k-1) degrees of freedom. OR:
5(b). Reject the null hypothesis if the LM statistic is exceeding the critical value on the χ2 distribution with k degrees of freedom at the desired significance level.
Rejecting the null indicates the existence of linear heteroskedasticity
Example. Regressing ︿u2on ’s to get x R2︿u2: regress uhat2 x1 x2 ...
Using the log form for in this case reduces heteroskedasticity:y
The White test allows us to detect non-linear heteroskedasticity by using squares and cross products of all the ’s e.g. x ︿u2= δ0+ δ1 1x + δ2 2x + δ3 1x 2+ δ4 2x 2+ δ5 1 2x x +v. However, the test could involve a huge number of regressors, using up degrees of freedom, making it hard with small sample sizes.
The modified White test uses the fact that the fitted values are linear functions of the x’s:
. So, if we square the fitted values, we have a function of all the squares
x .. x
yi
︿= β︿0+ β︿1 1i+ . + β︿k ki
and cross-products of the x’s!
Modified White test
1. Outline the test: u︿2i = δ0+ δ1y︿i+ δ2y︿2i +v H; 0: δ1= δ2= 0; Ha:heteroskedasticity of an unknown form.
2. Estimate the original regression of y on the x’s and obtain the residuals and fitted values.
3. Conduct the auxiliary regression u︿2i = δ0+ δ1y︿i+ δ2y︿2i +v, and obtain the R2u .
i
︿2
4a. Form the F-statistic, and reject if greater than the critical value of the F distribution with k (=2) DoF on the numerator and n-k-1 (=n-3) DoF on the denominator. OR:
4b. Form the LM statistic, and reject if exceeding the critical value on the χ2 distribution with k (=2) DoF at the desired significance level.
How to deal with heteroskedasticity? (1) Estimate the model by OLS and calculate robust standard errors (still unbiased and consistent, but inefficient); (2) Use an alternative estimator that directly accounts for the heteroskedasticity.
