Chapter15 - Multiple Regression Model Building

Statistics for Managers

Using Microsoft® Excel

5th Edition

Chapter 15

Learning Objectives

In this chapter, you learn:



_{To use quadratic terms in a regression model}



_{To use transformed variables in a regression model}



_{To measure the correlation among independent}

variables



_{To build a regression model, using either the}

stepwise or best-subsets approach



_{To avoid the pitfalls involved in developing a}

Nonlinear Relationships



_{The relationship between the dependent variable}

and an independent variable may not be linear



_{Can review the scatter plot to check for}

non-linear relationships



_{Example: Quadratic model}



_{The second independent variable is the square of the}

i

2

1i

2

1i

1

0

i

β

β

X

β

X

ε

Quadratic Regression Model

where:

β

0

= Y intercept

β

1

= regression coefficient for linear effect of X on Y

β

2

= regression coefficient for quadratic effect on Y

ε

i

= random error in Y for observation i

i

2

1i

2

1i

1

0

i

β

β

X

β

X

ε

Y

_

_

_

_

Quadratic Regression Model

X

si

du

al

s

X

Y

X

si

du

al

s

Y

Quadratic Regression Model

Quadratic models may be considered when the scatter

diagram takes on one of the following shapes:

Quadratic Regression

Equation



_{Test for Overall Relationship}



_H

₀

_{: β}

₁

_{= β}

₂

_{= 0 (no overall relationship between X and Y)}



_H

₁

_{: β}

₁

_{and/or β}

₂

_{≠ 0 (there is a relationship between X and Y)}

2

1i

2

1i

1

0

i

b

b

X

b

X

Yˆ

_

_

_

MSR



_{Collect data and use Excel to calculate the}

Testing for Significance

Quadratic Effect

Testing the Quadratic Effect

Compare quadratic regression equation

with the linear regression equation

Testing for Significance

Quadratic Effect

Testing the Quadratic Effect

Consider the quadratic regression equation

Testing for Significance

Quadratic Effect

Testing the Quadratic Effect

Hypotheses

(The quadratic term does not improve the model)

(The quadratic term improves the model)



_{The test statistic is}

Testing for Significance

Quadratic Effect

Testing the Quadratic Effect

If the t test for the quadratic effect is

Quadratic Regression

Example

Purity increases as filter time increases:

Purity

Filter

Time

3

1

Purity vs. Time

Quadratic Regression

Example

Regression Statistics

R Square

0.96888

Adjusted R Square

0.96628

Standard Error

6.15997

Simple (linear) regression results:

Y = -11.283 + 5.985 Time

 

Coefficients

Standard

Error

t Stat

P-value

Intercept

-11.28267

3.46805

-3.25332

0.00691

Time

5.98520

0.30966

19.32819

2.078E-10

F

Significance F

373.57904

2.0778E-10

^

Time Residual Plot

-5

t statistic, F statistic, and

adjusted r

2

_{are all high, but}

Quadratic Regression

Example

 

Coefficients

Standard

Error

t Stat

P-value

Intercept

1.53870

2.24465

0.68550

0.50722

Time

1.56496

0.60179

2.60052

0.02467

Time-squared

0.24516

0.03258

7.52406

1.165E-05

Regression Statistics

R Square

0.99494

Adjusted R Square

0.99402

Standard Error

2.59513

F

Significance F

1080.7330

2.368E-13



_{Quadratic regression results:}

Y = 1.539 + 1.565 Time + 0.245 (Time)

^

2

Time Residual Plot

Time-squared Residual Plot

-5

The quadratic term is significant and improves

the model: adjusted r

2

_{is higher and S}

Using Transformations in

Regression Analysis

Idea:



_{Non-linear models can often be transformed to a}

linear form



_{Can be estimated by least squares if}

transformed



_{Transform X or Y or both to get a better fit or to}

The Square Root

Transformation

The square-root transformation

Used to



_{overcome violations of the equal variance assumption}



_{fit a non-linear relationship}

i

1i

1

0

i

β

β

X

ε

The Square Root

Transformation

Shape of original relationship

Relationship when transformed

The Log Transformation

Original multiplicative model

Transformed multiplicative model

i

The Multiplicative Model:

Original multiplicative model

Transformed exponential model

i

The Exponential Model:

Interpretation of Coefficients

For the multiplicative model:

When both dependent and independent variables are

transformed:



_{The coefficient of the independent variable X}

_k

_{can be}

interpreted as follows: a 1 percent change in X

k

leads

to an estimated b

percentage change in the mean value

i

1i

1

0

i

log

β

β

log

X

log

ε

Y

Collinearity



_{Collinearity: High correlation exists among}

two or more independent variables



_{The correlated variables contribute redundant}

Collinearity



_{Including two highly correlated independent}

variables can adversely affect the regression

results



_{No new information provided}



_{Can lead to unstable coefficients (large}

standard error and low t-values)



_{Coefficient signs may not match prior}

Some Indications of Strong

Collinearity



_{Incorrect signs on the coefficients}



_{Large change in the value of a previous coefficient}

when a new variable is added to the model



_{A previously significant variable becomes}

Detecting Collinearity

Variance Inflationary Factor

VIF

_j

is used to measure collinearity:

where R

2

j

is the coefficient of determination from a regression

model that uses X

_j

as the dependent variable and all other X

variables as the independent variables

2

j

j

R

1

1

VIF

Model Building



_{Goal is to develop a model with the best set of}

independent variables



_{Easier to interpret if unimportant variables are removed}



_{Lower probability of collinearity}



_{Stepwise regression procedure}



_{Provide evaluation of alternative models as variables are}

added



_{Best-subset approach}



_{Try all combinations and select the model with the}

Stepwise Regression



_{Idea: develop the least squares regression}

equation in steps, adding one independent

variable at a time and evaluating whether

existing variables should remain or be

removed



_{The coefficient of partial determination is the}

Best Subsets Regression

Idea: estimate all possible regression equations using all

possible combinations of independent variables

Choose the best model by looking for the highest adjusted r

2

The model with the largest adjusted r

2

will also have the

smallest S

YX

Alternative Best Subsets

Criterion



_{Calculate the value C}

_p

_{for each potential}

regression model



_{Consider models with C}

_p

_{values close to or}

below k + 1

Alternative Best Subsets

Where k = number of independent variables included in a

particular regression model

T = total number of parameters to be estimated in the

full regression model

= coefficient of multiple determination for model with k

independent variables

= coefficient of multiple determination for full model with

8 Steps in Model Building

1. Choose independent variables to include in the model

2. Estimate full model and check VIFs and check if any VIFs > 5



_{If no VIF > 5, go to step 3}



_{If one VIF > 5, remove this variable}



_{If more than one, eliminate the variable with the highest VIF}

and repeat step 2

8 Steps in Model Building

5. Choose the best model.



_{Consider parsimony.}



_{Do extra variables make a significant contribution?}

6. Perform complete analysis with chosen model, including

residual analysis.

7. Transform the model if necessary to deal with violations of

linearity or other model assumptions.

Model Validation



_{Collect new data and compare the results.}



_{Compare the results of the regression model}

to previous results.



_{If the data set is large, split the data into two}

parts and cross-validate the results.

Model Building Flowchart

Choose X

₁

,X

₂

,…,X

_k

Run regression to

find VIFs

Remove

variable with

highest

VIF

Any

VIF>5?

Run best subsets

regression to obtain

“best” models in terms

of C

_p

Do complete analysis

Add quadratic term and/or

transform variables as indicated

Perform predictions

No

More than

one?

Remove this X

Yes

Pitfalls and Ethical

Considerations



_{Understand that interpretation of the estimated}

regression coefficients are performed holding all

other independent variables constant



_{Evaluate residual plots for each independent}

variable



_{Evaluate interaction terms}

Pitfalls and Ethical

Considerations



_{Obtain VIFs for each independent variable before}

determining which variables should be included in the

model.



_{Examine several alternative models using best-subsets}

regression.



_{Use other methods when the assumptions necessary for}

least-squares regression have been seriously violated.

Chapter Summary



_{Developed the quadratic regression model}



_{Discussed using transformations in regression}

models



_{The multiplicative model}



_{The exponential model}



_{Described collinearity}



_{Discussed model building}



_{Stepwise regression}



_{Best subsets}