Chapter 9

(1)

SIMPLE LINEAR REGRESSION AND

CORRELATION

(2)

Learning Outcomes

› After studying this chapter, the student will:

1. be able to obtain a simple linear regression model and use it to make predictions.

2. be able to calculate the coefficient of determination and to interpret tests of regression coefficients.

3. be able to calculate correlations among variables.

4. understand how regression and correlation differ and when the use of each is appropriate.

(3)

9.1 INTRODUCTION

9.2 THE REGRESSION MODEL

(4)

Regression vs. Correlation

› Regression is an inferential method which is employed usually is to predict or estimate the value of one variable corresponding to a given value of another variable.

› Correlation is an inferential method which is concerned with measuring the strength of the relationship between variables.

› Fundamentals of regression analysis are based on the theory of conditional probability.

(5)

Types of Variables in Regression Analysis

› An independent variable or predictor variable (𝑿) is the variable being influenced by the investigator.

› An dependent variable or response variable (𝒀) is the variable being influenced by the independent variable.

(6)

Independent variable vs. dependent variable

Independent

variable influences Dependent

variable

(7)

9.3 THE SAMPLE REGRESSION EQUATION 9.4. EVALUATING THE REGRESSION

EQUATION

9.5. USING THE REGRESSION EQUATION

(8)

The Scatter Diagram

› A scatter diagram consists of points that are plotted by assigning values of the independent variable 𝑿 to the horizontal axis and values of the dependent variable 𝒀 to the vertical axis.

(9)

The Least-Squares Line

› The method of least squares is used to obtain the least-squares regression line which has the following form:

𝒚 = ෡ෝ 𝜷_𝟎 + ෡𝜷_𝟏𝒙

› Here, 𝒚 is the predicted value of the dependent variable, 𝒙 is the corresponding value of the independent variable on which the prediction is based, 𝜷෡_𝟎is the point where the line crosses the vertical axis (i.e., the intercept), and 𝜷෡_𝟏shows the amount by which y changes for each unit change in 𝒙 (i.e., the slope).

(10)

The Least-Squares Line (cont.)

𝒚 = ෡ ෝ 𝜷

_𝟎

+ ෡ 𝜷

_𝟏

𝒙

› The above equation means that as the independent value increase by 1 unit, as the dependent variable increase by 𝒃 on the average.

𝒚 = ෡ ෝ 𝜷

_𝟎

− ෡ 𝜷

_𝟏

𝒙

› The above equation means that as the independent

value increase by 1 unit, as the dependent variable

decrease by 𝒃 on the average.

(11)

The Least-Squares Line (cont.)

𝜷 ෡

_𝟏

= σ

_𝒊=𝟏^𝒏

(𝒙

_𝒊

− ഥ 𝒙)(𝒚

_𝒊

− ഥ 𝒚) σ

_𝒊=𝟏^𝒏

𝒙

_𝒊

− ഥ 𝒙

^𝟐

𝜷 ෡

_𝟎

= ഥ 𝒚 − ෡ 𝜷

_𝟏

𝒙 ഥ

(12)

Types of Relations from Scatter Diagram

(13)

Types of Relations from Scatter Diagram (cont.)

› A positive 𝜷෡_𝟏 indicates that values of Y tend to increase as

values of X increase, and we say that there is a direct (positive) linear relationship between X and Y.

› A negative 𝜷෡_𝟏 indicates that values of Y tend to decrease as values of X increase, and we say that there is an inverse

(negative) linear relationship between X and Y.

› When there is no linear relationship between X and Y, 𝛽መ₁ = 0.

(14)

The Least-Squares Line (cont.)

› The least-squares regression line is called the “best fit” line for describing the relationship between our two variables since the sum of the squared

vertical deviations of the observed data points (𝒚

_𝒊

) from the least-squares regression line is

smaller than the sum of the squared vertical

deviations of the data points from any other line.

(15)

(16)

(17)

Using the Fitted Equation

› The fitted equation can be used to obtain a prediction for the value Y given a value of X.

(18)

MegaStat Application

(19)

Example 9.3.1

› Table 9.3.1 shows the measurements taken on each subject were deep abdominal adipose tissue (AT) obtained by CT and waist circumference (in cm). Construct a scatter plot and perform

regression analysis if you know that deep abdominal AT is the dependent variable, while the waist measurement is the

independent variable.

(20)

(21)

Example 9.4.2

› We wish to know if we can conclude that the slope of the

population regression line describing the relationship between X and Y is zero. Also, predict Y and estimate the mean of Y for a waist circumference of 1m.

(22)

Example 9.4.2

› We wish to know if we can conclude that the slope of the

population regression line describing the relationship between X and Y is zero. Also, predict Y and estimate the mean of Y for a waist circumference of 1m.

› Important Note: Recall that X is measured in cm. Before

prediction or estimating the mean, check the measurement units.

Here, change 1m to cm by multiplying 1 by 100, then predict Y or estimated its mean based on X = 100.

(23)

Regression Analysis (Scatter Diagram) (MegaStat Application)

1. In Data Ribbon, click on MegaStat icon, then select Correlation / Regression.

2. Select Scatterplot.

(24)

3. Uncheck if you do not want to include the regression line in the 3. Input the range of X.

4. Input the range of Y.

5. Uncheck if you do not want to include the regression line in the

(25)

› The scatter diagram indicates a direct linear relationship between X and Y.

› The equation line is:

ෝ

𝒚 = −𝟐𝟏𝟓. 𝟗𝟖𝟏 + 𝟑. 𝟒𝟓𝟗 𝒙

› The regression equation means that as the waist

measurement increases by 1 unit as the deep abdominal AT increases by 3.489 units on the average.

(26)

Regression Analysis (MegaStat Application)

2. Select Regression.

(27)

3. Input the range of X. 4. Input the range of Y.

5. Change the option in the drop box to “Type in predictor

values”, add the values of X.

then press OK.

5. Change the option in the drop box to “Type in predictor

values”, add the values of X, and then press OK.

(28)

› The coefficient of determination is equal to 0.670.

(29)

› The intercept and the slope of the regression equation.

(30)

› The predicted value of Y given that X = 100 is ෝ𝒚 ≈ 𝟏𝟑𝟎.

(31)

9.6. THE CORRELATION MODEL

9.7 THE CORRELATION COEFFICIENT

(32)

Regression Analysis vs. Correlation Analysis

› Regression analysis describes the relationship between the

dependent (Y) and independent (X) variables for the purposes of prediction.

› Correlation analysis is used to determine the strength and direction of the relationship between the Y and X.

› The population correlation coefficient 𝝆 measures the direction and strength of the linear relationship between X and Y. It is known as Pearson's correlation coefficient.

(33)

Properties of ANY Correlation Coefficient

› The range of the correlation coefficient is from +1 to -1.

› If the value of the correlation coefficient is close to +1, then there is a strong direct linear relationship between the variables.

› If the value of the correlation coefficient is equal to +1, then there is a perfect direct linear relationship between the variables.

(34)

Properties of ANY Correlation Coefficient (cont.)

› If the value of the correlation coefficient is close to -1, then there is a strong inverse linear relationship

between the variables.

› If the value of the correlation coefficient is equal to -1, then there is a perfect inverse linear relationship between the variables.

› If the value of the correlation coefficient is close to 0

(35)

Properties of ANY Correlation Coefficient (cont.)

› If the value of the correlation coefficient is close to 0 (from the negative side), then there is an inverse weak linear relationship between the variables.

› If the value of the correlation coefficient is equal to 0, then there is no linear relationship between the

variables.

› The sign of the correlation coefficient is the same as the sigh of the slope of the regression equation.

(36)

The Sample Correlation Coefficient

› The sample correlation coefficient 𝒓 describes the linear relationship between the sample observations of the two variables in the same way as the population correlation coefficient 𝝆.

› The sample correlation coefficient is calculate using the following formula:

𝒓 = 𝒏σ𝒙𝒚 − (σ𝒙)(σ𝒚)

𝒏σ𝒙^𝟐 − σ𝒙 ^𝟐 𝒏σ𝒚^𝟐 − σ𝒚 ^𝟐

(37)

MegaStat Application

(38)

Example 9.7.1 and Example 9.7.2

› Table 9.7.1 shows a subject’s height (cm) and the peak spinal

latency (Cv) of the SEP (a type of electrical activity of the brain).

Investigate the relationships between a subject’s height and the Cv of the SEP. Use the sample correlation coefficient to check if it is of sufficient magnitude to indicate that, in the population, height and Cv SEP levels are correlated.

(39)

Correlation Analysis (MegaStat Application)

2. Select Correlation Matrix.

(40)

6. Choose “greater than”.

3. Input the range of information, then

(41)

› MegaStat indicates that the

correlation coefficient is equal to 0.848 (strong positive linear

relationship).