Multiple Linear Regression

Chapter 11: Regression Analysis: Multiple Linear Regression

Multiple Linear Regression

Regression analysis is a statistical procedure for estimating the relationship between variables. The last subchapter introduced Simple Linear Regression, which is used to predict the value of an outcome variable on the basis of a single predictor variable.

Multiple linear regression is an extension of the Simple Linear Regression model to more than one predictor variable.
$\phantom{0}$

Multiple Linear Regression

Multiple Linear Regression is a statistical procedure that is used to predict the value of a continuous outcome (dependent) variable on the basis of two or more predictor (independent) variables.

The regression line of a Multiple Linear Regression with $n$ predictor variables is described by the following regression equation:

$\hat{Y} = b_0 + b_1X_1 + b_2X_2 + \ldots + b_nX_n$

Where:

$\hat{Y}$ is the predicted value of the outcome variable $Y$ .
$X_1 \ldots X_n$ are the predictor variables.
$b_0$ is the intercept of the regression line and is often labelled the constant.
$b_1 \ldots b_n$ are the partial regression coefficients.

Example: Multiple Linear Regression Equation

Consider the following regression equation that describes the relationship between an outcome variable $Y$ and three predictor variables $X_1, X_2,$ and $X_3$ :

$\hat{Y} = 5+2X_1-X_2+4X_3$

From this regression equation, it follows that:

If $X_1$ increases by one, $\hat{Y}$ increases by $b_1=2$ .
If $X_2$ increases by one, $\hat{Y}$ decreases by $1$ , since $b_2=-1$ .
If $X_3$ increases by one, $\hat{Y}$ increases by $b_3=4$ .
If all predictor variables $X_1 \ldots X_3$ are zero, then $\hat{Y} = b_0 = 5$ .

So for example, if $X_1 = 1$ , $X_2 = 2$ , and $X_3 = 3$ , then the predicted value of $Y$ is:

$\begin{array}{rcl} \hat{Y} &=& 5+2X_1-X_2+4X_3\\ &=& 5 + 2\cdot 1 - 2 + 4\cdot3\\ &=& 17 \end{array}$