Topic 7 of 7

Linear regression

Step 17 · Statistics and probability topics

Describe the relationship between two quantitative variables with a scatter plot and correlation, fit the least-squares line, and use residuals to judge and use the fit.

Scatter plots and correlation

A scatter plot shows pairs (x, y). Describe its direction, form and strength. The correlation r, between −1 and 1, measures the strength of a straight-line relationship: near ±1 is strong, near 0 is weak.

Hours studied and quiz scores out of 20 for 10 people: a strong, rising, straight-line pattern.

r	Meaning
Close to 1	Strong, rising
Close to −1	Strong, falling
Close to 0	No straight-line pattern

Example: Reading r

r = −0.85 between a used car's mileage and its price. Describe the relationship.

The sign is negative: cars with more miles sell for less.
0.85 is close to 1 in size: strong.

AnswerA strong negative linear relationship

The least-squares line

The least-squares line ŷ = a + bx makes the sum of the squared vertical distances from the points to the line as small as possible. Its slope is b = r × ( $s_y$ ÷ $s_x$ ), and it passes through the point of means (x̄, ȳ).

b = r\frac{s_{y}}{s_{x}} \qquad a = \bar{y} - b\bar{x}

Example: Building the line

x̄ = 4, $s_x$ = 2, ȳ = 13, $s_y$ = 3 and r = 0.8. Find the least-squares line.

b = 0.8 × 3 ÷ 2 = 1.2.
a = 13 − 1.2 × 4 = 8.2.

Answerŷ = 8.2 + 1.2x

Example: Interpreting

For ŷ = 8.2 + 1.2x, with x in hours and y the score, interpret the slope.

The slope is the predicted change in y for each extra unit of x.

AnswerEach extra hour predicts a score 1.2 points higher

Residuals and predictions

A residual is observed minus predicted: y − ŷ. Positive residuals sit above the line. A residual plot with no pattern means a line fits well; a curve in it means the relationship is not linear.

Example: A residual

With ŷ = 8.2 + 1.2x, a person who studied 5 hours scored 13. Find the residual.

ŷ = 8.2 + 6 = 14.2.
13 − 14.2 = −1.2.

Answer−1.2: they scored 1.2 points below the line

Example: Extrapolation

Should you use ŷ = 8.2 + 1.2x to predict the score after 30 hours of study?

The data only covered 1 to 8 hours.
The line predicts 44.2, but the quiz is out of 20.

AnswerNo: predicting far outside the data is unreliable

Try it yourself

Work each one on paper first, then open the answer to check.

r = 0.95. Is the linear relationship strong or weak?
Show answer
Strong, and rising
ŷ = 3 + 2x. Predict y when x = 7.
Show answer
17
ŷ = 3 + 2x, and an observed point is (4, 13). What is its residual?
Show answer
2
r = −0.6. What fraction of the variation in y does the line explain?
Show answer
0.36, or 36%
x̄ = 10, ȳ = 50 and b = 3. Find the intercept a.
Show answer
20
A strong correlation between two variables proves that one causes the other. True or false?
Show answer
False