MATH 1401 — Sections 4.1 & 4.2 — Least Squares Regression and Correlation
MATH 1401 · Dr. Barry Monk

Least Squares Regression and Correlation

Section 4.1 & 4.2
Two prospective buyers shake hands with an estate agent holding a clipboard, in front of a modern two-storey house with dark timber cladding.

Construct scatterplots

Suppose that a real estate agent wants to study the relationship between the size of a house and its selling price. It is reasonable to suspect that the selling price is related to the size of the house. Specifically, we expect that houses with larger sizes are more likely to have higher selling prices.

A good way to visualize a relationship like this is with a scatterplot. In a scatterplot, each individual in the data set contributes an ordered pair of numbers, and each ordered pair is plotted on a set of axes.

The Data

Size (Square Feet)Selling Price ($1000s)
2521400
2555426
2735428
2846435
3028469
3049475
3198488
3198455
Calculator

Constructing a Scatterplot

The following steps will create a scatterplot for the house sizes and prices data using the Statistics Calculator at barrymonk.com/stats-calculator.

Step 1: Enter the eight sizes in L1 and the eight prices in L2.

The tool strip across the top of the calculator: Summarize, Scatterplot, Regression, Confidence interval, Hypothesis test, Frequency, Binomial, Normal, Sampling, Discrete and Probability. Scatterplot is selected.

Step 2: Choose Scatterplot from the tool strip.

The resulting scatterplot of L2 against L1: eight points rising from lower left to upper right, with two points at the same horizontal position and different heights. No line is fitted.

Step 3: Set x values to L1 and y values to L2, then press Run.

Positive Linear Association

Observe that larger sizes tend to be associated with larger prices, and smaller sizes tend to be associated with smaller prices. We refer to this as a positive association between size and selling price.

In addition, the points tend to cluster around a straight line. We describe this by saying that the relationship between the two variables is linear.

Therefore, we can say that the scatterplot exhibits a positive linear association between size and selling price.

Size 2521
Selling Price 400
Drag across the plot to step from house to house.
Definition

Other Types of Association

Two variables are positively associated if large values of one variable are associated with large values of the other.

Two variables are negatively associated if large values of one variable are associated with small values of the other.

Two variables have a linear relationship if the data tend to cluster around a straight line when plotted on a scatterplot.

Check Your Understanding

State the type of association that is exhibited:

Choices: positive linear, negative linear, positive nonlinear, negative nonlinear, weak or no association.

a
Negative linear
b
Positive nonlinear
c
Positive linear
d
Weak or no association
e
Negative nonlinear
A gray clapboard house with a white porch behind a red-and-white HOME FOR SALE sign topped with a SOLD rider.
Definition

Correlation Coefficient

When two variables have a linear relationship, we want to measure how strong the relationship is. A numerical measure of the strength of the linear relationship between two variables is called the correlation coefficient.

r=1n1(xx¯sx)(yy¯sy)

Properties of the Correlation Coefficient

1

The correlation coefficient is always between −1 and 1. That is, 1r1.

2

The correlation coefficient does not depend on the units of the variables.

3

It does not matter which variable is x and which is y.

4

The correlation coefficient only measures the strength of the linear relationship. It can be misleading when the relationship is nonlinear.

Interpreting the Correlation Coefficient

If r is positive,

the two variables have a positive linear association.

If r is negative,

the two variables have a negative linear association.

If r is close to 0,

the linear association is weak.

The closer r is to 1, the more strongly positive the linear association is.

The closer r is to −1, the more strongly negative the linear association is.

Correlation Coefficient with Technology

Correlation coefficient r 0.901 between size and selling price

There is a strong positive linear relationship between size and selling price.

The Correlation Coefficient is Not Resistant

A statistic is resistant if its value is not affected much by extreme values (large or small) in the data set. The correlation coefficient is not resistant.

The plot shows area of farmland versus total land area for a selection of U.S. states. Alaska is an outlier. Because of the outlier, the correlation coefficient for this plot is −0.119, which is misleading. If the outlier is removed, the correlation coefficient for the remaining points is r = 0.710.

An aerial view of a green crop field, its rows converging toward a distant treeline under a blue sky.
A scatterplot of farmland area against total land area for a selection of U.S. states. Most states cluster at the left, below about 160 thousand square miles of area, and farmland generally increases with area. One point, labeled Alaska, lies far to the right at about 572 thousand square miles with almost no farmland.
Interactive

Correlation Explorer

Correlation coefficient r 0.901

Drag any house. The correlation coefficient is recomputed as it moves.

Think About It

Correlation is Not the Same as Causation!

A line chart from tylervigen.com titled "US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation", reporting a correlation of 99.79 percent. Two lines track each other closely from 1999 to 2009.

Correlation: 99.79%  (r = 0.99789126)

A line chart from tylervigen.com titled "Per capita cheese consumption correlates with Number of people who died by becoming tangled in their bedsheets", reporting a correlation of 94.71 percent. Two lines track each other closely from 2000 to 2009.

Correlation: 94.71%  (r = 0.947091)

Two prospective buyers shake hands with an estate agent holding a clipboard, in front of a modern two-storey house with dark timber cladding.
Section 4.2

The Least-Squares Regression Line

Definition

Least-Squares Regression Line

We want to find the line that “best fits” the data.

This line is called the Least-Squares Regression Line.

Total area of the eight squares 1345.4 It cannot be pushed below 1165.3.
Drag either end of the line.
Calculator

Computing the Least-Squares Regression Line

The following steps compute the least-squares regression line for the house size and selling price data using the Statistics Calculator at barrymonk.com/stats-calculator.

Step 1: Choose Regression from the tool strip.

Step 2: Set Explanatory variable x to L1 and Outcome variable y to L2.

Step 3: Press Run. The command bar composes LinReg(a+bx) L1,L2.

The Regression form. Explanatory variable x is set to L1, Outcome variable y to L2, and Predict at x is empty.
Calculator

Reading the Regression Output

Running LinReg(a+bx) reports the least-squares regression line, its intercept and slope, r and r2.

The regression result. Least-squares line: intercept b-zero 160.1939, slope b-one 0.0992. Fit: r squared 0.811, r 0.901, eight paired points. Prediction: none requested. Beside them, a plot of the eight houses with the least-squares line drawn through them in red.
The equation of the least-squares regression line is
y^=160.1939+0.0992x
Example

Predicting a Value

To predict the selling price of a house of a given size, substitute the size for x in the least-squares regression line.

y^=160.1939+0.0992x
Size (Square Feet)2900
Predicted Price ($1000s)447.9

A house of 2900 square feet is predicted to sell for about 447.9 thousand dollars.

Interpreting the Regression Coefficients

For the house size and selling price data,
y^=160.1939+0.0992x
Interpretation of y^

For a given value of x, the predicted value y^ estimates the average y-value for all individuals with that x-value.

For a house of 2900 square feet, y^ = 447.9 estimates the average price of all houses that size — not the price of any one house.

Interpretation of b0

The y-intercept, b0, is the point where the line crosses the y-axis. This has a practical interpretation only when the data contain both positive and negative values of x.

Interpreting the Regression Coefficients

Interpretation of b1

If the x-values of two points on a line differ by 1, their y-values will differ by an amount equal to the slope of the line. For example, if a line has a slope of 4, then two points whose x-values differ by 1 will have y-values that differ by 4. This enables us to interpret the slope b1. If the values of the explanatory variable differ by 1, their predicted values will differ by b1.

If the values of the explanatory variable differ by some other amount, say d, their predicted values will differ by db1.

Here b1 = 0.0992 thousand dollars per square foot — about $99 for every extra square foot.

Two houses, one difference

House A is 2700 square feet — predicted price 428.0.

House B is 2850 square feet — predicted price 442.9.

The long way  442.9 − 428.0 = 14.9

The short way  the sizes differ by 150, so (0.0992)(150) = 14.9

Two routes, one answer. Once you have the slope, you never needed the two predictions.

House B size (square feet) 2850

The long way  442.9 − 428.0 = 14.9

The short way  the sizes differ by 150, so (0.0992)(150) = 14.9

Think About It

Do Not Extrapolate

The sizes in the data run from 2521 to 3198 square feet. The regression line describes that range. It says nothing about houses outside it.

Size (Square Feet)4000
Predicted Price ($1000s)557.0

Outside the data. No house this size was observed, so this prediction rests on nothing.

Inside the range of the data, the prediction is supported by the houses around it.

Five students seated in a row at a long table, heads down, writing with pencils.
Check Your Understanding

At the final exam in a class, the professor asks each student to indicate how many hours they studied for the exam. After grading the exam, the professor computes the least-squares regression line for predicting the final exam score from the number of hours studied. The equation of the line is y^=50+5x.

Antoine studies for 6 hours. What is the expected exam score?

80

Emma studied for 3 hours longer than Jeremy did. How much higher is Emma’s score predicted to be?

15 points higher

You are ready for Homework 4.1 & 4.2

1 / 22 → ← advance · Z zoom · B blank · H hides this bar