MATH 1401 · Sections 4.1 & 4.2 · Least-Squares Regression and Correlation
MATH 1401 · Dr. Barry Monk

Least-Squares Regression and Correlation

Sections 4.1 & 4.2

Chapter 3 described one variable at a time. Now we look at two variables together.

Two prospective buyers shake hands with an estate agent holding a clipboard, in front of a modern two-storey house with dark timber cladding.

House Size and Selling Price

Suppose that a real estate agent wants to study the relationship between the size of a house and its selling price. It is reasonable to suspect that the selling price is related to the size of the house. Specifically, we expect that houses with larger sizes are more likely to have higher selling prices.

Do larger houses sell for higher prices?

A good way to visualize a relationship like this is with a scatterplot. In a scatterplot, each individual in the data set contributes an ordered pair of numbers, and each ordered pair is plotted on a set of axes.

The Data

Size (Square Feet)Selling Price ($1000s)
2521400
2555426
2735428
2846435
3028469
3049475
3198488
3198455

Two houses are both 3198 square feet and sold for different prices.

Calculator

Constructing a Scatterplot

The following steps create a scatterplot of the eight houses in the Statistics Calculator.

Step 1: Open the Statistics Calculator. The eight sizes are already in L1 and the eight prices in L2.

Step 2: Choose Scatterplot.

Step 3: Set x values to L1 and y values to L2. The scatterplot appears at once.

The Statistics Calculator’s Describe data tools, with Scatterplot chosen. Below them, x values is set to L1, Size (sq ft), and y values to L2, Price ($1000s). The calculator’s scatterplot of Price ($1000s) against Size (sq ft) for the eight houses: the points rise from lower left to upper right, with two points at the same size and different prices.

Positive Linear Association

Observe that larger sizes tend to be associated with larger prices, and smaller sizes tend to be associated with smaller prices. We refer to this as a positive association between size and selling price.

In addition, the points tend to cluster around a straight line. We describe this by saying that the relationship between the two variables is linear.

Therefore, we can say that the scatterplot exhibits a positive linear association between size and selling price.

Size 2521
Selling Price 400
Drag across the plot to step from house to house.
Definition

Other Types of Association

Two variables are positively associated if large values of one variable are associated with large values of the other.

Two variables are negatively associated if large values of one variable are associated with small values of the other.

Two variables have a linear relationship if the data tend to cluster around a straight line when plotted on a scatterplot.

Check Your Understanding

State the type of association that is exhibited:

Choices: positive linear, negative linear, positive nonlinear, negative nonlinear, weak or no association.

a
?
Negative linear
b
?
Positive nonlinear
c
?
Positive linear
d
?
Weak or no association
e
?
Negative nonlinear
A gray clapboard house with a white porch behind a red-and-white HOME FOR SALE sign topped with a SOLD rider.
Definition

Correlation Coefficient

When two variables have a linear relationship, we want to measure how strong the relationship is. A numerical measure of the strength of the linear relationship between two variables is called the correlation coefficient.

r=1n−1∑(x−x¯sx)(y−y¯sy)

In practice, r is computed with technology.

Properties of the Correlation Coefficient

1

The correlation coefficient is always between −1 and 1. That is, −1≤r≤1.

2

The correlation coefficient does not depend on the units of the variables.

3

It does not matter which variable is x and which is y.

4

The correlation coefficient only measures the strength of the linear relationship. It can be misleading when the relationship is nonlinear.

Interpreting the Correlation Coefficient

If r is positive,

the two variables have a positive linear association.

If r is negative,

the two variables have a negative linear association.

If r is close to 0,

the linear association is weak.

The closer r is to 1, the more strongly positive the linear association is.

The closer r is to −1, the more strongly negative the linear association is.

Correlation Coefficient with Technology

The Scatterplot output gives r=0.901 for the eight houses.

There is a strong positive linear relationship between size and selling price.

The Correlation Coefficient is Not Resistant

A statistic is resistant if its value is not affected much by extreme values (large or small) in the data set. The correlation coefficient is not resistant.

The plot shows area of farmland versus total land area for a selection of U.S. states. Alaska is an outlier. Because of the outlier, the correlation coefficient for this plot is −0.119, which is misleading. If the outlier is removed, the correlation coefficient for the remaining points is r = 0.710.

One outlier: r = −0.119 with Alaska, r = 0.710 without it.

A scatterplot of farmland area against total land area for a selection of U.S. states. Most states cluster at the left, below about 160 thousand square miles of area, and farmland generally increases with area. One point, labeled Alaska, lies far to the right at about 572 thousand square miles with almost no farmland.
Interactive

Correlation Explorer

Correlation coefficient r 0.901

Drag any house. The correlation coefficient is recomputed as it moves.

Think About It

Correlation is Not the Same as Causation!

A line chart from tylervigen.com titled "US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation", reporting a correlation of 99.79 percent. Two lines track each other closely from 1999 to 2009.

Correlation: 99.79%  (r = 0.99789126)

A line chart from tylervigen.com titled "Per capita cheese consumption correlates with Number of people who died by becoming tangled in their bedsheets", reporting a correlation of 94.71 percent. Two lines track each other closely from 2000 to 2009.

Correlation: 94.71%  (r = 0.947091)

Two prospective buyers shake hands with an estate agent holding a clipboard, in front of a modern two-storey house with dark timber cladding.
Section 4.2

The Least-Squares Regression Line

Definition

Least-Squares Regression Line

We square the vertical distance from each point to the line and add them up.

The line for which this sum of squared distances is as small as possible is called the least-squares regression line.

Total area of the eight squares 1345.4 It cannot be pushed below 1165.3.

We want to find the line that “best fits” the data.

Drag either end of the line.
Calculator

Computing the Least-Squares Regression Line

The following steps compute the least-squares regression line for the eight houses in the Statistics Calculator.

Step 1: Open the Statistics Calculator, with the eight houses in L1 and L2, and choose Regression.

Step 2: Set Explanatory variable x to L1 and Outcome variable y to L2.

Step 3: The result appears at once, on the next slide.

The Statistics Calculator’s tool groups with Regression chosen. Below them, Explanatory variable x is set to L1, Size (sq ft), Outcome variable y to L2, Price ($1000s), and the optional Predict at x field is empty.
Calculator

Reading the Regression Output

The Statistics Calculator and the TI-84 Plus both report the intercept and the slope of the least-squares regression line.

TI-84 Plus
A TI-84 Plus LinReg screen: y=a+bx, a=160.1939146, b=.0991979543, r²=.8110655049, r=.9005917526.

y^=160.1939+0.0992x

y^, read “y-hat,” is the predicted value of y.

Example

Example: Predicting a Value

To predict the selling price of a house of a given size, substitute the size for x in the least-squares regression line.

y^=160.1939+0.0992x
Size (Square Feet)2900
Predicted Price ($1000s)447.9

A house of 2900 square feet is predicted to sell for about 447.9 thousand dollars.

Interpreting the Regression Coefficients

For the house size and selling price data,
y^=160.1939+0.0992x
Interpretation of y^

For a given value of x, the predicted value y^ estimates the average y-value for all individuals with that x-value.

For a house of 2900 square feet, y^ = 447.9 estimates the average price of all houses that size, not the price of any one house.

Interpretation of b0

The y-intercept, b0, is the point where the line crosses the y-axis. This has a practical interpretation only when the data contain both positive and negative values of x.

For these houses, b0 has no useful interpretation: it corresponds to x=0, which is outside the range of the data.

Interpreting the Regression Coefficients

Interpretation of b1

If the x-values of two points on a line differ by 1, their y-values will differ by an amount equal to the slope of the line. For example, if a line has a slope of 4, then two points whose x-values differ by 1 will have y-values that differ by 4. This enables us to interpret the slope b1. If the values of the explanatory variable differ by 1, their predicted values will differ by b1.

If the values of the explanatory variable differ by some other amount, say d, their predicted values will differ by d⋅b1.

Here b1 = 0.0992 thousand dollars per square foot.

Two houses that differ by 1 square foot have predicted prices that differ by b1 = 0.0992 thousand dollars, about $99.

Example

Example: Two Houses, One Difference

House A is 2700 square feet and House B is 2850 square feet. By how much should we predict their prices to differ?

The long way: predict both

y^=160.1939+0.0992(2700)=428.0

y^=160.1939+0.0992(2850)=442.9

442.9−428.0=14.9

The short way: use the slope

The sizes differ by 2850−2700=150.

(0.0992)(150)=14.9

Predicted prices differ by 14.9 thousand dollars, about $14,900. The slope gives the difference without either prediction.

House B size (square feet) 2850

The short way  the sizes differ by 150, so (0.0992)(150) = 14.9

Think About It

Do Not Extrapolate

The sizes in the data run from 2521 to 3198 square feet. The regression line describes that range. It says nothing about houses outside it.

Making predictions outside the range of the data is called extrapolation.

Size (Square Feet)4000
Predicted Price ($1000s)557.0

Outside the data. No house this size was observed.

Inside the range of the data, the prediction is supported by the houses around it.

Five students seated in a row at a long table, heads down, writing with pencils.
Check Your Understanding

Hours Studied and Exam Scores

At the final exam in a class, the professor asks each student to indicate how many hours they studied for the exam. After grading the exam, the professor computes the least-squares regression line for predicting the final exam score from the number of hours studied. The equation of the line is y^=50+5x.

Antoine studies for 6 hours. What is the expected exam score?

50+5(6)=80

Emma studied for 3 hours longer than Jeremy did. How much higher is Emma’s score predicted to be?

(5)(3)=15, so Emma’s score is predicted to be 15 points higher.

You are ready for

4.1 HW: Correlation

4.2 HW: The Least-Squares Regression Line

1 / 22 → ← advance · Z zoom · B blank · H hides this bar