Basic Principles of Hypothesis Testing

Section 9.1

Null and Alternate Hypotheses

Every hypothesis test begins with two competing statements about a population parameter.

The null hypothesis, denoted $H_{0}$ , states that a parameter is equal to a specific value. It represents "no effect" or "no difference."

The alternate hypothesis, denoted $H_{1}$ , states that the parameter differs from the value specified in the null hypothesis.

Example

Formulating Hypotheses

A manufacturer claims their smartphone battery lasts 8 hours of continuous use. Consumer advocates suspect it's actually less.

Six identical smartphones lie face up in a row on a test bench, each plugged into its own charging cable.

Solution

H_{0} : μ = 8

H_{1} : μ < 8

This is a left-tailed test.

Example

Formulating Hypotheses

A restaurant lists a salad as containing 450 calories. A nutrition researcher suspects that the actual average calorie count is higher.

Solution

H_{0} : μ = 450

H_{1} : μ > 450

This is a right-tailed test.

A plated garden salad of lettuce, cucumber, cherry tomatoes, shredded carrot and radish on a restaurant table.

Example

Formulating Hypotheses

An airline reports that flights depart an average of 12 minutes late. After an overhaul, the operations team wants to determine whether it has changed.

Solution

H_{0} : μ = 12

H_{1} : μ \neq 12

This is a two-tailed test.

Four travelers wait in a row of seats at an airport gate, one reading a book, while a plane sits outside the window.

Check your understanding

State the Hypotheses

For each situation, state the null and alternate hypotheses, and say whether the test is left-tailed, right-tailed, or two-tailed.

(a)A coffee shop claims its medium latte contains 16 ounces on average. A customer suspects it's less.

H_{0} : μ = 16

H_{1} : μ < 16

Left-tailed

(b)Last year, the mean commute for a company's employees was 28 minutes. After a new flexible schedule, the HR director wants to know whether it has changed.

H_{0} : μ = 28

H_{1} : μ \neq 28

Two-tailed

(c)A tire maker says its tires last 60,000 miles on average. Its engineers believe a new tread design lasts longer.

H_{0} : μ = 60,000

H_{1} : μ > 60,000

Right-tailed

The Logic of Hypothesis Testing

The reasoning behind hypothesis testing is similar to a criminal trial.

In a trial, the defendant is assumed innocent until proven guilty. The prosecution must present evidence strong enough to overcome this presumption.

In hypothesis testing, we assume the null hypothesis is true until the evidence suggests otherwise.

We ask: "If $H_{0}$ were true, how likely would we be to observe data like this?"

A gavel rests on its block on a polished courtroom bench, with the scales of justice beside it.

The Logic of Hypothesis Testing

Assume $H_{0}$ is true

Collect data

Gather evidence from a sample

Assess the evidence

How unusual is our result?

Make a decision

Reject $H_{0}$ or fail to reject $H_{0}$

How Unusual Is Unusual?

The farther the sample statistic is from the value specified by $H_{0}$ , the less likely the difference is due to chance.

Key Question

How big should the difference be before we reject $H_{0}$ ?

Distance from the claimed value in

H_{0}

Size of the sample

Amount of spread in the distribution

The method of hypothesis testing accounts for all three.

Distance from the Claimed Value

Curve: where sample means would fall if

μ = 80

Shaded: more than 2 standard errors from 80 (unusual)

What we'll do: drag $\bar{x}$ from 78 down to 60.

What to look for: when $\bar{x}$ moves into the shaded region, the verdict changes to unusual.

Try it

Drag $\bar{x}$ down to 60.

The farther $\bar{x}$ is from the claimed value, the harder it is to believe $H_{0}$ .

At 60, $\bar{x}$ is ten standard errors from 80: unusual.

Size of the Sample

Curve: where sample means would fall if

μ = 80

Shaded: more than 2 standard errors from 80 (unusual)

What we'll do: keep $\bar{x}$ at 78 and raise the sample size, $n$ , from 25 to 200.

What to look for: how the curve and the verdict change.

Try it

Drag $n$ up to 200.

A larger sample makes the same distance more unusual.

With $n = 200$ , 78 is almost three standard errors from 80: unusual.

Amount of Spread

Curve: where sample means would fall if

μ = 80

Shaded: more than 2 standard errors from 80 (unusual)

What we'll do: keep $\bar{x}$ at 78 and $n$ at 200, and raise $σ$ from 10 to 20.

What to look for: how the curve and the verdict change.

Try it

Drag $σ$ up to 20.

More spread makes the same distance less unusual.

With $σ = 20$ , 78 is less than one and a half standard errors from 80: plausible again.

Distance, sample size, and spread: all three matter.

Stating Conclusions

There are only two possible outcomes of a hypothesis test:

If we reject $H_{0}$ :

We conclude that there is enough evidence to support $H_{1}$ .

If we do not reject $H_{0}$ :

We conclude that there is not enough evidence to support $H_{1}$ . This does not mean $H_{0}$ is true, only that we could not rule it out. That is, $H_{0}$ might be true.

An Important Distinction

We never "accept" the null hypothesis. We either reject it or fail to reject it.

Failing to reject $H_{0}$ means the evidence was not strong enough to conclude $H_{1}$ . It doesn't prove that $H_{0}$ is true.

Similarly, in a trial, a jury delivers "guilty" or "not guilty," never "guilty" or "innocent."

A wooden gavel resting on its sound block.

Example

Stating Conclusions

A manufacturer claims their smartphone battery lasts 8 hours of continuous use. Consumer advocates suspect it's actually less.

H_{0} : μ = 8

H_{1} : μ < 8

If we reject $H_{0}$ : There is sufficient evidence to believe the mean battery life is less than 8 hours.

If we do not reject $H_{0}$ : There is not sufficient evidence to believe the mean battery life is less than 8 hours.

Check your understanding

State the Conclusion

A transit agency claims that the mean wait for a bus on a certain route is 10 minutes. Regular riders suspect the mean is longer, and test $H_{0} : μ = 10$ versus $H_{1} : μ > 10$ .

(a)Suppose $H_{0}$ is rejected. State the conclusion in words.

There is sufficient evidence to believe the mean wait time is greater than 10 minutes.

(b)Suppose $H_{0}$ is not rejected. State the conclusion in words.

There is not sufficient evidence to believe the mean wait time is greater than 10 minutes.

Type I and Type II Errors

When we conduct a hypothesis test, we might make the wrong decision. There are two ways this can happen.

Type I Error

False Positive

We reject $H_{0}$ when $H_{0}$ is actually true.

We conclude something has changed when it really has not.

Type II Error

False Negative

We fail to reject $H_{0}$ when $H_{0}$ is actually false.

We miss detecting a change when it has really happened.

One wall panel with two labeled symbols: an exclamation mark in a magnifying glass labeled False alarm, and a crosshair labeled Missed detection.

Type I and Type II Errors

	$H_{0}$ is True	$H_{0}$ is False
Reject $H_{0}$	Type I Error	Correct Decision
Do Not Reject $H_{0}$	Correct Decision	Type II Error

Example

Types of Errors

A business school dean wants to determine whether the mean starting salary of graduates is greater than $50,000. She tests:

H_{0} : μ = $ 50,000

H_{1} : μ > $ 50,000

For each case, decide: Type I error, Type II error, or correct decision.

A smiling graduate in cap and gown shakes hands across a desk with a woman in a blazer, in an office lined with books.

Example

Types of Errors

Case 1: The true mean is $μ = $ 50,000$ , and the dean rejects $H_{0}$ .

$H_{0}$ is true but rejected: Type I error

Case 2: The true mean is $μ = $ 55,000$ , and the dean rejects $H_{0}$ .

$H_{0}$ is false and rejected: Correct decision

Case 3: The true mean is $μ = $ 55,000$ , and the dean does not reject $H_{0}$ .

$H_{0}$ is false but not rejected: Type II error

Recall

$H_{0} : μ = $ 50,000$

$H_{1} : μ > $ 50,000$

Example

Types of Errors

An email spam filter calculates a spam score for each incoming email:

$H_{0} : S = 0$ (the email is legitimate)

$H_{1} : S > 0$ (the email is spam)

A hand rests on a laptop's touchpad; the screen shows an inbox list with a few of the messages flagged.

	Email Is Legitimate	Email Is Spam
Mark as spam	Type I Error	Correct Decision
Deliver to inbox	Correct Decision	Type II Error

Significance Level of a Test

Type I errors are generally considered more serious because we are making a false claim.

Hypothesis tests are designed so that the maximum probability of a Type I error is small.

This value is called the significance level, denoted $α$ .

Common choices: $α = 0.05$ or $α = 0.01$

A smaller $α$ makes it harder to reject $H_{0}$ , reducing Type I errors but increasing the risk of Type II errors.

A smoke detector on a ceiling, with a thin wisp of smoke drifting below it.

A less sensitive smoke detector: fewer false alarms, more missed fires.

You Are Ready For:

9.1: Basic Principles of Hypothesis Testing

State the null and alternate hypotheses, and tell a left-tailed, right-tailed, or two-tailed test apart

Describe the reasoning: assume $H_{0}$ , then ask how unusual the data would be

State a conclusion in words, whether or not $H_{0}$ is rejected

Tell a Type I error from a Type II error, and say what the significance level controls

1 / 29 → ← advance · Z zoom · B blank · H hides this bar

Basic Principles of Hypothesis Testing

The Continue Button

The Continue Button

The Continue Button

Null and Alternate Hypotheses

Three Kinds of Alternate Hypothesis

Formulating Hypotheses

Formulating Hypotheses

Formulating Hypotheses

State the Hypotheses

The Logic of Hypothesis Testing

The Logic of Hypothesis Testing

Is the Sample Mean Plausible?

How Unusual Is Unusual?

Distance from the Claimed Value

Size of the Sample

Amount of Spread

Stating Conclusions

An Important Distinction

Stating Conclusions

State the Conclusion

Type I and Type II Errors

Type I and Type II Errors

Types of Errors

Types of Errors

Types of Errors

Significance Level of a Test

You Are Ready For: