Hypothesis Tests Using Confidence Intervals

The text below is a transcript of the video above, edited with AI assistance for clarity and flow. Where the two differ, the video is the authoritative version.

You’ll learn to

In the previous video, you learned about constructing a confidence interval around a point estimate. Now we use the confidence interval to test hypotheses: hypotheses about differences between groups, about correlations, about whether two variables go together. Along the way you will learn what the null hypothesis is. In this video you will learn to:

  • Use a confidence interval to assess evidence against a null hypothesis.
  • Interpret results using the language of rejecting or failing to reject the null hypothesis.
  • Explain what it means to fail to reject the null hypothesis, and what it does not mean.

Where we left off

We left off from building confidence intervals around point estimates. The example was the Prime Minister’s approval rating, with an approval rating of 42%. There is uncertainty around this point estimate. The 42 was our best guess, so we have to build a confidence interval to express the uncertainty.

A 95% confidence interval gives us the range of plausible values for the true population quantity. Now we ask how we can use that interval to test a specific claim, and we will see what types of claims we test.

The null hypothesis as a claim about zero

The null hypothesis is typically a claim about zero, and we will see why. It typically claims that there is no difference between groups. In other words, it claims that the true difference is zero. We want to test this claim using a confidence interval.

The logic of the test

The logic of the test is that we construct a 95% confidence interval for the difference between two groups in our sample. Then we ask: does the interval include zero?

  • If zero is outside the interval, then zero is not a plausible value for the true difference. We reject the null hypothesis.
  • If zero is inside the range of plausible values, we fail to reject the null hypothesis.

If this sounds too abstract so far, it becomes clearer with an example.

Example 1: Gender and Reform vote

We have two variables: gender, and whether or not each person voted for Reform in the last general election.

  • Research question: Are men more likely than women to vote for Reform?
  • Null hypothesis: There is no difference between men and women in their likelihood of voting Reform.
  • Alternative hypothesis: Men are more likely than women to vote for Reform.

The null hypothesis says zero difference. It says that there is no correlation between gender and support for Reform, no difference between men and women in their levels of support for Reform. The alternative hypothesis is the hypothesis that we are trying to test.

Example 1: The estimates

These estimates are from the British Election Study. This is what we get if we construct the confidence intervals for the vote shares among men and among women, and the confidence interval around the difference.

Group Vote share (%) 95% CI
Men 13.7 [11.4, 16.3]
Women 9.8 [7.8, 12.1]
  • Difference (Men − Women): +3.9 percentage points
  • CI for the difference: [+0.7, +7.1]

Looking only at the level of support for Reform among men, we have 13.7%, and the confidence interval states that the range of plausible values for the level of support for Reform among men is between 11.4 and 16.3. For women, the point estimate is 9.8 and the 95% confidence interval is the range between 7.8 and 12.1.

The difference is 3.9 percentage points. Below it you see the confidence interval for the difference: what is the range of plausible values for the difference? When we construct it, we see that the lower bound is 0.7 percentage points and the upper bound is 7.1 percentage points.

Example 1: The test

The confidence interval for the difference has a lower bound of 0.7 and an upper bound of 7.1. The entire interval lies above zero: it starts from 0.7 and goes up to 7.1, so everything is above zero. This interval does not include zero. Zero is not a plausible value for the true difference in support for Reform between women and men. So we reject the null hypothesis.

This is how we proceed: we construct a confidence interval around a difference, and then verify whether zero is a plausible value for that difference.

Example 2: Newspaper readership and ideology

A second example is newspaper readership and ideology.

  • Research question: Do Daily Mail and Daily Telegraph readers differ in their political self-placement?
  • Null hypothesis: There is no difference in mean left-right self-placement between Mail and Telegraph readers.
  • Alternative hypothesis: Telegraph readers are more right-leaning than Mail readers.

Example 2: The estimates

Group Mean ideology 95% CI
Mail 6.1 [5.9, 6.4]
Telegraph 6.5 [6.1, 6.8]
  • Difference (Mail − Telegraph): −0.4
  • 95% CI for the difference: [−0.8, +0.1]

Ideology here is self-expressed on a scale from 0 to 10, so means of 6.1 and 6.5 both indicate that the readers of these newspapers are right-leaning. The point estimates suggest that Telegraph readers are more right-leaning than Mail readers, and the point estimate of the difference is −0.4. When we construct the 95% confidence interval for the difference, we get a lower bound of −0.8 and an upper bound of +0.1.

Example 2: The test

Zero is contained within that interval. The interval crosses zero, so zero is a plausible value: with a 95% level of confidence, zero could be the true value of the difference in the mean ideological leanings of Telegraph and Mail readers. So we fail to reject the null hypothesis.

Two outcomes, two conclusions

CI for difference Decision
Gender × Reform [+0.7, +7.1] Reject the null
Mail × Telegraph [−0.8, +0.1] Fail to reject the null

The confidence interval for the difference in support for Reform between women and men is entirely above zero, so we reject the null. By contrast, the Mail and Telegraph comparison has a confidence interval that includes zero, so we fail to reject the null. In both cases the same logic applies: does the interval include zero? Does the interval cross zero?

What “fail to reject” does not mean

Failing to reject the null hypothesis does not mean that we are proving that the null hypothesis is true. We cannot conclude that Mail and Telegraph readers have identical ideologies. It is just that we do not have enough evidence that Telegraph readers are more right-leaning than Mail readers: the evidence in our sample is not strong enough to rule out zero.

Recap

  • A confidence interval can be used to test a null hypothesis.
  • If the 95% confidence interval for a difference excludes zero, we reject the null hypothesis.
  • If it includes zero, we fail to reject the null hypothesis.
  • Failing to reject is not the same as confirming the null: it does not mean that the null hypothesis is true.

Why this matters

Confidence intervals appear in almost every piece of quantitative research. Knowing how to read them as hypothesis tests is a core skill for evaluating empirical evidence.