Confidence Intervals and Margins of Error

The text below is a transcript of the video above, edited with AI assistance for clarity and flow. Where the two differ, the video is the authoritative version.

You’ll learn to

When we use a sample to draw inferences about the population, that process inevitably carries uncertainty. Confidence intervals and margins of error are the instruments we have to deal with that uncertainty. In this video you will learn to:

  • Interpret a margin of error and explain where it comes from.
  • Construct and interpret a confidence interval from a point estimate and a margin of error.
  • Explain what makes a confidence interval wider or narrower.

Random samples differ from one another

Random samples are always going to be different from one another. Suppose different pollsters are trying to find out the approval rating of a prime minister. Each pollster is going to have a different sample, and each sample is going to be slightly different from the others.

This is not a flaw. It is an unavoidable consequence of working with samples, of collecting a subset of units from the population to learn about the population. And it means that any estimate we produce from a sample will always carry uncertainty.

From sample to estimate

Suppose we want to know the Prime Minister’s approval rating among all adults. Because we cannot ask everyone, we take a random sample of 1,000 people. And suppose the sample tells us that 42% approve of the job the Prime Minister is doing. This single number, 42%, is our point estimate.

The problem with a single number

The 42% point estimate is our best guess: the single number that is the best we can do to find out the approval rating of the Prime Minister. But we will never be sure whether that is the exact approval rating we would get if we were to interview every single adult.

The true population value is unlikely to match that point estimate exactly. We need to know how much uncertainty surrounds the estimate, and this is what the margin of error does.

The margin of error

The margin of error captures the uncertainty that comes from working with samples to draw inferences about the population. A margin of error of ±3% means that the true value, the true approval rating, is likely within this boundary: 42% plus or minus 3 percentage points.

From margin of error to confidence interval

The confidence interval is this boundary. We construct it by applying the margin of error to the point estimate:

  • Lower bound: point estimate − margin of error
  • Upper bound: point estimate + margin of error

In this example, we take 42% as the point estimate. 42% plus or minus a margin of error of 3% gives us a confidence interval of [39%, 45%].

Interpreting the confidence interval

Confidence intervals vary depending on how confident we want to be. In social science we mostly work with a 95% level of confidence to draw inferences and test hypotheses.

So in this example, we are 95% confident that the true approval rating in the population lies between 39% and 45%.

What makes an interval wider or narrower?

When are we going to see a narrower or a wider confidence interval?

The first factor is sample size: larger samples produce narrower intervals. So long as the sample is a probability sample, the bigger it is, the narrower the interval is going to be. More data gives us less uncertainty and a tighter estimate.

The second factor is the confidence level. If we want to be more confident, if instead of saying we are 95% confident that the value is within the interval we want to be 99% confident, then we are going to need wider intervals. If we want more certainty, we require a broader net.

A worked example: two polls

Imagine that two organisations poll the same Prime Minister approval rating. Each is going to have its own sample.

  • Poll A: 42% approve, margin of error ±3%, giving an interval of [39%, 45%]
  • Poll B: 44% approve, margin of error ±3%, giving an interval of [41%, 47%]

The point estimates differ, but the intervals overlap. Both of these polls are consistent with the same population value. The population value could be at any point in the overlapping part of these intervals, if we trust both polls to be unbiased and accurate, and if we trust the job done by both pollsters.

When the margin of error is meaningless

The whole idea of a margin of error only applies to probability samples. Suppose we collect a non-probability sample: a pollster who only samples from people who read the same newspaper or watch the same TV channel. That pollster could still go ahead and calculate a margin of error, but it would be misleading. It would not serve its purpose. It would be a meaningless margin of error.

Recap

  • A point estimate is the best guess from the sample.
  • The margin of error expresses the uncertainty around that estimate.
  • A confidence interval is the point estimate plus or minus the margin of error.
  • Margins of error are only meaningful for probability samples. They are meaningless for non-probability samples.

Why this matters

Poll results and survey findings are reported with margins of error every day, and you will see margins of error in the studies you encounter. Knowing how to interpret the margin of error and the confidence interval is critical for reading empirical evidence.