Hypothesis Testing in Stata
The text below is a transcript of the video above, edited with AI assistance for clarity and flow. Where the two differ, the video is the authoritative version.
You’ll learn to
Now that you have learned about hypothesis testing, p-values, confidence intervals and sampling, it is time to see how all of that works in Stata. You will learn to:
- Run a difference-in-proportions test in Stata.
- Run a difference-in-means test in Stata.
- Read the output and apply the confidence interval decision rule.
Materials
We use the Quality of Government dataset to test hypotheses. It is available in the Shared Project Folder.
- The dataset is
Quality_of_Government_2026.dta, in thedata-rawfolder. If you have not already downloaded it, be sure to have it in thedata-rawfolder on your own computer. - The companion do-file for this video is
hypothesis-qog.do, in thedofolder. Save it into your owndofolder, so that you can follow along and apply the same commands as we go through them.
Load the dataset
Remember to change the line that sets the working directory. It is cd followed, in quotation marks, by the path to the folder on my computer. Below it is the command that loads the dataset.
cd "C:\Users\leona\OneDrive - King's College London\methods-education\shared-project-folder"
use "data-raw/Quality_of_Government_2026.dta", clearReplace the path in cd with the location of the project folder on your own computer.
Read through the do-file comments and execute the commands
The do-file is your companion for learning how to run hypothesis tests in Stata. Its commands are the ones we go through in this video. It covers two tests: a difference in proportions and a difference in means.
From description to inference
Earlier we looked at cross-tabulations, bar charts of means and scatterplots. Now the question is about the patterns we observed, for example the differences in turnout between countries with proportional and majoritarian representation systems. Do we observe them because of chance variation, the luck of the draw? Or can we be confident that there is a real difference between the levels of turnout of those countries?
Test 1: Difference in proportions
Let us start with a different question, a difference in proportions question. Here we have two nominal variables: colonial origin, where we focus on countries with a British versus a French colonial history, and whether the country is democratic today. We are looking at the proportion of democratic countries among those with a British colonial history versus the proportion of democratic countries among those with a French history of colonisation.
- Research question: Are countries with a British colonial history more likely to be democracies today than countries with a French colonial history?
- Null Hypothesis (H0): There is no systematic difference between British- and French-colonised countries in their likelihood of being democracies.
- Alternative Hypothesis (H1): Countries with a British colonial history are more likely to be democratic than countries with a French colonial history.
Why test hypotheses when we have all the countries?
The whole context of hypothesis tests comes from a reflection about samples and populations. We began with examples from the British Election Study: we cannot interview every single voter in Britain, so we get a sample. Here, you might be asking yourself: we are looking at all British-colonised countries and all French-colonised countries, so we are looking at a population. Why then test hypotheses as if we were looking at a sample? Why even build confidence intervals and margins of error?
Social scientists still use the logic of sampling and inference when looking at countries, because the underlying idea is that whatever state of the world we observe is always limited to certain points in time. Countries that are democratic today can become non-democratic tomorrow, and countries that are non-democratic today can become democratic tomorrow. So we treat this snapshot, the country-year observations we have in this dataset, as a sample of history, so to speak.
The philosophy behind building confidence intervals here is to ask whether the difference between British- and French-colonised countries is just due to random chance, or whether there is an underlying systematic reason for the difference in the likelihood of those countries being democratic. If the proportions are really close to each other, then it could be something that in 10 or 20 years’ time will be completely different. But if we construct a confidence interval around the difference and see that it does not include zero, then we will be more confident about it.
If you are a bit lost in this discussion, that is okay. It is a bit of abstract thinking that doing social science with numbers will always involve.
Cross-tabulation
The first thing we do is look at the cross-tabulation again.
keep if ht_colonial == 5 | ht_colonial == 6
tabulate ht_colonial bmr_dem, row nofreqThe first line restricts the dataset: it only keeps the countries that were colonised by the British (5) or the French (6). The second gives you the cross-tabulation with row percentages, showing the share of democracies in each group. These are the proportions the test will compare.
Here we see that, of all British-colonised countries, 56% are democratic and almost 44% are not. By contrast, among the French-colonised countries, rounding the numbers, about 85% are non-democratic and 15% are democratic. We are going to construct a confidence interval around this difference in proportions.
Run the test: prtest
The command is prtest, which compares the proportion of a binary variable across two groups.
prtest bmr_dem, by(ht_colonial)The democracy variable, bmr_dem, is already coded 0 (non-democracy) and 1 (democracy), so we do not need to recode it. Stata will compare the proportion of 1s (democracies) in each group.
Reading the output: difference in proportions
| Element | What it tells you |
|---|---|
| Proportion 1 / Proportion 2 | Share of democracies in each group |
diff |
Estimated gap (Group 1 − Group 2) |
| 95% Conf. Interval for diff | Range of plausible values for the true gap |
Pr(|Z| > |z|) |
p-value: probability of this gap if H0 were true |
You have the share of democracies in each group, diff shows you the estimated gap, and you have the 95% confidence interval for the difference and the p-value. You can rely on this table as a reference when looking at the output.
In the output we have the proportions. A proportion of 0.56 is equivalent to 56% of the countries colonised by the British being democratic, and 15% of the countries colonised by the French are democratic today. Then comes the difference and the confidence interval around it. The confidence interval does not include zero, and the p-value is very small, a lot less than 5%. Zero is pretty far from the confidence interval around the difference.
The decision rule
- Focus on the 95% confidence interval for the difference.
- Ask: does the interval include zero?
| CI for the difference | Decision |
|---|---|
| Does not include 0 | Reject the null hypothesis |
| Includes 0 | Fail to reject the null hypothesis |
Because the confidence interval does not include zero, we reject the null hypothesis. The null hypothesis here is that there is no systematic difference in the likelihood of being democratic between the countries colonised by the French and by the British.
Test 2: Difference in means
- Research question: Is turnout higher in countries with proportional systems than in countries with majoritarian systems?
- Null Hypothesis (H0): Mean voter turnout is the same in majoritarian and proportional electoral systems.
- Alternative Hypothesis (H1): Proportional systems have higher turnout than majoritarian systems.
Before moving on: reload the dataset
use "data-raw/Quality_of_Government_2026.dta", clearWe have to reload the dataset before we move on. Earlier we ran a command to keep only the countries that were British or French colonies. If we do not reload, we will only be looking at those countries; reloading gives us the full dataset again.
Compare means
keep if gol_est == 1 | gol_est == 2
by gol_est, sort: summarize ideavt_legvtThe first command keeps only the countries with majoritarian (1) and proportional (2) systems. Mixed systems (3) are excluded: we are comparing the two pure types. The second command gives us the means by group.
The mean turnout for countries with majoritarian systems is 61%. The mean turnout for countries with proportional systems is 64.5%. Is there something meaningful in that difference? We are going to construct a 95% confidence interval around the difference in turnout levels between proportional and majoritarian systems.
Run the test: ttest
The command is ttest, which compares the mean of a continuous variable across two groups.
ttest ideavt_legvt, by(gol_est)ideavt_legvt is the percentage of the voting-age population that cast a vote, the turnout level, and gol_est is the electoral system. Stata will compare the mean turnout in each electoral system type.
Reading the output: difference in means
| Element | What it tells you |
|---|---|
| Mean | Average turnout in each group |
diff |
Estimated gap (Group 1 − Group 2) |
| 95% Conf. Interval for diff | Range of plausible values for the true gap |
Pr(|T| > |t|) |
p-value: probability of this gap if H0 were true |
Once we run the test, we see the difference: −3 percentage points between majoritarian and proportional. Next to it is the confidence interval, and the confidence interval includes zero. Zero is a plausible value for the real difference between majoritarian and proportional systems.
The p-value is larger than 0.05. It is saying that, assuming that the null hypothesis is true, there is a 30% probability of observing a result at least as extreme as this one. A 30% probability is still plausible.
Again, this table is there for you to rely on as a reference for reading the output in Stata.
The decision rule: same rule, different outcome
The decision rule remains the same.
- Focus on the 95% confidence interval for the difference.
- Ask: does the interval include zero?
| CI for the difference | Decision |
|---|---|
| Does not include 0 | Reject the null hypothesis |
| Includes 0 | Fail to reject the null hypothesis |
In this case the confidence interval for the difference includes zero, so we fail to reject the null hypothesis. The idea is that this difference in turnout levels between proportional and majoritarian systems could well be just due to random chance. If, instead of the most recent parliamentary elections, we had collected the data on parliamentary elections 10 or 20 years ago, or if we were to collect it 10 or 20 years from now, it would be likely for us not to observe a real difference between the turnout levels of these countries. This is the logic behind the test.
Note. A simplification. Failing to reject the null hypothesis does not show that there is no difference. It means only that these data do not give us enough evidence of one.
Recap
prtesttests whether two groups differ in their proportion of a binary outcome. It is the case of two nominal variables: you see a cross-tabulation, you compare proportions, and you run a difference in proportions test.ttesttests whether two groups differ in their mean of a continuous outcome. It is the case of one nominal and one numeric variable, when we are comparing means.- Both tests produce a difference, a confidence interval, and a p-value: with
prtestthe confidence interval is around the difference in proportions, and withttestit is around the difference in means. - If the 95% confidence interval for the difference excludes zero, we reject the null hypothesis. Otherwise, if it includes zero, we fail to reject the null hypothesis.
Why this matters
Descriptive analysis tells us what patterns look like in our data. Hypothesis testing is what tells us whether those patterns are too large to dismiss as chance variation. These two commands, prtest and ttest, are how you run those tests in Stata, and we will continue to run hypothesis tests in Stata in later videos.