Random Assignment and Causal Inference
The text below is a transcript of the video above, edited with AI assistance for clarity and flow. Where the two differ, the video is the authoritative version.
You’ll learn to
You have learned that confounding variables and reverse causality are the two main reasons why correlation doesn’t imply causation. Now you are going to learn about a method that can be implemented to deal with those threats when we want to test causal hypotheses. In this video you will learn to:
- Explain the counterfactual problem in causal inference.
- Describe how random assignment addresses confounding variables and reverse causality.
- Identify the limits of experimental designs.
A causal question
Let’s think of a hypothetical research question that is quite important nowadays. We could ask whether exposure to misinformation online makes a person more likely to believe in conspiracy claims. Say you are watching YouTube channels that are full of misinformation. Does that make people believe in conspiracy claims and conspiracy theories? This is something we might ask whenever we are trying to understand the reasons for certain attitudes and behaviours of voters in different countries.
We could then observe a correlation. We could run a survey of a sample of different people, and ask them what sources of information they use and what they believe, asking them directly about certain conspiracy theories. And we could observe a correlation, with more exposure being correlated with more agreement with conspiracies.
The question then becomes: are we looking at a causal relationship? Judging just from the observational pattern, from looking at the data and observing what we can observe without manipulating anything out in the world, can we conclude that one thing is causing the other? You have learned about reverse causality and confounding variables, and both of these problems are applicable to this example.
Problem 1: Reverse causality
People who already believe conspiracies could be more willing to seek out misinformation. They would be more attuned to the channels and the TikTok or Instagram accounts where people spread misinformation online. So the correlation could run from beliefs to exposure, instead of from exposure to beliefs. The causal direction is going to be unclear if all we do is collect a sample and ask people about their beliefs and their exposure to the channels or accounts that spread misinformation.
Problem 2: Confounding variables
A second problem is confounding variables. Variables such as political distrust, ideology and the level of education could influence both variables. Political distrust could lead someone both to seek out channels or accounts that spread misinformation and to believe in conspiracy theories. People of a certain ideology could be more likely to exhibit both behaviours, consuming this type of media and believing in conspiracies. The same goes for education.
So you have both the reverse causality problem and the confounding variable problem. Together they mean that you can’t conclude that there is a causal relationship just by observing the people who do or do not believe in conspiracy theories and who do or do not consume misinformation. People with those characteristics could be more exposed to misinformation and hold more conspiracy beliefs, so the confounders can make a relationship appear causal when it is actually not causal.
The counterfactual problem
Here is a key problem for causal inference: the counterfactual problem. If we really wanted to know the extent to which exposure to misinformation causes someone to hold conspiracy beliefs, we would need to be able to observe the same person in two states simultaneously. We would need something like a time machine. In one universe, someone spends a month watching misinformation; in the alternative universe, they don’t. Then we would ask them about their beliefs in conspiracy theories in both of these alternate universes.
So the person would be exposed to misinformation in one state and not exposed in the other. But that is something we cannot do. It is impossible: we can never observe the counterfactual. This is a fundamental problem that researchers have to deal with when trying to establish a causal relationship.
The logic of the counterfactual
The logic is that we want to know what a person would have believed without exposure to misinformation. That unobserved outcome, the thing that we cannot see, is the counterfactual. Causal inference is exactly the problem of estimating this counterfactual. We cannot observe it, but there are certain things that we can do to try to estimate what would have happened if, for example, someone hadn’t been exposed to misinformation.
The experimental solution
The experiment is a solution. You manipulate the environment with random assignment, by randomising exposure to misinformation. A study could randomly assign participants to different information environments. Someone with a contact at a social media company, at YouTube for example, could randomly manipulate the algorithms of different people.
- Treatment group: exposed to more misinformation content.
- Control group: exposed to more neutral and factual content.
The randomisation is really important. It is what is really doing the work: it is what allows the researcher to establish a causal relationship in this particular case.
Why randomisation works
Randomisation works because it makes the two groups similar on average. Whenever you take the characteristics of one group and measure them against the characteristics of the other group, they will be very similar on average. This allows you to rule out concerns about confounding variables and reverse causality.
When you randomise, you are making sure that the people in the treatment group and in the control group are, on average, similar to each other in all of these confounding variables: distrust, ideology, education, and just about any characteristic that you can possibly think of. So any difference in beliefs afterwards would be attributed to exposure itself, and not to these confounding variables, not to these characteristics.
What randomisation achieves
Randomisation breaks the link between exposure and those pre-existing characteristics. Confounders are not going to be correlated with the treatment assignment if the treatment assignment is completely random. It is not correlated with whether people have certain levels of education, a certain ideology, or distrust, for example. At the same time, reverse causality cannot operate, because the exposure is being imposed. It is not being self-selected.
A brief note on ethics
There are, of course, many ethical concerns about interventions like this. Deliberately exposing participants to misinformation, for example, raises ethical concerns. There are often protocols that researchers have to follow to mitigate these types of concerns. Using simulated content, or debriefing participants after a study, are potential ways to mitigate them.
But there will be certain studies that will simply not be approved. There are also cases of studies that violated certain ethical boundaries and weren’t stopped, and that then generated a lot of debate. Nowadays there are more protocols and regulations for anyone wanting to conduct research with human beings in a way that manipulates their environment or what they are exposed to.
So not every causal question is going to be answerable experimentally, for exactly these ethical reasons. This is one limitation of experiments.
When experiments are not possible
There are going to be situations where experiments are simply impossible. You cannot randomly assign people to years of education, war, or poverty. Maybe in certain situations a powerful government could, but again the ethical implications would be enormous. A lot of important political and socioeconomic questions involve variables that cannot be manipulated, so experiments are very limited in that regard.
There are other methods, other statistical methods, that social scientists can use to approximate causal inference from observational data. We are distinguishing here between observational and experimental data. Experimental data is data where you can manipulate something. Observational data is data that you collect and observe out there in the world.
Recap
- Random assignment creates comparable groups, isolating the effect of one variable: the one thing that the researcher is manipulating.
- It addresses both confounding variables and reverse causality simultaneously.
- There are situations where experiments are impossible. Then you need other tools, which statistics can deliver. Further on in your education, you will be able to learn about some of these statistical ways of inferring causality from observational data.
Why this matters
Most data in political science and in the social sciences is observational, not experimental. Understanding what experiments achieve helps you evaluate what observational studies cannot establish. This is, again, a very important skill for being able to critically evaluate claims made with data.