In a previous post I described the difference between mediators and moderators, and I talked about mediators being the “why” of experiments. In other words, we can do an experiment and measure the effect of a treatment, but why did that effect occur? One way to formalize the why question is with direct and indirect effects. However, as we will see, once we define these effects clearly, they are very difficult to estimate using standard experimental designs that only manipulate the treatment.
Before defining direct and indirect effects, it is helpful to start off defining the total effect of a treatment. Using the potential outcome notation, we can write the total effect of a treatment T as
By doing a standard experiment where we randomly assign the treatment to participants, we are able to observe some people with and some with enabling us to estimate the average treatment effect. However, if we are interested in decomposing the total effect into the amount that can be attributed to some mediator M, it is useful to rewrite the total effect as
where is the value of the mediator when the and is the value of the mediator when .
Notice that the only difference between the two equations is that in the second we are now explicit about the proposed mediator. That is, and . Thus, given Eq (2), we can see that the standard experimental design enables us to estimate what happens if we change the treatment and the let the mediator change concurrently.
However, we could try to decompose that total effect, and we could ask:
- What happens if you keep the treatment fixed and change the mediator? (indirect effect)
- What happens if you change the treatment and keep the mediator fixed? (direct effect)
More formally, we can define the indirect effect of treatment t as
Note that this equation makes clear that there are actually two different indirect effects, depending on whether we fix the value of treatment to be 1 or 0.
Similarly to how we defined the indirect effect, we can also define the direct effect of a treatment t as
Again, there are actually two different direct effects depending on whether we fix the value of mediator.
Something about these quantities should look strange: they both involve quantities that we never observe for any units. That is, we never observe and we never observe . Without observing these quantities we cannot estimate the direct and indirect effects.
One way to think about why it might be hard to estimate direct and indirect effects is that we are trying to estimate two causal effects—the direct effect and the indirect effect—while only doing one experiment. This suggests that one possible way to estimate direct and indirect effects would involve doing more complex experiments that manipulate the treatment and mediators. However, as we will see in a future post, these experiments turn out to be very difficult in practice. Thus, even though mediators are very important, estimating direct and indirect effects is very difficult. Therefore, in a future post, I’ll also describe some alternative approaches learning about mechanisms.
For further reading:
- Imai, Keele, Tingley, and Yamamoto (2011). Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies.
- Gerber and Green (2012) Field Experiments: Design, Analysis, and Interpretation.