Although experiments can be extremely clean, they are also limited in what they can teach us about the world. By itself a single experiment often does not tell us why the observed effect happened or under what conditions the effect might be larger or smaller. That’s where “mediators” and “moderators” come in. These two concepts are about the why and where of experiments. Without understanding the moderators or the mediators, I don’t think that a single experiment actually teaches us all that much.
First, what is a moderator? A moderator is a feature of the experimental situation that might change the size of the effect of the treatment. For example, imagine studying the effect of a criminal record on the chance of finding a job. This effect might depend on the age of the applicant; that is, employers might be especially wary to hire a young ex-offender but not concerned about hiring an older ex-offender.
Having a moderator like this complicates the interpretation of the experiment and can even lead to failures to replicate if experimenters are looking in different parts of the parameter space. If there was only one moderator (e.g., employee age) then we might stand a chance of figuring this out. But, what if there are many moderators: type of job, level of education of the applicant, race of the applicant, unemployment rate in the local labor market, and so on? Very quickly the parameter space becomes large, and it is easy to imagine that one experiment—no matter how beautifully crafted—cannot tell us much about the set of all possible experiments. Thus, moderators are closely related to the problem of external validity. That is, to what range of settings does this result hold?
So what are mediators? Mediators are the pathways through which the treatment effects the outcome. Taking an example from Gerber and Green (2012), vitamin C is the mediator in the relationship between limes and scurvy. James Lind conducted the first experiment which demonstrated the link between citrus and scurvy in 1747 aboard the HMS Salisbury (Carpenter, 1986, p 51-54). However, it was not until 1932 that scientists could reliably show that vitamin C prevented scurvy (Carpenter, 1986, p 191). That is, it took about 200 years to identify the mediator in the relationship between limes and scurvy. As this example illustrates mediators, while extremely important, can be very difficult to discover, even in the case where there is one very strong mediator (e.g., vitamin C). In many social settings, treatments probably operate through many pathways which makes isolation mediators extremely difficult. In a later post, I’ll write more about why mediators are so hard and what we can do about it.
Finally, it is important to keep a conceptual distinction between moderators and mediators. Both moderators and mediators involve a variable related to effect of the treatment on the outcome. However, moderators are conditions which adjust the size of the effect and mediators are the pathways through which the effect occurs.
For more on moderators and mediators see:
- Gerber and Green (2012) Field Experiments: Design, Analysis, and Interpretation
- Baron and Kenny (1986) The mediator-moderator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations.
Wheels on the bus 