Mediation Analysis and the ‘Sequential Unconfoundedness’ Assumption

Students with the Centre for the Study of African Economies (CSAE) at the University of Oxford are creating a wonderful public good. The Coders’ Corner is a collection of tips and tricks for implementing useful statistical techniques in common statistical software (e.g., mostly Stata). This product represents a tremendous service to the broader research community. Almost anyone reading this blog should check out previous posts.

A recent Coders’ Corner post takes on the topic of mediation analysis. In short, mediation analysis is a type of statistical analysis potentially helpful for understanding mechanisms explaining *how* the main estimated effects affect the outcome of interest. Specifically, mediation analysis is feasible (although not necessarily valid, I will get to this later), when a researcher has variables measured at the same unit of analysis of the main treatment and outcome variables. The Coders’ Corner post highlights the work of Acharya, Blackwell, and Sen (2016), one available method for performing mediation analysis.

I like the Acharya et al. (2016) paper for at least two reasons: (i) It clearly demonstrates the core challenge implicit in mediation analysis and (ii) it develops a method for performing mediation analysis without bias. In this blog post, I will summarize this core challenge and discuss the necessary assumptions behind the Acharya et al. (2016) mediation analysis method.

A very common (and naive) approach to mediation analysis is to simply “control for” the potential mediating variable. In this method, if the estimated magnitude on the coefficient of interest falls in absolute value, then the included variable mediates the main estimated effect to some extent. Although this approach is quite common, it is wrong. If the potential mediating variable is endogenous, then this approach allows selection bias to creep into (possibly) previously unbiased effect estimates. Acharya et al. show via simulation that conditioning on an endogenous variable on the causal path between treatment and outcome can lead to considerable bias. They also demonstrate this idea in their Figure 3.

Figure 3 shows that in order to identify the effect of treatment (A) on the outcome (Y) through the mediator (M), the researcher needs to control for (or otherwise account for) all potential pretreatment (X) and intermediate (Z) confounders. This is a challenging task. It is difficult enough to defend the unconfoundedness of a single treatment in an empirical study, let alone the unconfoundedness of the treatment and the mediator. This leads to the two core necessary assumptions of unbiased mediation analysis, as highlighted in the Coders’ Corner post:

1. Unconfounded treatment. There exist no omitted variables that confound the effect of the treatment on the outcome.
2. Sequential unconfoundedness. There exist no omitted variables that confound the effect of the mediator on the outcome, conditional on (i) the treatment, (ii) pretreatment confounders, and (iii) intermediate confounders.

Applied researchers have a number of tools and tricks (e.g., randomized experiments, natural experiments, etc.) to plausibly satisfy the first assumption. The second assumption is much more challenging. The Coders’ Corner post mentions that this is a strong and ultimately untestable assumption. I’d like to dwell on this second assumption briefly because I feel it needs to be properly understood and often is obscured in applied research performing mediation analysis.

Figure 4, presented in Acharya et al. (2016), highlights a violation of the sequential unconfoundedness assumption. The gray dashed lines represent omitted confounding variables. The first omitted confounder, U_{1}, represents potential bias on the effect of treatment (A) on the outcome (Y). The existence of U_{1} will lead to bias of the core effect estimate and bias in the mediation effect estimate. The second omitted confounder, U_{2}, represents potential bias on the effect of the mediator (M) on the outcome (Y). The existence of U_{2} is what leads to the bias discussed above when conditioning on an endogenous mediating variable.

The sequential unconfoundedness assumption is difficult to satisfy. Even in the case of an experimental setting, where U_{1} does not exist, finding a meaningful mediating variable that is even plausibly unconfounded by U_{2} is tricky. Moreover, alternative methods, such as Preacher and Selig’s approach or Pearl’s front-door criterion, also require very strong assumptions that likely fail to hold in most empirical settings.

In one of my own papers, we try to support these assumptions by using Emily Oster’s unobservable selection method. We feel this is helpful, but ultimately our mediation analysis relies on an untestable assumption and theoretical reasoning. The key takeaway for me is although many applied researchers are interested in understanding the mechanisms that explain their core results, formal mediation analysis is far from credible in most cases. So, perform and interpret mediation analysis with extreme caution and care.