A new (and better) coefficient stability test

Similar to many other applied microeconomists, I find myself using coefficient stability tests in nearly all my papers. Apparently, I am not unique. The methods developed by Altonji, Elder, and Taber (2005) and Oster (2019) have thousands of citations, many from top general interest and field journals.

A new paper, by Paul Diegert, Matthew Masten, and Alexandre Poirier seeks to improve upon these now-standard methods of testing coefficient stability to unobservable sources of selection bias. I think this paper is set up to provide the next standard method for sensitivity analysis. Here is why…

But first, what is the problem Diegert, Masten, and Poirier are trying to solve? Oster’s (2019) method relies on several assumptions. One of which is the “exogenous controls assumption.” This states that the omitted variables must be uncorrelated with the included controls. This is, to put it lightly, a big assumption. To quote Dieger, Masten, and Poirier:

For example, consider a classic regression of wages on education and controls like parental education. Typically we are worried that the coefficient on education in this regression is a biased measure of the returns to schooling because unobserved ability is omitted. To apply Oster’s (2019) method for assessing the importance of this unobservable variable, we must assume that unobservd ability is uncorrelated with parent’s education, along with all other included controls.

The method developed by Diegert, Masten, and Poirier aims to provide an approach to sensitivity analysis that allows for the included control variables to be endogenous. This seems like a very useful approach since, in many (or most) practical empirical settings, the included control variables are likely to be endogenous. The entire paper is worth a read, but perhaps the most useful contribution of the authors is their release of a new Stata command: “regsensitivity” along with an accompanying vignette.

I’ve just started playing around with the vignette, but the results look very useful! For example, the Stata command allows researchers to generate graphs illustrating the “breakdown” point for their main results and that allows for variation of “cbar” a parameter indicating the level of endogeneity of the included control variables.

I am looking forward to applying this method in a paper as soon as possible. And I expect many others will be as well if they have not done so already.