How to Assess “Economic Significance”

Back in 2015, I read a book by Morten Jerven, in which the author makes the point that over 145 variables have been found to be statistically significant explanatory variables for long-run economic growth. Morten’s point is more nuanced than this, but this suggests that when interpreting regression results we need to not only consider statistical significance, but also economic significance.

This raises the question: How do we assess economic significance?

A relatively new working paper by Olivier Sterck aims to address this question.

The paper first discusses why traditional methods for assessing economic significance, such as standardized beta coefficients or R-squared decomposition methods, are unsatisfactory. Perhaps counter to intuition, standardized beta coefficients of some explanatory variable can be large for two reasons. Either its effect size is large or because it is negatively correlated with another important explanatory variable. Additionally, the sum of the standardized beta coefficients can be much larger than 1. This is particularly likely if the error term is relatively small and if the number of explanatory variables is large. This implies that standardized beta coefficients cannot be easily interpreted and compared across regression models. Similar issues are associated with various R-squared decomposition methods.

Next, the paper presents two methods for calculating economic significance. Both approaches focus on decomposing the variation of a given dependent variable into contributions associated with each explanatory variable and with the error term.

The first “ceteris paribus” approach holds the variation associated with other explanatory variables constant. This measure calculates a percentage of the “ceteris paribus” variation generated by each explanatory variable. The second “non-ceteris paribus” approach is similar but now lets the covariance with other explanatory variables influence the measure of economic importance. Taken together these two approaches have nice complementary attributes. Olivier writes,

If this latter term is positive, it means that the effect of the variable is reinforcing the effect of other explanatory variables on average. On the contrary, if it is negative, it means that the effect of the variable is suppressing the effect of other explanatory variables on average” (pp. 14-15).

The paper goes on to show an empirical application of this approach by assessing the economic importance of various factors thought to influence long-run economic growth. I’d recommend reading the entire article (not just my summary of it), here is the abstract:

The economic literature has identified dozens of statistically significant determinants of long-run growth, from malaria ecology and ruggedness to genetic diversity and the timing of the Neolithic transition. Yet, the economic importance of these factors – understood as their contribution to variation in current GDP per capita – is unknown. In this paper, I propose two complementary approaches to measure economic importance, and apply these methods to assess the importance of the determinants of longrun growth. I find that distance to coast, malaria ecology, and legal origins are the three most important factors explaining contemporary development, ceteris paribus. Temperature, the share of the population from European descent, and the timing of the Neolithic transition are also important. In comparison, ruggedness, genetic diversity, slave trade intensity, and ethnolinguistic fragmentation appear to be relatively unimportant. The effects of malaria ecology, of temperature, of the share of the population from European descent, and of the timing of the Neolithic transition are mutually reinforcing.