The Impact of Christian Theology on Economic Outcomes

Religion, spiritual practices, and faith are easily observable factors in the daily lives of people almost anywhere in the world. This leads many to speculate and theorize about the role of religion in driving economic and social outcomes. Positive correlations abound between religiosity and a host of factors that may influence economic success. Correlation, however, does not imply causation and pinning down the real causal relationship is complicated by the fact that people tend to choose their religion. Therefore, observing that people of faith experience different economic outcomes fails to account for the fact that unobservable personal characteristics may cause both religiosity and economic outcomes.

A randomized control trial, performed by a group of economists, aims to overcome some of these methodological challenges. In a recent NBER Working Paper entitled, “Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes” Gharad Bryan, James Choi, and Dean Karlan report their findings from their collaboration with International Care Ministries in the Philippines. The study randomly assigned households to attend different components of ICM’s “Transform” program, which consists of three parts: (1) A Christian Theology module, (2) a Health module, and (3) a Livelihood skills training module. The idea of the study is to disentangle the role of the first component, the Christian Theology module, in causing economic outcomes.

To do this 6,276 “utra-poor” Filipino households were randomly assigned into one of four groups: Group A received the full program consisting of each of the three modules. Group B received only the Health and Livelihood skills training modules. Group C received only the Christian Theology module. Finally group D received nothing and served as a pure control group. Therefore, the causal effect of the Christian Theology module is estimated by comparing group A with group B, and by comparing group C with group D.

To get an idea of the Christian Theology module, here is Bryan, Choi, and Karlan describing this component of ICM’s program.

Transform’s [Christian Theology] curriculum begins by teaching participants to recognize the goodness of the material world and their own high worth as God’s creation. The theme then shifts towards humanity’s rebellion against God and its negative consequences, while contrasting that with the message that “believers of Jesus will discover joy in sorrow, strength in weakness, timely provision in time of poverty, and peace in the midst of problems and pain.” The Protestant doctrine of salvation by grace—a person cannot earn her way into heaven by performing good works, but can only be saved by putting her faith in Jesus, upon which God forgives her sins as a free act of grace—is taught. The proper response to God’s grace is to do good works out of gratitude. The final section of the curriculum covers what such good works would be. They include stopping wasting money on gambling and drinking, saving money, treating everyday work as “a sacred ministry,” and becoming active in a local church community. Participants are encouraged to mitigate natural disaster risk, find hope in the midst of disasters through faith, and generally see that “life’s trials and troubles” are “God’s pruning knife” that will result in “more fruitfulness.”

So, what did the researchers find?

First, the Christian Theology module increases all four measures of religiosity. The authors consider this the “first stage” of effects, since a necessary condition to understand how religiosity impacts economic outcomes is that random assignment of the treatment increased religiosity.

Second, the authors find a statistically significant increase in income of 9.2%. This translates to about an extra $8.60 more income per month. However they also find no statistically significant effects on consumption, food security, total labor supply, or life satisfaction. This absence of evidence does not seem to be due to a methodological issue, since the study has enough statistical power to reject the null hypothesis — at the 95% level — of increases in outcome variables as small 0.06 standard deviations. This is a puzzle, since it seems reasonable that consumption, food security, total labor supply, and life satisfaction would all change with income. The lack of a change in consumption or food security is perhaps troubling because it indicates that the increase in income is not being directly used to reduce one very costly dimension of poverty.

So, what are some explanations for this finding?

1. Although there is not a change in total labor supply, the authors do note a shift away from agricultural labor activities to non-agricultural labor activities. This could lead to increased income.
2. The study cannot rule-out increased effort per work hour, so it could be that people are simply working harder and thus earning more. This seems to be a plausible outcome of the Christian Theology module that teaches that work in the public sphere is also a form of worship unto God.
3. The authors bring up the possibility of an increase in tithes and offerings, but note that due to reported giving estimates provided by pastors it is highly unlikely that these findings are entirely explained by an increase in tithes and offerings.
4. Another possibility is life in poverty presents a host of challenges that complicate the prediction of behavior that we may all expect. It could be, therefore, that households are using this extra income in a way that does actually improve their welfare and the authors simply fail to measure this dimension of life.
5. It could also be the case that the finding of an increase in income is spurious in some way. Since the study examines a large number of outcomes, probability theory suggests that the study will find a statistically significant finding about 5% of the time, even if all the data was totally random. The authors correct for multiple hypothesis testing, but concern of spurious results may still persist.

I think this is a fascinating study. Obviously, there are still a lot of unanswered questions, but this study presents a wonderful benchmark for faith-based organizations that build their entire programming strategy based on the theory of positive feedback loops between the spiritual and material spheres of life. The implementing organization, International Care Ministries, deserves a lot of credit for agreeing to go along with this project and rigorously testing the effectiveness of their program. Seeing this study discussed by the likes of The Economist and The New York Times, it seems like the effort put into the hard and tedious work of rigorous evaluation pays off.

The authors also deserve credit for designing and executing a remarkably excellent study. Even the basic evaluation design elements highlighted in the working paper show a measurable improvement on my study in Kenya that had similar aspirations, but ultimately encountered challenges. The study was randomized using a relatively large sample with sufficient statistical power. Data was collected by an independent third party (Innovations for Poverty Action) and survey methods (such as list randomization) were used to cut down on the concern of desirability bias among respondents.

The paper still needs to go through peer review, so some of the key findings may change slightly (although probably not much since the study was pre-registered with the American Economic Association). One bit of feedback I have is in regards to the handling of the outcome variables. Many of the economic outcomes are standard in empirical development economic studies, and do not require much comment. However, many of the religious outcomes (e.g. religiosity indexes) and psycho-social mechanisms (trust, locus of control, optimism, grit, self-control) are concepts that have no real quantitative measure and therefore are difficult handle econometrically. These variables are measured using an ordinal scale that ranks response categories, but is silent about the interval between these categories.

I’ve blogged about this topic (here and here) and have a new working paper out that examines ‘how much the cardinal treatment of ordinal variables matters’. The key point, I think, is that these empirical results need to be robust to monotonic increasing transformations of the ordinal scale. If the results are not robust to such transformations, then alternative econometric strategies may be necessary. I’d love to see future drafts of this paper show robustness of results to various monotonic transformations of the religious and psycho-social variables.

Nevertheless, this is an excellent study and I am looking forward to where this work will lead. My sense is that both faith-based development organizations and empirical development economists can learn from this study. Faith-based development organizations will do well to engage in more rigorous impact evaluation of their programs. At the same time empirical development economists can improve their research by considering factors, such as faith and spiritual practices, not normally included in economic models.