RCTs as “Gold Standard”?

Earlier this year I read Tim Ogden’s book entitled, “Experimental Conversations: Perspectives on Randomized Trials in Development Economics“*. Including interviews from the “randomistas” (e.g. Michael Kremer, Ester Duflo, Abijit Bannerjee, Dean Karlan), “skeptics” (e.g. Angus Deaton), and folks not typically associated with RCTs (e.g. Tyler Cowen), it was an interesting book to read. One of the “dogs that didn’t bark” in the book was the statement that RCTs are the gold standard.

Continue reading “RCTs as “Gold Standard”?”

[Book Review] Experimental Conversations – Forthcoming in the AJAE

Randomized control trials (RCTs) have taken development economics, and even the broader discipline of economics, by storm. Whether this is “good” or “bad” or just another methodological “fad” largely depends on one’s perspective on development and empirical science. Timothy Ogden, the Managing Director of NYU’s Financial Access Initiative, has documented a number of these perspectives from not only those who implement RCTs, but also those who are skeptical of the use of RCTs, and those who fund and consume research. These conversations, recorded verbatim as interviews, make the subject of a new book that was published at the beginning of this year: Experimental Conversations: Perspectives on Randomized Trials in Development Economics.

Continue reading “[Book Review] Experimental Conversations – Forthcoming in the AJAE”

Links I Like [3.16]

Michael Kremer on Conducting Field Research in Developing Countries

A Guide to Debunking Debunking News Stories – Thoughts after finding out that the replication of the replication study in psychology didn’t replicate.

RCTs in development economics: What’s all the fighting about?

Microeconomists’ claims that they are doing real science turn out to be true

Bringing Farmville to the Tropics

You can finally stop feeling guilty for eating quinoa

Explaining the (negligible) Impacts of Microfinance

In a recent article over on Five Thirty Eight, Ben Casselman writes about why microloans don’t solve poverty. It is an excellent summary of the recent rigorous evaluations into the impacts of microfinance. He notes that microfinance is a $60 billion industry across 6 continents, won a Nobel Peace Prize back in 2006, and yet we are only just now understanding if it actually works.

I’ve written before (links here and here) on the 6 independent randomized control studies finding that microfinance fails to make the average participant better off and yet that it doesn’t make anyone worse off. Since these studies were published the pertinent task has been to explain these findings. Here again, Ben does an excellent job summarizing this research.

The ‘not everyone is an entrepreneur’ explanation: 

One popular explanation is that many participants in microfinance don’t use the loans to start or grow a business. Instead they use the extra cash to cover expenses or smooth consumption. When loans are used in this manor they are unlikely to cause any measurable changes in the long run. This explanation suggests that perhaps microloans is only truly valuable to a small group of people with a specific set of attributes and characteristics. The typical business owner in a developing country has not chosen to be an entrepreneur as they are simply participating in this activity by default. Expecting the poorest around the world to ‘entrepreneur themselves’ out of poverty is beginning to seem like nothing more than a pipe dream of an idealist. It seems quite obvious that only those who truly desire and aspire to grow their business will have a chance of benefiting from microloans.

What is challenging about this explanation is that there remains a lack of knowledge as to how to predict who will benefit from microloans prior to receiving the loans. This lack of knowledge causes confusion as to the explicit goals of microcredit and hinders positive iteration. This issue is further complicated by some of my (and other’s) research that examines if living under conditions of poverty for years (and perhaps generations) can squelch aspirations and other essential elements of hope. If this is the case, then microfinance programs could benefit from aiming to boost aspirations, rebuild personal agency, and diminish internalized perceived constraints. (In fact a study on such a program is ongoing as I type.)

The ‘early-adopter vs. late-adopter’ explanation:

This second explanation has a lot of nuance to it. Maybe the negligible and small impacts of microloan programs (measured in the 6 RCTs) are driven by the fact that these studies were evaluating microloan programs in places where similar type programs already substantially existed. Perhaps way back in the 1990s and early 2000s when microfinance was first being rolled out the impacts were positive and maybe even large (for those who had the skills and desire to invest in their business). We actually don’t know, but given the exuberance of many of the primary actors in the early days of microfinance (Muhammad Yunus et al.) it may be safe to assume that the impacts in the early days were not as small or negligible as they are today.

This actually is not a new idea. Ever since way way back in 1957 with the work of Zvi Griliches the idea that technology adoption within a given population follows an S-AAEAAQAAAAAAAABxAAAAJGNjNTY3OTYzLTAwNDgtNGVmMC1iNjY5LTExOWM0Yzg5ZTg4Ywshape curve has been standard. The figure to the right shows first their are “innovators” then the “early adopters” followed by the “early majority” and finally technology reaches “saturation”. (If you’re interested here is a figure showing the adoption and diffusion of technologies such as the TV, electricity, cars, etc.) What is important to note here is that this behavior largely occurs over and over for almost any technology because the benefit of adopting the technology is the greatest for those who first adopt. Other than a “strategic delay” for purposes of social learning, the benefits of a technology diminish as more and more of the given population adopts the technology. As a technology nears saturation, the “laggards” as they are sometimes called, adopt simply to “keep up with the Jones’s” and receive little to no benefit.

For example: my grandparents just got iPhones. Prior to a couple months ago they owned flip phones that were not even enabled to send or receive text messages. They didn’t spring for the iPhone 1 through 5 because they didn’t have the skills or desires to use those phones to their potential. Now they both have iPhone 6’s. The technology is no-doubt better than their old flip phones. But does the fancy “smart” capabilities benefit their day-to-day lives very much? I’d venture to guess the benefits are very small or negligible.

Now, it’d be wrong and misleading to evaluate the social and economic impact of the iPhone based on those who adopted it in 2015. The iPhone has transformed the way the world runs, specifically for those who adopted it five to six years ago. This might be what is going on with microloans. The “early borrowers” of microloans first took loans back in the late 1990s and early 2000s. The 6 randomized studies above evaluated the impact of microloans several years later, probably on the “late borrowers”, and found negligible and small benefits.

(For nerdy readers, Bruce Wydick has a note forthcoming the in Journal of Development Effectiveness specifically on this explanation.)

The Future of Microfinance

What does this mean for the future of microfinance? I think it is instructive to consider how Apple has handled iPhone technology. They’ve continued to innovate and improve. Better cameras, longer lasting batteries, increased functionality, etc. I think, when these two explanations are taken in conjunction, it is clear what microfinance organizations need to do. Innovate. Make the product better. Apply insights from behavioral science. Inspire increased aspirations. Encourage personal agency. Break down internalized constraints. Don’t just stand pat and continue to offer the plain vanilla “microfinance 1”.