A lot has been said recently about the reduction in global poverty over the past few decades. Although the positive coverage of these encouraging statistics is certainly justified, important questions still remain. Many of these questions relate to the dynamics of poverty, rather than simply snapshots of static poverty. In short, static poverty measurements (which are used when estimating global poverty at given points in time) cannot distinguish between people who have been in poverty their entire life and people who happen to have had an unfortunate circumstance in the year the poverty data was collected.
I spend some of my time (technically half my time, but #gradstudentlife) working as a Graduate Research Assistant at the Minnesota Population Center primarily with the Integrated Public Use Microdata (IPUMS) Project. Most of my work is with the User Support Team helping users figure out how to make use of IPUMS data and fixing strange quirks in the data. After over a year of this job, I began to notice some frequently asked questions. In a new blog series on the Minnesota Population Center blog “Use It For Good“, I answer some of these FAQs.
A couple weeks ago I was enjoying the warm evening air in Myanmar when an article popped up multiple times on my Twitter feed. This happens from time to time in the development economics Twitterverse – and I like to take note. This particular article was published by WhyDev, and online community of development professionals that are “committed to getting development right”. I, and I suspect many readers of this blog, would fit in quite nice with the folks over at WhyDev.
The article was entitled “We’re all Storytellers (and Why it Matters)“, was well written, but (I think) skipped over some critical details. Quoting the famous Nigerian novelist, Chimimanda Ngozi Adichie, from her famous Ted Talk on the Danger of a Single Story the article concluded:
Stories matter. Many stories matter. Stories have been used to dispossess and to malign, but stories can also be used to empower and to humanize. Stories can break the dignity of a people, but stories can also repair that broken dignity.
I responded with the following comment:
Thanks for writing this Stephanie.
I’d like to point out some challenges and caveats to telling stories in development. First, one of the points of Ms. Adichie’s famous Ted Talk is that everyone has their own biases about the world. They are unavoidable. Biases are how we simplify and make sense of a complex and dynamic world. Therefore, it is very challenging to make observations about events in the world and tell stories independent of these biases. Second, straight observation is tricky and is often misleading. It may be observed that those who participate in a program (say microcredit) are better off after participating in the program. It could be, however, that these differences are caused by a selection bias where individuals who are well-organized, more motivated, and risk-loving participate in the program while individuals who don’t posses these attributes and abilities don’t participate. Straight observation will leave us unable to untangle what is truly happening. Is the program causing the observed differences or is there some other observable characteristic that is the real determining factor? Without data we don’t know.
So, yes! Stories matter and many stories matter, but the plural of story isn’t data. It is precisely due to the power of stories that we must ensure that our stories represent reality. Good storytelling must be coupled with good data analysis.
To which the author responded:
Thanks for your comment! And I totally agree that good storytelling must be coupled with good data analysis. I think the two can (and should) be quite complementary.
Rigorous data analysis is important for understanding what works and what doesn’t, and also for helping understand why. Gathering in-depth interviews can also help give insight into data trends. I recognize that taking one story (or even several stories) without context (or even with context, but that is subject to a series of biases) and then trying to say that these anecdotes are indicative of broader trends is just not a good idea. And, unfortunately, it does happen a lot.
I think the sweet spot is really in finding the stories that reflect the realities that data point to, and using each to support the other. And, where they contradict each other, to find out why. Data can be subject to manipulation and misinterpretation, and so can personal stories. I think we’re all trying to work hard to make sure that neither of those things happen.
Yesterday, my full response to the original article was posted and featured on WhyDev, entitled: “We’re all Storytellers, but the Plural of Story isn’t Data“.
I do a bit of consulting work on the side from time to time. It is usually fun and always interesting. Last week when sitting in a meeting developing an organizational toolbox for global monitoring and evaluation the question was raised, “Why don’t we just ask ‘Are you poor?’ isn’t that the most important thing to measure?”
The logic was right. When I graduated college I was, technically speaking, poor. But if you’d have asked me then ‘Are you poor?’ I would surely have said “no”. I had just graduated college and was navigating the job market for the first time. It was a stressful time with not a whole lot of income but I had opportunities on my horizon, so I did not consider myself poor.
The question picks at a more general question: Why use objective measurements (like income or assets or calorie intake) when subjective attitudes (probably) matter more?
My answer in the meeting sited the recent research on happiness. A lot of researchers are attempting to unwrap the complexity of human happiness as an angle to measure well-being. The inherent difficultly in asking somebody, ‘Are you happy?’ is happiness is multidimensional and relative on an extensive laundry list of (probably) unknown factors. For example: You could ask somebody whose relative has just died ‘Are you happy?’ and get different answers simply based on how they interpret the question rather than actual changes in well-being. Almost certainly in the strict context of the funeral of a close relative they will not admit to be happy, but in a more broad context of the direction their life is going they could no doubt admit to being happy.
It turns out, in many ways, poverty is a lot like happiness. It is inherently multidimensional and relative on an extensive list of (probably) unknown factors.
After the meeting I did some searching for anybody who had actually gone out somewhere and asked a significant number of people ‘Are you poor?’ I didn’t find much, and for good reason. I turned to Angus Deaton, currently one of the world’s leading experts in poverty measurements, and his book Measuring Poverty for some guidance.
Everyone has some idea of what poverty is, and most people have little difficulty answering the question, “Do you consider yourself poor?” although some people need a moment or two to think about it. Nor do people find it hard to answer the same question about their neighbors or other people that they know. Yet these simple ideas turn out to be hard to extend to countries, and harder still to the world as a whole.
Why is it hard to extend these ideas to nations or the world in general? It is tricky to ask somebody if they are poor because the actual measurement may pick up on warped incentives rather than a subjective measurement of poverty identification.
A Participatory Rural Assessment, usually known by its acronym PRA, is a procedure often used by researchers and by non-governmental organizations (NGOs) working in villages in poor countries. These researchers sit with the villagers at the local gathering place and find out about the village, mapping its houses, the school, the water supply, its agricultural activities, and who lives where. It is common to ask the villagers to say who is well-off, who is not so well-off, and who is poor and, in most cases, villagers have no difficulty in making the identification. No doubt there are some mistakes, and some people conceal some assets from their neighbors, but the results usually make sense. The poor are often people who cannot work because they are ill or suffer from a long term disability, or are elderly. They are also poor and vulnerable groups in specific locations, such as those Indian widows who are unfortunate enough not to have sons to support them. Such information can sometimes be used as part of poverty relief efforts. In India, one scheme, the Antyodaya (last man first) food program, relies on local councils to identify the very poorest few percent of rural households, who receive subsidized food rations. There is a similar scheme in Indonesia. But it is possible to push this local poverty identification too far. If the sums to be distributed are large enough, they become worth misappropriating, and their is an incentive for people to identify their friends and relatives (or themselves) as poor. Similarly, some NGOs have discovered that, if they use the poverty identification to enroll people into employment or training schemes, then after a few visits everyone is reported to be poor.
The key lesson is clear. Measuring poverty is an approximation and always wrong in one way or another. Poverty identification almost certainly is a measurement we care most about, but it is extremely difficult – if not impossible – to measure with any sort of precision or accuracy. The next best thing is to measure a bunch of objective indicators that tend to be associated with people who most likely identify as being poor. Which is, basically, what we already try to do.