Data Collection in Developing Countries… Stories from the Field

Consider the following unrelated facts:

  • About 1 in every 19 Kenyan children dies before his or her fifth birthday.
  • Only 9 percent of health facilities in Bangladesh offer diagnostic services for tuberculosis.
  • The percentage of households in Uganda owning at least one insecticide-treated anti-malarial net increased from 16 percent in 2006 to 90 percent in 2014-2015.

Our first reaction to numbers like these is to imagine what they imply for the lives of people in those countries. But there’s another dimension to the statistics that’s easy to overlook: someone had to go to the field and collect the underlying data. Behind each of these neatly summarized findings (from the U.S. Agency for International Development’s Demographic and Health Surveys (DHS) program) lie stories of personal hardship, risks, and logistical challenges faced by data gatherers—the unsung heroes of international health and poverty research.

This is from a recent article that made the rounds in ‘development Twitter’, which concludes…

Most charities and aid agencies don’t publicize these behind-the-scenes stories, mainly because they see the challenges of working in the field to be an everyday part of the job. But these aren’t everyday jobs. Next time you see health or poverty statistics, or data used to evaluate the effectiveness of charities, take a moment to reflect on the hard work and dedication that must have gone into collecting that information.

I’ll add two short stories. I hope others add more.

The first is from Kenya. I was working on an impact evaluation for an NGO. A local pastor and I were driving around Kitale Kenya and had to drive about an hour out of town to catch up with an individual in our sample. Just as we arrived at the farm the individual owned, a heavy rain began. After performing the survey the individual, refusing to let us leave in the middle of the rain, fead us tea and mandazi. After the rain stopped we returned to our car, which we found had sunken into the mud. We spent roughly the next hour digging our car out of the mud. The road (actually pictured in the header of this blog) leading back to town was long, narrow, and now sufficiently muddy. As we attempted to drive, the car inevitably slid off the side of the road and into the surrounding bushes and ditch. Eventually a fairly large group of kids came and helped guide our car on the center of the road and away from the deep and muddy puddles on each side until we reached a semi-paved road. All told, we spent an entire afternoon just recording one individual’s survey.

The second is from Myanmar. I was working with the Myanmar Development Resource Institute implementing a rural household survey in Mon State, a coastal region close to the boarder with Thailand. The survey was designed to provide statistics that were representative of Mon State. Representative statistics are so essential to modern policy-making that the effort required to collect such data is often glossed over. To collect this data our wonderful research team and enumerators traveled throughout rural Mon State – in cars, but also by boat across water and on foot along rice paddy bunds. Once at the survey location, enumerators tirelessly walked through a 2-3 hour survey covering topics such as assets, agricultural activities, and household consumption. Below are a few pictures from the Mon State Rural Livelihoods Household Survey.



IMG_3425 IMG_2300


Myanmar Days [Part 2]

Here’s a quick update on my time Myanmar (which is quickly coming to an end, for now). Two weeks ago I shared some pictures from my first several days in the country. Since then I have started to do some pretesting for a questionnaire I will be developing on the economics of hope in rural Myanmar. I have hope (hope 2) that this will be able to contribute to my masters thesis.

More on that will be discussed and shared later. For now, I’d like to share some pictures.

Me in front of (formally) the largest lounging Buddha
Me in front of (formally) the largest lounging Buddha
On my running route, overlooking the city of Mawlamyine
On my running route, overlooking the city of Mawlamyine
Two members of the research team and me, pretesting the for the hope module
Two members of the research team and me, pretesting for the hope module
Taking a boat to an island village for pretesting visit
Taking a boat to an island village for a pretesting visit
The research team had a weekend retreat, and was able to tag along!
The research team had a weekend retreat (at the base of a cliff), and they let me tag along! 

Myanmar Days

Friends and family will be happy to know I am safely in Mawlamyine, where I will be living and working for the next three(ish) weeks. I have yet to really get in the trenches with work so for now I’ll share some pictures of Mawlamyine.

Mawlamyine is the capital of Mon State (the region MDRI, IFPRI, and MSU are administering a survey representative of the entire region) and the former capital of Burma when the British ran everything. In that time, Mawlamyine was called “Little England” because of all the British who lived here. The city’s claim to fame in popular culture is being referenced to by two famous authors. Rudyard Kipling’s poem “Mandalay” opens with the stanza:

By the old Moulmein pagoda
Lookin’ lazy at the sea
There’s a Burma girl a-settin
‘and I know she thinks o’ me.

George Orwell, Author of “Burmese Days” began his famous essay “Shooting an Elephant” with the striking passage:

In Moulmein, in Lower Burma, I was hated by large numbers of people—the only time in my life that I have been important enough for this to happen to me

I was able to go for a run through town this evening and took some pictures. This is the third country (not counting the United States) where I’ve lived/worked that was colonized by the British. Notice how European it looks in places yet remains distinctly Southeast Asian:

Bardge coming into port
Barge coming into port
The boardwalk
The boardwalk
View from my room
View from my room
Fishing boat at sunset
Fishing boat at sunset
One of the Mawlamyine pagodas
One of the Mawlamyine pagodas

What Makes a Good Economist?

Today is the day I start my graduate training and take one more step towards becoming an actual economist! It is exciting for sure, but certainly daunting as well. Just listen to the ambiguity of this video’s answer to the question, “What makes a good economist?” 

I like this video for many reasons, I hope I can (someday) become a good economist.


One Year in Kenya [Lesson 5/5]

In any foreign country, I will always be a foreigner.


I know, I know! This seems obvious. Maybe it is. I think it is a really important lesson, however. I’ve written a lot about the concept of ‘home’ before (here and here). Here’s the critical nuance: while certain aspects of life in Kenya have come to feel like home and it may be important to strive to create ‘home’ wherever we happen to be at any time, it is important to understand that in any foreign country I will always be a foreigner. I am an outsider and it would be a mistake to think otherwise.

There are large swaths of life that I will never fully understand. There is stark need for humility in all aspects of life. I’ll illustrate this thought with a fun story, to be sure however, the lesson goes much deeper.

A couple months ago I was riding a boda boda (motorbike taxi) into town to run a couple errands. At this point I was fairly confident with myself in the task of riding a boda boda into town. I had been living in Kitale for over 6 months, I could hardly go anywhere in town without being recognized, I was maybe a little cocky.

About halfway into our 5 minute ride, the driver turned to me and asked, “Is it ok if I stop for petrol?” Of course, I politely agreed, but inside I was panicking. What am I supposed to do? Do I stay seated on the bike? Do I get off the bike? How am I supposed to behave? It seems like such a meaningless part of life, and it probably is for many local Kenyans, but for me it was existential.

I was forcing this place to become my home so much that I was overlooking the necessary humility I needed to have. The difficulty is, constantly behaving with humility causes constant anxiety, because you realize that you actually have no idea of what is going to happen next and how you ought to react to whatever happens. This anxiety is exhausting.

1342902911_lawrence-of-arabia-movieWithout this humility (and constant anxiety) outsiders force themselves to reduce reality down to something that they can understand, comprehend, and manage. There is a term for this used by art and literary historians (and increasingly cultural and development scholars): orientalism. In art and literature this is the depiction of Middle Eastern or East Asian cultures by artists and writers from the West. (See Lawrence of Arabia as an example.) The phrase often used by orientalist critiques is reductive repetition. Reducing aspects of culture down to something manageable and creating something that gels with these basic characteristics.

Orientalism is dangerous in development practice and rife in development studies. Consider the following from the abstract of the most cited paper on this topic:

Contemporary Africa is generally depicted as a ‘failure’. ‘Progress’ has eluded the continent throughout the 20th century, and despite new ways of thinking about the reasons for failure and possibilities for success, allusions to the ‘natural weakness and incapacity’ of Africans and their social realities remain evident in theoretical, policy and political discourse on development in Africa. The practice of ‘reductive repetition’, as identified by Abdallah Laroui and Edward Said, has been imported into African development studies from Orientalist scholarship. Reductive repetition reduces the diversity of African historical experiences and trajectories, sociocultural contexts and political situations into a set of core deficiencies for which externally generated ‘solutions’ must be devised. In the field of development studies, the notion of development is introduced to Africa as a deus ex machina [god from the machine]. (emphasis added)

Said differently, outsiders suffer from ‘the god complex‘ and pretend to ‘be experts’ and ‘know what they are doing’ by bypassing humility and reducing a setting down to core deficiencies which make sense and are manageable. I find myself writing the following over and over on this blog: most development programs that fail, fail because a complex situation was oversimplified.

This brings me back to my first lesson. The more I learn about the world the more I find myself saying, “I don’t know”. Perhaps this is the main reason why experience abroad is so important for a career in development work. Spending time in Kenya is not so that I will be able to understand life in Kenya. That will never fully coalesce. Experience abroad teaches you that ‘being an expert’ and ‘knowing what to do’ are impossible for an individual to achieve.

One Year in Kenya [Lesson 4/5]

Effective organizational marketing does not equal ‘good’ development work.

One of my work responsibilities was to write stories for the marketing departments of the two organizations I was affiliated with. Initially I was excited about this responsibility, but I soon recognized frustrations and limitations.

Since people everywhere relate best to stories, the marketing departments wanted stories from me about individual people. Conversely, my job, as a researcher, was to analyze and evaluate impact in the aggregate, for a population of individuals. As a result, any single story I wrote about an individual felt weak, anecdotal, and potentially misleading. Who’s to know whether this individual is an extraordinary outlier or an average Joe? (For more on this see The Danger of a Single Story.)

Several months into my time in Kenya, I recognized the distinction at the heart of my troubles. It is sometimes difficult to differentiate, but the difference between someone doing advocacy and someone doing science is always crucial.

Organizational marketing, particularly for non-profits, is essentially advocacy work. In order to garner support (both financial and otherwise) marketing must ‘sell’ people on the idea of the organization’s work. In order to do this, marketing must be confident and convicted that the work the organization does is, in fact, ‘good’.

This attitude flies in the face of that of a scientist. Particularly in evaluation work, the default attitude is to be constantly critical and almost superficially unsure. The strongest conviction of a good evaluator is his or her own doubt. The three most important words for a scientist are: “I don’t know”.

It is often difficult to differentiate between the two, as good advocacy actually sites and quotes science. Good science does this too, but there is a key difference. Advocacy aims at pushing an idea or concept while science simply (or not so simply) aims at pushing the truth.

This distinction comes to the fore when making sense of this article by Michael Mathethson Miller of Acton Institute latently written about the work of the organization I work for. It is a brilliant form of advocacy and marketing, but it is certainly not science. It absolutely and unequivocally should not be used to inform or direct program strategy or policy decisions. The article is designed to get people, who usually don’t think about this sort of stuff, to think about poverty alleviation (or ‘wealth creation’, whatever you want to call it. It’s the same thing) more deeply. It is wonderful advocacy, but terrible science, and confusing the two can be fatal both for the work of the organization and, sadly, for the people the organization aims to serve.

The development blog WhyDev recently wrote about the juxtaposition between effective marketing and good development. Effective marketing raises attention and donations. Good development work should improve the lives of poor people. Here are five reasons why the two are incompatible:

  1. We have short attention spans
  2. There is no incentive to translate complexity
  3. Even if it offends some, on balance, dumb simple is better
  4. Money drives the work, not the need
  5. Effective marketing draws on herd mentality

[For a remarkably well written story (yes, story!) about the danger of confusing advocacy and science – and really much more, read Nina Munk’s brilliant book: The Idealist: Jeffrey Sachs and the End of Poverty.]

HT: Brett Keller on the advocate vs. scientist distinction

One Year in Kenya [Lesson 3/5]

‘Africa time’ is hokum.

This is not a rant about all the time I’ve waited for meetings to start or for events to end. I actually think the idea that a conception of time is geographically based is rife with lazy logic.

Yes, people from some parts of the world often seem to act based on so-called cultural perceptions of time. Yes, during my time in Kenya, I’ve waited for almost every meeting to start well after the agreed upon time. Yes, I’ve attended weddings where ‘lunch’ was served at ‘dinner time’. I think the reason for this is a bit deeper and more complicated than simple geographical differences.

An example:

At the Africa Theological Seminary, where I live and usually work, everything was on time. Always. The remarkable thing is that everything is always on time and there is no bell system or clocks on the classroom walls to remind teachers to wrap up their lectures. In fact, from my reckoning, the only room with a clock in it is the chapel.

Somehow, the same people can be very late for church or a meeting in town but can be on time, with almost freakish precision, when at the Africa Theological Seminary. So what is going on?

Clearly time is not necessarily a geographical phenomenon. It has more to do with social norms and institutional traditions.

Think of it this way: It is often believed that the United States is home to people who are always ‘on time’. It would be considered strange, however, for me to show up for a party at a friend’s house at exactly the time it was advertised it to begin. If the party is to start at 6:00, we will undoubtedly show up at 6:30 or later. Again, conceptions of time do not seem to be geographical.

More correctly, conceptions of time seem most fundamentally tied to social norms and the traditions set up by the various institutions in our lives. It may seem that conceptions of time are geographically based, because history plays out in specific geographical locations. Behind every seemingly geographical cultural characteristic I can think of, there lies a more fundamental reason for the perceived difference that actually influences the local culture. It usually points back to the influence of political, economic, social, and religious institutions on our daily lives.

P.S. This lesson, perhaps even more than others, is not completely settled with me. I’m very open to discussing this further.

One Year in Kenya [Lesson 2/5]

When reading anything start with the presumption that this is almost certainly wrong.

Being a research assistant really requires you to read and think for whomever’s research you are assisting. Research requires a certain level of non-emotional attitude. Reading must be done quickly and summaries must be with pith. Starting with this presumption helps, it may not be how you read in leisure, but it will make you a more efficient and a more appreciated research assistant.

There is an important caveat to this lesson: Starting with the presumption that everything is almost certainly wrong does not always lead to a contrarian conclusion, questioning everything no matter what is cliché and bullish. True critical thinking knows what to question, when, and where and what to appreciate and objectively support.

An example of this in the negative form comes from Pikettymania: Thomas Piketty, the now-famous French economist, published his take on wealth inequality in his best-selling book Capital in the 21st Century.

The book, as it should, made a huge splash upon its translation into English. Many (I mean, almost everybody who thinks about this stuff) wrote and published their thoughts on the book (and its famous equation: r>g) all over the economics blogosphere in the first half of 2014.

Chris Giles, an economics editor for the Financial Times, wrote his ‘critique’ on the so-called mathematical and statistical errors of Piketty’s analysis. His conclusion, Piketty is wrong, inequality is not actually rising.

Several weeks later, Piketty responded to Giles’s comments clearly (I mean with head spinning statistics) showing that his so-called mistakes were actually regular data maintenance methods accepted by anyone performing analysis on decades long time-series datasets.

While a careful reading of Piketty’s book starts with the presumption that it is almost certainly wrong, quality criticism should not follow the path Giles took: Starting with a pre-set conclusion and combing for a story to support that conclusion. Due to the weight of the implications of Piketty’s book much rigorous criticism must be deployed, but this must be done objectively and credit must be given where credit is due.

One Year in Kenya [Lesson 1/5]

Over the next couple weeks I will be posting on several of the most salient lessons I’ve learned while living and working in Kenya over the past year. These lessons will be skewed toward what I am currently able to articulate and write about. Certainly many of the most profound lessons will be realized in later months and years.

The more I learn about the world, the more I find myself saying, “I don’t know”.

This lesson is, perhaps, half due to reading a book (Aid at the Edge of Chaos) and half from experiences. booknew21

The book should be an important book for anyone interested in a career focusing on international issues. It’s influence, however, may not be as wide as it deserves. The book, particularly the middle section, is kind of a slog. Ben Ramalingam introduces the rich but young history of complexity science, the study of complex systems such as ecosystems, traffic patterns, deep-sea oilrigs, epidemiology, etc. Ben’s point is that not only are political and economic systems categorically complex, so are the aid and development systems of providing information, aid, assistance, knowledge, and resources. His conclusions are important, I recommend the book, but only if you are seriously pursuing an ID career.

Saying the world is complex is almost cliché now a days. This lesson, however, bypasses the cliché, as a complex world is now a ‘jumping off’ point for me rather than a landing area. One story encapsulates this well:

One of the components of the program I was a part of was providing loans to groups of borrowers. We had $25,000 to loan out during a one-year period. Due to the experimental nature of our project we chose to give out loans on a term of 6 months. This would allow us to make adjustments halfway through our time with the money.

The first round was complicated by the nightmare that is the global money transfer system. Due to several mistakes along the way by accountants entering faulty routing and account numbers, it took about two weeks for the money (when I say money, I mean numbers recorded in a computer) to travel from the United States to our bank account here in Kenya. After we had the money we had to transfer chunks of it to several groups of borrowers who have their own bank accounts. Again, accountants made mistakes entering account numbers and two of the transfers were reversed. One time the money was “lost” in cyberspace with record of one bank initiating the transfer but no record of the other bank receiving it. All this eventually was ironed out, but delayed the lending by another two weeks.

The second transfer has just recently occurred. In the weeks leading up to the loan application deadline, I found myself sitting with several of the leaders of the savings groups. They explained to me why taking a 6 month loan starting in July is a less than optimal option for them.

For those who would use the money to invest in farming, the seeds are already planted and harvest will not begin until November. Any investment in farming at this point is not financial.

Due to this, and because Kitale is primarily agricultural, the majority of the other businesses from a medical laboratory to a women’s clothing boutique are currently in a season characterized by low sales numbers. Not generally an optimal time for investment for individuals who often provide for their family and run their business out of the same pile of money.

Furthermore, there is an added dynamic of group lending. We lend to several groups, which are officially registered and have a set list of individual members. These groups then provide financial services to their members. The groups all have different methods for loaning money, but generally individuals will not take a loan until they are ready to use it. One group leader told me he’s worried that if his group takes a 6 month loan now, the money will simply sit in a bank account for 5 months until the harvest season begins and then the group will be required to repay the loan (with interest) before the majority of their most productive season has completed.

There are so many surface level conversations about loans being inherently ‘good’ or ‘bad’. The fact is, both of these opinions are right and both are wrong. More money does not necessarily mean more money. In what season was the money given or loaned? What are the terms? Is it one time transfers or multiple smaller transfers spread out over a longer period of time? Group lending is often considered less risky than individual lending, but what are the dynamics of the group? How did the group itself originally form? More generally, what are the other options for credit available in the area? How does the ‘market interest rate’ compare? What are the terms and conditions of the ‘competition’?

This is why, the more I learn about access to capital, microfinance, and financial products for the poor, the more I find myself saying, “I don’t know”.

One of my favorite movie quotes points to this idea of ‘the more you know, the less you know’.

“Conviction, as it turns out, is a luxury for those on the sidelines…”
A Beautiful Mind (2001)

I do not consider myself to be Hayekian, nor do I generally sympathize with Austrian Economics, this quote by F.A. Hayek, however, is really quite good:

“The curious task of economics is to demonstrate to men how little they really know about what they imagine they can design.”
– Friedrich von Hayek

Learning About Learning

“Adults are obsolete children,” is an oft quoted saying of Dr. Seuss. One of the major characteristics that differentiates adults and children is adults actively try NOT to make mistakes. I don’t think this is wrong, I just don’t think it is right. Mistakes are how we learn as children and the amount of mistakes children make are precisely why we learn fastest when we are a child.

Continue reading “Learning About Learning”