Concepts such as subjective well-being, satisfaction, happiness, trust, measures of quality, and even standardized test scores are all measured using an ordinal variable. This means that we know the rank of the response categories (e.g. a respondent reporting being “very satisfied” indicates they are more satisfied than if they had reported being “satisfied”), but we do not know the interval between response categories (e.g. we don’t know how much more satisfied “very satisfied” is compared to “satisfied”). This is contrasted with cardinal variables, such as earnings, where we know $10 is more than $5 and represents twice as much money.
Last weekend, I came across a paper published in the European Economic Review by Carsten Schroder and Shlomo Yitzhaki entitled “Revising the Evidence for Cardinal Treatment of Ordinal Variables” (2017). I found the paper to be well-written, intuitive, and important. Although there are no direct policy implications of this paper, papers like these need to be publicized, for the sake of all research that does aim to provide meaningful implications for the world.
That’s an okay explanation, but I think there is an even more profound lesson about economics woven into this classic movie. It’s a topic I interact with regularly and one that is often misunderstood.
In the movie, “It’s a Wonderful Life” George Bailey gets the outrageous opportunity to see what the world would be like if he had never been born. He’s able to see the impact he’s made on the people around him. What a gift!
Notice how I use the word “impact” in the last paragraph. It is used correctly under the definition that impact is the outcomes of something minus the outcomes if that something did not happen.
Economists try to measure the impact of all sorts of things: from government policies and social programs to business innovations and market forces. When undertaking this endeavor we are presented with the same problem Clarence, George’s guardian angel, faces. We need to show impact. In Clarence’s case he needed to prove to George that he had a meaningful and positive impact on the people around him and that the world was a better place because he was born. Economists generally need to measure the effectiveness (or non-effectiveness) of a specific policy, program, or innovation.
The problem economists have is we aren’t angels and can’t go back in time and re-do the world in the absence of the thing we want to evaluate. Economists encounter what is called the “problem of the counterfactual”. We don’t have an alternate universe in which to compare hypothetical outcomes.
We have to come up with work-arounds – creating proxies of the counterfactual – to solve this problem.
One way is to randomly assign receipt from the program, policy, or innovation. With a large enough sample we would expect that the group of people assigned to receive the program, policy, or innovation would be exactly equal to the group of people who did not receive the program, policy, or innovation. Impact would then be measured by subtracting the outcomes from the group that didn’t receive the program, policy, or innovation from the group that had. This is the randomized experimental method and if it is done correctly it allows economists to show the exact same thing that Clarence showed George. Pause and reflect on that for a moment.
Sometimes (most of the time) a randomized experiment is not feasible or ethical, in which case the experimental ideal (described above) needs to be approximated. There are many (in fact, seemingly infinite) ways to do this, each has it’s own strength and is best suited for a particular situation. Going into the specifics of these is beyond the scope of this blog. (Much more can be learned by reading the excellent Mastering Metrics: The Path From Cause to Effect)
Suffice to say, “It’s a Wonderful Life” has a lot to teach us about economics, particularly the problem with the counterfactual and the power of the randomized experiment.