The Cardinal Treatment of Ordinal Variables

Last weekend, I came across a paper published in the European Economic Review by Carsten Schroder and Shlomo Yitzhaki entitled “Revising the Evidence for Cardinal Treatment of Ordinal Variables” (2017). I found the paper to be well-written, intuitive, and important. Although there are no direct policy implications of this paper, papers like these need to be publicized, for the sake of all research that does aim to provide meaningful implications for the world.

In this paper Schroder and Yitzhaki discuss the use of well-being variables in econometric analysis. This is an area of analysis that is quickly becoming “classic” in economics. The authors point out that over 3700 published articles are listed on the RePEc IDEAS website containing synonyms of “happiness” in the abstract. Despite recent popularity, well-being is notoriously difficult to measure. In fact subjective well-being (or satisfaction measures, happiness, or… erm… hope) are latent variables that are impossible to directly observe and have no natural measurement unit.

Nevertheless many social scientists (and an increasing number of economists) are collecting subjective well-being data using ordinal scales. (i.e. scales ranging from 0 through 10 or categories ranging from “very satisfied” to “not very satisfied”.) Econometric techniques explicitly exist for the use of ordinal left-hand-side variables (i.e. ordered logit/probit or nonparametric regressions), but these methods are often less preferred to more standard and straightforward approaches (i.e. comparison of means and OLS). The problem is, comparisons of means and OLS, are actually designed with use of cardinal variables in mind.

Schroder and Yitzhaki make the point that the assumption of treating ordinal variables as if they are cardinal may lead to biased estimates. They show this beautifully in their paper through the following theoretical example. Say we have two groups of people and we want to see which group is “happier”. In the first group two people report being “unhappy”, three report being “neither happy, nor unhappy”, and five report being “happy”. In the second group one person is “unhappy”, five are “neither happy, nor unhappy”, and four are “happy”. So, which group is happier?

It all depends on the implied concept of distance between these three categories (displayed above). Say the distance is equal between each category. “Unhappy” is coded as 1, “neither happy, nor unhappy” as 2, and “happy” as 3. In this case the weighted average between the groups is equal; they are both equally happy. But what if the distance concept is concave; say: 1, 3, 4, respectively? In this case group two is happier. Conversely, what if the distance concept is convex; say: 1, 3, 7, respectively? In this case group one is happier. As you can see the implied distance between the categories or scores on an ordinal scale matter, a lot.

The problem is, with ordinal variables we don’t know what the distance concept is between the categories. It may be equal, convex, concave, or some combination. Worse, different individuals may hold different distance concepts. Yikes! The point about interpersonal comparability can be addressed with somewhat intensive survey methods such as anchoring vignettes (see Gary King’s website) or perhaps qualitative validation (see Blattman et al. 2015). The main issue for Schroder and Yitzhaki, however, remains unaffected and potentially troublesome.

The paper develops two conditions under which the cardinal treatment of ordinal variables, and the valid use of comparisons of means or OLS, is permitted. Essentially, in order to calculate unbiased comparisons of means or OLS estimates these methods must produce results that are robust to any monotonic increasing transformation of the ordinal variable. This is a descriptive, but unhelpful axiom because there are seemingly infinite increasing monotonic transformations one could perform on their data.

The authors test their conditions on an existing dataset (The German Socio-Economic Panel) and find that 47 out of 48 comparisons in means tests fail their robustness check and in one OLS specification the signs on coefficients flipped in the opposite direction while retaining statistical significance when the ordinal variables are transformed.

I’ll leave the derivation and discussion of the conditions to the authors as I cannot do them justice here and now. Bottom line, anyone who uses ordinal variables in their research should read this paper. It has important insights that are often being ignored.