Reactions to the “cardinal comparability” objection (4.2):
Unlike validity, this is not a well-studied topic. When I came to look at it for my PhD, I struggled to find much of a literature on it. There were bits and pieces, but nothing that seemed to convincingly offer an overall assessment of the issue (see Plant 2020 where I try to offer one).
Understudied topics are where non-expert input is more likely to be useful. However, in this case, we do have a literature on the topic. The term is “scope insensitivity.” It’s one of the key cognitive biases Kahneman and others have examined in the behavioral economics literature. People will say they’d pay $80, $78, and $88 to protect 2,000, 20,000, or 200,000 birds from drowning in oil ponds. Just so, they’ll report an 8⁄10 score for their marriage just about no matter how happy it makes them.
Although this is a weedy topic, it might be the key area where the intuitions of growth advocates diverge. Scale shifts mean anchoring on a default value for self-reports. There are cultural defaults for how to describe or quantify your relationship with your spouse, kids, parents, hometown, job, and sports team. These defaults are independent of their true quality. When people’s circumstances worsen or improve, the normal way to describe them stays the same.
People’s culture and identity informs their default response. If they see themselves as a grateful person, that means they have to give grateful-sounding responses, regardless of circumstances. This is the explicit, fundamental teaching of many philosophies and religions. They don’t just teach that you should learn to feel happy in challenging circumstances. They teach that you should start by describing those circumstances with equanimity. They, along with psychologists, also talk about the difficulty of noticing and honestly reporting your true feelings. This challenge is the basis for the perennial human potential movement.
Default responses also help people preserve their mental wellbeing. Too much complaining makes people feel worse. It would just feel sarcastic to claim life’s 10⁄10 when it’s not—and realistically can’t be, since nothing’s ever perfect. So we default to a description somewhere from 4/10-8/10 in most cases. When things are worse, we search for a way to see them as better than they are. When they’re better, we find a way to commisserate.
In broad terms, I think we should be somewhat reassured about the cardinal comparability of subjective data. As one piece of evidence, see Figure 9 below, taken from a YouGov poll. Here, American individuals were asked to give ratings from 0 (very negative) to 10 (very positive) for different words like: “very bad”, “terrible”, “outstanding”, “excellent”, and “perfect.”
The ‘bumps’ represent the proportion of people that give each answer. The overlap isn’t perfect, but it’s pretty good. If you ask people to score ‘perfect’, basically everyone says it means 10⁄10. If people thought this task was meaningless, they’d answer at random, and the lines would be flat. So, it seems that people are sensibly able to compare verbal labels and numerical labels and do so in the same sort of way.
This supports the hypothesis that people can put their language on a scale. It doesn’t mean they are. There are no cultural or emotional stakes in this study. At most, it leaves the possibility open that people are able to report their real subjective wellbeing. It doesn’t surprise me that the cultural default in some countries is a 2-3, while in others it’s a 7-8, which is how I interpret figure 3. That data is perfectly compatible with both the idea that people are scaling their happiness according to, say, relative wealth, and that these countries simply have stable default reporting values, which cluster geographically as so many other cultural traits do.
People aren’t using “bespoke scales which change from moment to moment.” They’re using broken scales that stay fixed across time, regardless of their true feelings or changing circumstances.
As you point out:
There’s some evidence from memory data that individuals keep the same scales over their own lives (this is from Prati and Senik 2020, which I discuss in Plant 2020).
Scale shift could still happen generationally: an 8⁄10 to someone born in 1950 represents a lower level of whatever subjective thing is being measured than an 8⁄10 to someone born in 2000. I can’t think of any research that addresses this specific concern. It doesn’t strike me as particularly likely, though. It implies that if me, my parents, and my grandparents were each to say we were 10⁄10 happy, we would assume I would be happier than my parents, who would be happier than their grandparents.
I would absolutely believe that your 10⁄10 represents a higher level of felt happiness or satisfaction than your grandparents experienced. This corresponds perfectly to the idea of an intergenerational fixed default report.
An alternative to arguing about possible scale changes would be to take a general theory of how happiness works, how economic growth changes our lives and society, and whether we should expect it to increase or reduce happiness as a result. For my money, the most promising option is to conceive of happiness and unhappiness as “Mother Nature’s” reward and punishment mechanisms for evolutionary fitness. In this light, we want to consider humanity’s environment of evolutionary adaptation, i.e. from about 100,000 years ago, to the present day but it’s not obvious this analysis favours the growth-advocate. Notably, Hidaka (2012) argues that depression is rising as a result of modernity, and points to the fact that “modern populations are increasingly overfed, malnourished, sedentary, sunlight-deficient, sleep-deprived, and socially-isolated”.[5]
Evolutionary fitness is the number of offspring you produce, and how many they produce, and so on ad infinitum. This predicts that people will report greater happiness the more children they have. In fact two children, which is below replacement, is the preferred number across many countries.
Overfed? Sounds evolutionarily adaptive to me. Malnourished? Your breakfast cereal is fortified, nobody gets scurvy, and vitamin shops are everywhere. Sedentary? Relaxed. Sunlight-deficient? It’s brighter in my house than it is outside at the moment. Sleep-deprived? Watching Netflix while not sick with cholera. Socially isolated? People have options about whether and whom to be friends with.
Weirdly, I feel somewhat anxious in saying this, as though I’ve broken a mild taboo by responding in these ways to Hidaka’s list. That’s what I get for violating the cultural default report.
Reactions to the “cardinal comparability” objection (4.2):
Understudied topics are where non-expert input is more likely to be useful. However, in this case, we do have a literature on the topic. The term is “scope insensitivity.” It’s one of the key cognitive biases Kahneman and others have examined in the behavioral economics literature. People will say they’d pay $80, $78, and $88 to protect 2,000, 20,000, or 200,000 birds from drowning in oil ponds. Just so, they’ll report an 8⁄10 score for their marriage just about no matter how happy it makes them.
Although this is a weedy topic, it might be the key area where the intuitions of growth advocates diverge. Scale shifts mean anchoring on a default value for self-reports. There are cultural defaults for how to describe or quantify your relationship with your spouse, kids, parents, hometown, job, and sports team. These defaults are independent of their true quality. When people’s circumstances worsen or improve, the normal way to describe them stays the same.
People’s culture and identity informs their default response. If they see themselves as a grateful person, that means they have to give grateful-sounding responses, regardless of circumstances. This is the explicit, fundamental teaching of many philosophies and religions. They don’t just teach that you should learn to feel happy in challenging circumstances. They teach that you should start by describing those circumstances with equanimity. They, along with psychologists, also talk about the difficulty of noticing and honestly reporting your true feelings. This challenge is the basis for the perennial human potential movement.
Default responses also help people preserve their mental wellbeing. Too much complaining makes people feel worse. It would just feel sarcastic to claim life’s 10⁄10 when it’s not—and realistically can’t be, since nothing’s ever perfect. So we default to a description somewhere from 4/10-8/10 in most cases. When things are worse, we search for a way to see them as better than they are. When they’re better, we find a way to commisserate.
This supports the hypothesis that people can put their language on a scale. It doesn’t mean they are. There are no cultural or emotional stakes in this study. At most, it leaves the possibility open that people are able to report their real subjective wellbeing. It doesn’t surprise me that the cultural default in some countries is a 2-3, while in others it’s a 7-8, which is how I interpret figure 3. That data is perfectly compatible with both the idea that people are scaling their happiness according to, say, relative wealth, and that these countries simply have stable default reporting values, which cluster geographically as so many other cultural traits do.
People aren’t using “bespoke scales which change from moment to moment.” They’re using broken scales that stay fixed across time, regardless of their true feelings or changing circumstances.
As you point out:
I would absolutely believe that your 10⁄10 represents a higher level of felt happiness or satisfaction than your grandparents experienced. This corresponds perfectly to the idea of an intergenerational fixed default report.
Evolutionary fitness is the number of offspring you produce, and how many they produce, and so on ad infinitum. This predicts that people will report greater happiness the more children they have. In fact two children, which is below replacement, is the preferred number across many countries.
Overfed? Sounds evolutionarily adaptive to me. Malnourished? Your breakfast cereal is fortified, nobody gets scurvy, and vitamin shops are everywhere. Sedentary? Relaxed. Sunlight-deficient? It’s brighter in my house than it is outside at the moment. Sleep-deprived? Watching Netflix while not sick with cholera. Socially isolated? People have options about whether and whom to be friends with.
Weirdly, I feel somewhat anxious in saying this, as though I’ve broken a mild taboo by responding in these ways to Hidaka’s list. That’s what I get for violating the cultural default report.