Thanks very much for writing this — I’m inclined to agree that results from the happiness literature are often surprising and underrated for finding promising neartermist interventions and thinking about the value of economic growth. I also enjoyed hearing this talk in person!
The “aren’t people’s scales adjusting over time?” story (‘scale norming’) is most compelling to me, and I think I’m less sure that we can rule it out. For instance — if I’m reading you right, you suggest that one reason to be skeptical that people are adjusting their scales over time is that people mostly agree on which adjectives like “good” correspond with which numerical scores of wellbeing. This doesn’t strike me as strong evidence that people are not scale norming, since I wouldn’t be surprised if people adjust the rough meaning of adjectives roughly in line with numbers.
If people thought this task was meaningless, they’d answer at random, and the lines would be flat.
I don’t see a dichotomy between “people use the same scales across time and context for both words and adjectives” and “people view this task as meaningless”.
You also suggest a story about what people are doing when they come up SWB scores, which if true leaves little room for scale norming/adjustment. And since (again, if I’m reading you right) this story seems independently plausible, we have an independently plausible reason to be skeptical that scale norming is occurring. Here’s the story:
the way we intuitively use 0 to 10 scales is by taking 10 to be the highest realistic level (i.e. the happiest a person can realistically be) and 0 as the lowest (i.e. the least happy a person could realistically be) (Plant 2020). We do this, I claim, so that [...] we can use the same scales as other people and over time. If we didn’t do this, it would make it very difficult for our answers to be understood.
I think I don’t find this line of argument super compelling, and not even because I strongly disagree with that excerpt. Rather: the excerpt underdetermines what function you use to project from an extremely wide space onto a bounded scale, and there is no obvious such ‘Schelling’ function (i.e. I don’t even know what it would mean for your function to be linear). And indeed people could change functions over time while keeping those 0 and 10 pegs fixed. Another thing that could be going on is that people might be considering how to make their score informationally valuable, which might involve imagining what kind of function would give a relatively even spread across 0–10 when used population-wide. I don’t think this is primarily what is going on, but to the extent that it is, such a consideration would make a person’s scale more relative to the population they understand themselves to be part of[1], and as such to re-adjust over time.
Two extra things: (i) in general I strongly agree that this question (about how people’s SWB scales adjust across time or contexts) is important and understudied, and (ii) having spoken with you and read your stuff I’ve become relatively less confident in scale-norming as a primary explanation of all this stuff.
I would change my mind more fully that scale norming is not occuring if I saw evidence that experience-sampling type measures of affect also did not change over the course of decades as countries become/became wealthier (and earned more leisure time etc). I’d also change my mind if I saw some experiment where people were asked to rate how their lives were going in relation to some shared reference point(s), such as other people’s lives descibed in a good amount of detail, and where people’s ratings of how their lives were going relative to those reference points also didn’t change as countries became significantly wealthier.
(Caveat to all of above that I’m writing in a hurry!)
[Caveat I’m not an expert on this topic, just interested]
I think if people make their scales consistent with others they speak to, and use that same scale across time, for that to not imply similar scales across generations would mean that people don’t really speak to people outside of their generation. Which could be plausible, I’m not really sure. It seems like a relatively heavy lift to ask someone “Hey, could you make sure the best and worst life you reference is the same as your grandparents?”
But really I think what we need is more data! We can speculate til the cows come home.
(i.e. I don’t even know what it would mean for your function to be linear)
That people attach equal value to each unit change in a scale?
And indeed people could change functions over time while keeping those 0 and 10 pegs fixed.
That’d be pretty odd, wouldn’t it? I think it’s likelier that the scale stretches or shifts instead of the reporting function changing.
I would change my mind more fully that scale norming is not occuring if I saw evidence that experience-sampling type measures of affect also did not change over the course of decades as countries become/became wealthier (and earned more leisure time etc).
Why do you think that ESM scales wouldn’t change over the course of time if other scales did? Alas, I’m not sure this data exists. I think the closest thing is looking at time use data. Using time use happiness data Han & Kaiser (2021) seem to suggest that Americans are a tiny tiny bit happier since the 80s?
I’d also change my mind if I saw some experiment where people were asked to rate how their lives were going in relation to some shared reference point(s), such as other people’s lives descibed in a good amount of detail, and where people’s ratings of how their lives were going relative to those reference points also didn’t change as countries became significantly wealthier.
Interesting point. What about if there was evidence that people across ages tended to use subjective wellbeing scales in similar ways?
Thanks very much for writing this — I’m inclined to agree that results from the happiness literature are often surprising and underrated for finding promising neartermist interventions and thinking about the value of economic growth. I also enjoyed hearing this talk in person!
The “aren’t people’s scales adjusting over time?” story (‘scale norming’) is most compelling to me, and I think I’m less sure that we can rule it out. For instance — if I’m reading you right, you suggest that one reason to be skeptical that people are adjusting their scales over time is that people mostly agree on which adjectives like “good” correspond with which numerical scores of wellbeing. This doesn’t strike me as strong evidence that people are not scale norming, since I wouldn’t be surprised if people adjust the rough meaning of adjectives roughly in line with numbers.
I don’t see a dichotomy between “people use the same scales across time and context for both words and adjectives” and “people view this task as meaningless”.
You also suggest a story about what people are doing when they come up SWB scores, which if true leaves little room for scale norming/adjustment. And since (again, if I’m reading you right) this story seems independently plausible, we have an independently plausible reason to be skeptical that scale norming is occurring. Here’s the story:
I think I don’t find this line of argument super compelling, and not even because I strongly disagree with that excerpt. Rather: the excerpt underdetermines what function you use to project from an extremely wide space onto a bounded scale, and there is no obvious such ‘Schelling’ function (i.e. I don’t even know what it would mean for your function to be linear). And indeed people could change functions over time while keeping those 0 and 10 pegs fixed. Another thing that could be going on is that people might be considering how to make their score informationally valuable, which might involve imagining what kind of function would give a relatively even spread across 0–10 when used population-wide. I don’t think this is primarily what is going on, but to the extent that it is, such a consideration would make a person’s scale more relative to the population they understand themselves to be part of[1], and as such to re-adjust over time.
Two extra things: (i) in general I strongly agree that this question (about how people’s SWB scales adjust across time or contexts) is important and understudied, and (ii) having spoken with you and read your stuff I’ve become relatively less confident in scale-norming as a primary explanation of all this stuff.
I would change my mind more fully that scale norming is not occuring if I saw evidence that experience-sampling type measures of affect also did not change over the course of decades as countries become/became wealthier (and earned more leisure time etc). I’d also change my mind if I saw some experiment where people were asked to rate how their lives were going in relation to some shared reference point(s), such as other people’s lives descibed in a good amount of detail, and where people’s ratings of how their lives were going relative to those reference points also didn’t change as countries became significantly wealthier.
(Caveat to all of above that I’m writing in a hurry!)
If almost everyone falls between 6–7 on the widest scale I can imagine, maybe the scale I actually use should significantly zoom in on that region.
[Caveat I’m not an expert on this topic, just interested]
I think if people make their scales consistent with others they speak to, and use that same scale across time, for that to not imply similar scales across generations would mean that people don’t really speak to people outside of their generation. Which could be plausible, I’m not really sure. It seems like a relatively heavy lift to ask someone “Hey, could you make sure the best and worst life you reference is the same as your grandparents?”
But really I think what we need is more data! We can speculate til the cows come home.
That people attach equal value to each unit change in a scale?
That’d be pretty odd, wouldn’t it? I think it’s likelier that the scale stretches or shifts instead of the reporting function changing.
Why do you think that ESM scales wouldn’t change over the course of time if other scales did? Alas, I’m not sure this data exists. I think the closest thing is looking at time use data. Using time use happiness data Han & Kaiser (2021) seem to suggest that Americans are a tiny tiny bit happier since the 80s?
Interesting point. What about if there was evidence that people across ages tended to use subjective wellbeing scales in similar ways?