Imagine that someone has an experience that causes them to re-conceptualize the scale they’ve been using to measure their well-being.
e.g. They used to assess themselves as an 8 out of 10 on some psychometric scale. Then they have this experience, and they think “Oh wow, now it seems that the scale I was using was wrong. I had been thinking that I was close to as good as I could be (8 out of 10), but now it feels like ‘as good as I can be’ is actually way higher than I thought...”
So they bump up the top end of the scale they’re using. Now they assess themselves as 8 out of 50, not because they think their life has gotten worse, but because they’re conceptualizing “as good as I can be” differently.
But this shift isn’t reflected on the psychometric instrument they’ve been using, so when they assess themselves again, they have to normalize their new view to match the instrument’s scale:
“I used to feel like I was 8 out of 10, and that matched up pretty well with this 10-point scale, so I recorded myself as 8 out of 10”
“Then I had an experience that caused me to believe that the potential upside can be way, way higher, so now I feel like I’m 8 out of 50”
“I have to fill out another one of these 10-point scales – should I still say I’m 8 out of 10? Or should I say I’m 2 out of 10, because that’s proportional to my new internal assessment? But recording 2 out of 10 makes it seem like my life has gotten way worse since I last filled out an assessment, and that’s not true...”
This issue (how to translate a person’s internal state into a standard measure, then track that measure over time) seems pretty central to assessing any intervention aimed at improving subjective well-being, given that we’re in a world where every person starts by bootstrapping their own internal scale / their own sense of what’s possible.
Has anyone seen discussion of how to handle shifts like this in the SWB measurement literature?
TL;DR Evidence suggests there aren’t shifts in SWB scales over time. This topic isn’t well understood. I’ve got a paper on this area in the works.
The question you’re asking here—do individuals rescale, that is, alter what the end-points of their scales refer to? - is one component of a broader concern.
The broader question is whether subjective scales, those where individuals give numerical ratings of subjective phenomena are cardinally comparable, that is, whether a one-point change, on a given scale, represents the same size change in subjective experience for different people and at different times. For instance, if I say my happiness has gone from a 4 to an 5 out of 10, and you say your happiness have gone from a 3 to 4, can we conclude we each had the same increase in happiness?
Given how fundamental the concern is—it applies to all subjective data, not just SWB data—I’ve been surprised to find the topic hasn’t been looked into a great deal. Two leading SWB researchers, Stone and Krueger, said this in an 2018 review article
I’ve been looking at the cardinality of subjective scales. I’ve got a working paper that I’m not quite ready to put online—this should only be another couple of months. The paper is an evolution of work I had in my DPhil thesis (pp. 135), where I broke cardinal comparability into a number of components, reviewed the evidence for each, and concluded SWB data probably best interpreted as cardinally comparable.
The topic is pretty complicated and addressing all of it would take too long here. I’ll just provide a ‘quick and dirty’ answer to the specific concern you raise about rescaling (aka ‘intertemporal cardinality’). Prati and Senik (2020) compare remembered SWB—how satisfied individuals recall being—with observed past SWB—how satisfied individuals they said they were at the time. The use a German panel data where individuals were given 9 different pictures of changes in life satisfaction over time (e.g. staying flat, going up, going up then going down, etc) and asked to pick the one that best represented their own life.
There turns out be an (I think) pretty amazing match between the patterns of observed past and remembered SWB. This is only possible if either (A) individuals both use the same scale over time and have good memories or (B) individuals change the scale use and have bad memories. If individuals used the same scales and had bad memories, or used different scales and had good memories, there would be an inconsistency between the recalled and past observed patterns. Of the two options, (A) seems far more probable than (B). It’s hard to believe individuals really can’t remember how their lives have gone. Further, we might expect individuals will try not to rescale so that their answers are comparable over time.*
Hence, there doesn’t seem to be rescaling at the population level. Further research into whether there are some individuals who rescale, and what causes this to happen, would be good. I’m not aware of any.
*In fact, (B) requires quite specific and implausible patterns of memory failure. To illustrate, suppose your experienced satisfaction has been flat but, because your scale has been shrinking, your reported 0-10 level of satisfaction had been rising over time. To make your observed past satisfaction and your recalled satisfaction consistent, given this scale shrinkage, you would need to falsely recall that your satisfaction has increased. If you instead erroneously recalled that your satisfaction had decreased, then there would be an inconsistency between observation and recall.
Thanks for this!
How do you think the potential consistency over time (A) squares with the inconsistency between scales & sub-scales that Kaj pointed out?
It’s not clear to me what relationship one should expect between the cardinality (or not) of subjective scales and the relationship between ratings of overall SWB and rating of sub-domains (and thus what one could infer about the other from results in the first).
As a separate point, I’m not sure how to make sense of the putative inconsistency Kaj’s notes. I haven’t looked into the relationships between overall rating and rating of sub-domains; it’s not something that I’ve heard SWB researchers discuss much either. The most obvious explanations, in addition to those mentioned below, are to appeal to missing domains and/or different temporal foci (i.e. you just think about sub domains are they are now, but your life you also think about the future.