Interesting! I would have thought you could test this empirically? For example, though it wouldn’t tell you the answer to this question, it would be informative if someone:
Created vignettes about people in various conditions (e.g. with certain diseases) and asked people to rate those people’s “happiness, life satisfaction” etc. Ask people in different countries and check for systematic differences by country. Ask the same people at different time points to see how much variation there is in the answers from time to time.
(Less useful?) Showed people vignettes about people in various conditions or about different events and stories, didn’t ask any questions about those vignettes, but then asked participants (in different countries or at different time points) to rate their own “happiness, life satisfaction” etc. The test here is whether the effect of external stimuli on these ratings is similar in different cultures and across time, or whether the effects vary systematically.
Asked people questions about their own “happiness, life satisfaction” etc, then asked them to qualitatively describe what 1 and 10 would look like. Do some sort of content analysis of the answers.
(I’ve only read your summary and I’m not familiar with the literature, so apologies if people already talk about this sort of thing or have run these sorts of studies.)
Hello Jamie. Thanks for your astute comment! The paper is quite long and I do cover all of this apart from your third bullet point.
We can’t objectively measure subjective states and this seems to have led some people to think that you can’t use any empirical evidence at all. But you’re right that if you make some assumptions e.g. about vignettes, then if the data go one way that raises your confidence in there being/not being cardinality. This approach is just the basic “inference to the best explanation” used across the sciences (one might even say it’s the fundamental method of science).
I discuss vignettes specifically in section 5.5. What you suggest has been done. Angelini et al (2014) asked people their own life sat, then show them this (and another) vignette
John is 63 years old. His wife died 2 years ago and he still spends a lot of time thinking about her. He has four children and ten grandchildren who visit him regularly. John can make ends meet but has no money for extras such as expensive gifts for his grandchildren. He has had to stop working recently due to heart problems. He gets tired easily. Otherwise, he has no serious health conditions. How satisfied with his life do you think John is?
And then asked people to rate how satisfied John is. The is we can assume ‘vignette equivalence’ - everyone will agree how satisfied John is—use that to make inferences about differential scale use and therefore adjust each individual’s scores. The issue, as I say (p24) is that:
However, respondents do not seem to agree [how satisfied John is]. For instance, Angelini et al. (2014) find about 30% of Germans rate ‘John’ from the above vignette as satisfied or very satisfied, but 30% rate him dissatisfied or very dissatisfied. To assume that the respondents agree about John’s life satisfaction requires us to conclude that respondents must mean the same thing by “satisfied” as “dissatisfied”, which strains credulity seeing as one is positive and the other negative. Faced with a choice of vignette equivalence or semantic equivalence (that respondents attach the same meaning to words) the latter seems more plausible
The general point people we need to think carefully about which assumptions we take as ‘ground truths’ to test to cardinality. Vignette equivalence is, I think, not rock solid.
Re your third bullet point, I think it would be really hard to do it that way around—I can’t see any way to use that to a numerical interpretation from the answers, which is what’s needed.
Interesting! I would have thought you could test this empirically? For example, though it wouldn’t tell you the answer to this question, it would be informative if someone:
Created vignettes about people in various conditions (e.g. with certain diseases) and asked people to rate those people’s “happiness, life satisfaction” etc. Ask people in different countries and check for systematic differences by country. Ask the same people at different time points to see how much variation there is in the answers from time to time.
(Less useful?) Showed people vignettes about people in various conditions or about different events and stories, didn’t ask any questions about those vignettes, but then asked participants (in different countries or at different time points) to rate their own “happiness, life satisfaction” etc. The test here is whether the effect of external stimuli on these ratings is similar in different cultures and across time, or whether the effects vary systematically.
Asked people questions about their own “happiness, life satisfaction” etc, then asked them to qualitatively describe what 1 and 10 would look like. Do some sort of content analysis of the answers.
(I’ve only read your summary and I’m not familiar with the literature, so apologies if people already talk about this sort of thing or have run these sorts of studies.)
Hello Jamie. Thanks for your astute comment! The paper is quite long and I do cover all of this apart from your third bullet point.
We can’t objectively measure subjective states and this seems to have led some people to think that you can’t use any empirical evidence at all. But you’re right that if you make some assumptions e.g. about vignettes, then if the data go one way that raises your confidence in there being/not being cardinality. This approach is just the basic “inference to the best explanation” used across the sciences (one might even say it’s the fundamental method of science).
I discuss vignettes specifically in section 5.5. What you suggest has been done. Angelini et al (2014) asked people their own life sat, then show them this (and another) vignette
And then asked people to rate how satisfied John is. The is we can assume ‘vignette equivalence’ - everyone will agree how satisfied John is—use that to make inferences about differential scale use and therefore adjust each individual’s scores. The issue, as I say (p24) is that:
The general point people we need to think carefully about which assumptions we take as ‘ground truths’ to test to cardinality. Vignette equivalence is, I think, not rock solid.
Re your third bullet point, I think it would be really hard to do it that way around—I can’t see any way to use that to a numerical interpretation from the answers, which is what’s needed.