Why I’m suss on wellbeing surveys
TL;DR: hunter-gatherers would probably rate themselves at a similar level to us on subjective wellbeing surveys, therefore they’re silly.
Big claims from wellbeing surveys
The Happier Lives Institute (HLI) has recently made big claims based on WELLBYs (wellbeing-adjusted life years) interpreted from wellbeing/life-satisfaction surveys.
Their report on StrongMinds (a charity which offers therapy to people in developing countries to improve their mental health) estimates StrongMinds as being between 0.75× to 12×as effective as The Against Malaria Foundation. Great news if true!
Their analyses suggest some surprising findings, such as that the grief from the death of a child equates to a loss of 7.26 WELLBYs [Appendix 2 of The Elephant In The Bednet] while an individual treated in the StrongMinds program can expect a gain of approx. 3 to 20 WELLBYs!!! [Figure 2, Don’t just give well, give WELLBYs: HLI’s 2022 charity recommendation]
HLI have just produced a study supporting the use of wellbeing surveys in charity evaluation: Can we trust wellbeing surveys? A pilot study of comparability, linearity, and neutrality.
Something doesn’t sit right with me about the use of subjective wellbeing surveys for serious charity analysis. I will try to articulate it with this reductio ad absurdum argument:
Are you happier than the Sentinelese?
Consider the Sentinelese people of the North Sentinel islands, who are one of the few remaining uncontacted (mostly) peoples in the world, and who probably live with a similar level of technology and culture as the average human did 20,000 years ago.
I ask myself: If asked to rate their life satisfaction in a survey, would the Sentinelese people rate their life satisfaction as significantly lower than mine?
If NO (my personal suspicion):
Using WELLBYs, the implication is that technological and cultural development over the last 20,000 years has largely been a mistake for wellbeing and that a spontaneous return to pre-agricultural society would be better for global wellbeing. This is an established worldview, but one I think should be discarded because of its drastic implications.
If YES:
Is the difference in WELLBYs significant enough to justify the hundreds of trillions of dollars and hours of effort and suffering (and negative WELLBYs) that have gone (and continue to go) into technological, economic and cultural development to give us our modern lives?
This seems unlikely to me given the magnitude of resources and negative WELLBYs that have been spent on development efforts
My conclusion is that wellbeing surveys shouldn’t be used to compare wellbeing on a global scale, because doing so leads to absurd conclusions. Instead, objective measures of health, wealth, education, and access to amenities should be used at this scale, with wellbeing surveys possibly useful when comparing similar groups.
Hello Henry,
Thank you for presenting this thought experiment.
The core here is about whether groups like the Sentinelese who do not have the same levels of development as others would give similar levels of SWB. I think the other comments here have done a great job at pointing out possible explanations.
if Sentinelese have ⇒ wellbeing
maybe their lifestyle works really well for their wellbeing (as Charlie mentions, we might not want to be too quick to dismiss this possibility). It would be a cool area to research.
maybe there are issues of interpersonal comparability in scale use, which is what we are exploring with our pilot.
maybe they have access to factors (e.g., social ties) that improve wellbeing without being affected as much by reference frames (e.g., the benefit I get from a higher income is relative to other people) or hedonic adaptation (as pointed out by Alex).
if Sentinelese have < wellbeing
As Nick pointed out, countries with lower levels of development tend to have lower levels of LS. [edit: adding this figure here because it represents this well and supports the validity of SWB scales]
It might be well worth the investment (especially as Vasco pointed out, it can increase the number of humans and life expectancy)
It is possible that whilst our development reached a point where we have higher wellbeing, it did so through inefficient periods and our task now is to think more carefully about how to do (and whether there are adverse consequences like climate change, x-risks, etc.)
Some briefs answers / pointers. Many of these things have been discussed in more details elsewhere.
The estimate for grief is shallow [edit: I want to make this point a bit stronger—this was a quick estimate and it is a tad unfair to compare it to the SM estimate which represents hundreds of hours of work and meta-analyses] but important in making the difficult work of live-saving vs life-improving accurate. You can see some discussion about it from my colleague. I, personally, wouldn’t be too surprised if we found a higher estimate in the future but there is some reasoning as to why this might not be as big.
I don’t think there are ‘objective’ measures of health widely used. DALYs and QALYs are not objective, they rely on humans reports—and contrary to SWB, these are reports about things humans are know to struggle to report on. In DALYs it is people (who do not have the conditions!) making binary judgements about which conditions are more or less healthy. In QALYs it generally involves people forecasting how their life will be in the future. See To WELLBY or not to WELLBY where we discuss the strengths and weaknesses of SWB.
health, wealth, education, are all instrumental. Why do we want these? Because they contribute to what we believe is ultimately good for humans. Most charity evaluators give there answer in some form of wellbeing / good, either by measuring SWB as directly as possible (HLI) or converting lives and income into the moral weights of the evaluators (GW).
There is extensive debate about the relationship between SWB and growth through the lens of the Easterlin Paradox
I’m confused why you and everyone else in this thread are so quick to dismiss the idea that hunter gathers have more happiness/ life satisfaction/well being.
This is not at all obvious to me.
No it’s not obvious, but the implications are absurd enough (agricultural revolution was a mistake, cities were a mistake) that I think it’s reasonable to discard the idea
Depending on what you mean by mistake I don’t think those implications are absurd at all.
The agricultural revolution wasn’t a decision humanity made, it’s game theory. More resources, more babies, and your ideas survive.
I’m not even saying that modernization was a mistake, which btw we could be less happy and i would still not necessarily say it was a mistake (again, depending on what you mean by mistake). It’s just that I think you are anthropromorphizing cultural natural selection as a well thought out decision with the intention of maximizing current utility.
I think that it may be helpful to unpack the nature of perceived happiness and wellbeing a little bit more than this post does. I think the idea of hedonic adaptation is pretty well known—most of us have probably heard of the hedonic treadmill (see Brickmann & Campbell, 1971). The work on hedonic adaptation points to the fact that perceived happiness and wellbeing are relative constructs that largely depend on reference points which are invoked. To oversimplify things a little bit, if everyone around me is bad off, I may already be happy if I am only slightly better of than them. At the same time, I might be unhappy if I am pretty good off but everyone around me is much better off. As such, it is entirely reasonable to expect that hunter-gatherers when asked about their life feel quite good and happy about it as long as they don’t feel like everyone else around them is much better off.
The conclusion of this post should not be that perceived happiness and wellbeing should not be used to compare the effects of interventions but that they simply measure something different than “objective measures”. They aim to measure how people feel about their life in general as they compare it to others, not how they score on a particular metric in isolation. Whether you prefer one or the other approach largely depends on your perspective on what is valuable in life. Some people may find making progress on metrics that they find particularly valuable is the way to go and others prefer a more self-organizing perspective where the affected people themselves are more involved in determining what is valuable.
In sum, this post seems a little bit confused on what the WELLBY debate is about. I can recommend the cited article to get some idea on why something like a WELLBY approach may be interesting to consider even if one doesn’t like it at first glance.
Brickman, P., & Campbell, D. (1971). Hedonic relativism and planning the good society. In M. H. Appley (Ed.), Adaptation-level theory: A symposium (pp. 287–305). Academic Press. https://archive.org/details/adaptationlevelt0000unse_x7d9/page/287/mode/2up
Just a short follow up: I just wrote a post on the hedonic treadmill and suggest that it is an interesting concept to reflect about in relation to life in general:
https://forum.effectivealtruism.org/posts/WMaeBDPdSJLKDzk2d/the-hedonic-treadmill-dilemma-reflecting-on-the-stories-of-1
The way you describe WELLBYs—as being heavily influenced by the hedonic treadmill and so potentially unable to distinguish between the wellbeing of the Sentinelese and the modern Londoner—seems to highlight their problems. There’s a good chance a WELLBY analysis would have argued against the agricultural revolution, which doesn’t seem like a useful opinion.
To me it seems like you have a wrong premise. A wellbeing focused perspective is explicitly highlighting the fact that Sentinelese and the modern Londoners may have similar levels of wellbeing. That’s the point! This perspective aims to get you thinking about what is really valuable in life and what the grounds for your own beliefs about what is important are.
You seem to have a very strong opinion that something like technological progress is intrinsically valuable. Living in a more technically advanced society is “inherently better” and, thus, everyone who does not see this is “objectively wrong”. That argument would seem strange to even the most orthodox utilitarian. Even if your argument is a little bit more nuanced in the sense that you are seeing technological progress only as instrumentally valuable to sustain larger population sizes at similar levels of wellbeing, this perspective is still somewhat naive because technological progress also has potentially devastating consequences such as climate change or AI risks. In that sense, one can actually make the case that the agricultural revolution was maybe the beginning of the end of the human race. So maybe if there would have been a way to grow our societies more deliberately and to optimize for wellbeing (rather than economic growth) from the beginning, it wouldn’t have been such a bad idea? I just want to illustrate that the whole situation is not as clear cut as you make it out to be.
Altogether, I would encourage you to keep more of an open mind regarding other perspectives. The post but also this comment of yours make it seem like you might be very quick in dismissing perspectives and being vocal about it even if you have not really engaged with them deeply. This makes you come across as naive to a somewhat more knowledgeable person which could put you at a personal disadvantage in the future and, in addition, could also be contributing to bad epistemics in your community if the people you are talking to are less informed and, thus, not able to spot where you might be cutting corners. Hope you don’t resent me for this personal side note, it’s meant in a constructive spirit.
Well said. I share @Henry Howard ’s reservations about WELLBYs, but I would argue that even if WELLBY comparisons are near-meaningless between New Yorkers and Sentinelese, they are probably much more meaningful when comparing one individual’s wellbeing before and after treatment, or even comparing control and intervention groups drawn randomly from the same population.
First I don’t agree with your assumption that hunter gatherers might are likely their wellbeing the same as ours now. The best proxy we might have for “hunter gatherers” today is poorer, less developed countries. People in those countries have on average have lower average wellbeing than richer countries. My assumption would be in the other direction, that hunter gatherers would most likely rate their wellbeing lower than we would today.
I don’t really understand your argument in this paragraph “Is the difference in WELLBYs significant enough to justify the hundreds of trillions of dollars and hours of effort and suffering (and negative WELLBYs) that have gone (and continue to go) into technological, economic and cultural development to give us our modern lives?
The answer surely is a resounding yes! If the hunter gatherers rated their wellbeing lower than us and our wellbeing has improved, then surely all that effort into “technological economic and cultural development” is completely worth it!
Hi Henry,
A few notes:
Even if hunter-gatherers globally were significantly less happy than current humans, we should expect some hunter-gatherer groups to be happier than the current mean human. The larger the number of hunter-gatherer groups we study, the higher the chance of getting unusually happy ones. I do not know whether the one you mention is an outlier or not.
I think wellbeing surveys which look into affect are more reliable than ones assessing life satisfaction. In essence, because I am much more confindent on the ability of humans to assess their present state than the mean state over the past year (which is susceptible to lots of biases, like the peak-end rule).
Even if wellbeing surveys are not great, increasing their adoption could be instrumentally valuable to increase the focus on wellbeing (i.e. conscious states, including non-hedonistic sounding ones like those related to relationships, freedom, and artistic expression), which is what ultimately matters.
I agree optimising for nearterm wellbeing may hinder logterm wellbeing. However, I believe measured related to existential risk are better proxies for longterm progress than health, wealth, and education. For example, for the risk of:
Nuclear war, number of nuclear warheads.
Engineered pandemics, cost of sequencing a full human genome, and vaccine hesitancy.
Advanced AI, AI timelines.
Assuming constant wellbeing per human per year, technogical progress could still be useful to increase the number of humans, and therefore increase the total wellbeing of all humans.
I think the missing step here is that revealed preference suggest that most Sentinelese people would change their lives to become like us, but we wouldn’t change to become like them. So the model doesn’t predict what people actually want.
To caveat this:
If we have higher wellbeing, and they are trying to maximise wellbeing, then all these things align.
But people might not be maximising wellbeing or might not be good at it.
I agree that revealed preference and survey responses can differ. Unless WELLBYs take account of revealed preferences they’ll fail to predict what people actually want
Are you claiming that this would be a neccasary thing to prove or this is true but op didn’t include it. I would certainly consider becoming a hunter gatherer if I didn’t have an established life and friends and family