I’m the Director of the Happier Lives Institute and Postdoctoral Research Fellow at Oxford’s Wellbeing Research Centre. I’m a philosopher by background and did my DPhil at Oxford.
MichaelPlant
Karma: 6,666
Both comments by this author seemed in bad faith and I’m not going to engage with them.
Hello William, thanks for this. I’ve been scratching my head about how best to respond to the concerns you raise.
First, your TL;DR is that this post doesn’t address your concerns about the WELLBY. That’s understandable, not least because that was never the purpose of this post. Here, we aimed to set out our charity recommendations and give a non-technical overview of our work, not get into methodological and technical issues. If you want to know more about the WELLBY approach, I would send you to this recent post instead, where we talk about the method overall, including concerns about neutrality, linearity, and comparability.
Second, on scientific validity, it means that your measure successfully captures what you set out to measure. See e.g. Alexandrova and Haybron (2022) on the concept of validity and its application to wellbeing measures. I’m not going to give you chapter and verse on this.
Regarding linearity and comparability, you’re right that people *could* be using this in different ways. But, are they? and would it matter if they did? You always get measurement error, whatever you do. An initial response is to point out that if differences are random, they will wash out as ‘noise’. Further, even if something is slightly biased, that wouldn’t make it useless—a bent measuring stick might be better than nothing. The scales don’t need to be literally exactly linear and comparable to be informative. I’ve looked into this issue previously, as have some others, and at HLI we plan to do more on it: again, see this post. I’m not incredibly worried about these things. Some quick evidence. If you look at map of global life satisfaction, it’s pretty clear there’s a shared scale in general. It would be an issue if e.g. Iraq gave themselves 9⁄10.
Equally, it’s pretty clear that people can and do use words and numbers in a meaningful and comparable way.
In your MacAskill quotation, MacAskill is attacking a straw man. When people say something is, e.g. “the best”, we don’t mean the best it is logically possible to be. That wouldn’t be helpful. We mean something more like the “the best that’s actually possible”, i.e. possible in the real world. That’s how we make language meaningful. But yes, in another recent report, we stress that we need more work on understanding the neutral point.
Finally, and the thing I think you’ve really missed about all this, is that: if we’re not going to use subjective wellbeing surveys to find out how well or badly people’s lives are going, what are we going to use instead? Indeed, MacAskill himself says in the same chapter you quote from of What We Owe The Future:
You might ask, Who am I to judge what lives are above or below neutral? The sentiment here is a good one. We should be extremely cautious to figure our how good or bad others’ lives are, as it’s so hard to understand the experiences of people with lives very different to one’s own. The answer is to rely primarily on self-reports
Hello Richard. I’m familiar with the back-and-forths between McMahan and others over the nature and plausibility of TRIA, e.g. those in Gamlund and Solberg (2019) which I assume is still the state of the art (if there’s something better, I would love to know). However, I didn’t want to get into the details here as it would require the introduction of lots of conceptual machinery for very little payoff. (I’ve even been to a whole term of seminars by Jeff McMahan on this topic when I was at Oxford)
But seeing as you’ve raised it …
As Greaves (2019) presses, there is an issue of which person-stages count:
Are the relevant time-relative interests, for instance, only those of present person-stages (“presentism”)? All actual person-stages (“actualism”)? All person-stages that will exist regardless of how one resolves one’s decision (“necessitarianism”)? All person-stages that would exist given some resolution of one’s decision (“possibilism”)? Or something else again?
Whichever choice the TRIA-advocate makes, they will inherit structurally the same issues for those as one finds for the equivalent theories in population ethics (for those, see Greaves (2017)).
The version of TRIA you are referring to is, I think, actualist person-stage version: if so, then the view is not action-guiding (the issue of normative invariance). If you save the child, it will have those future stages, so it’ll be good that it lived; if you don’t save the child, it won’t, so it won’t be bad that it didn’t. Okay, should you save the child? Well, the view doesn’t tell you either way!
The actualist version can’t be the one at hand, as it doesn’t say that it’s good (for the child) if you save it (vs the case where you don’t).
I am, I think, implicitly assuming a present-stage-interest version of TRIA, as that’s the one that generates the value-of-death-at-different-ages curve that is relevantly different from the deprivationist one.
Serious question: Tanae, what else would you like to see? We’ve already displayed the results of the different ethical views, even if we don’t provide a means of editing them
Hello Rhyss. We actually hadn’t considered incorporating a suicide-reducing effect of talk therapy onto our model. I think suicide rates in eg Uganda, one place where SM works, are pretty low—I gather they are pretty low in low-income countries in general.
Quick calculation. I came across these Danish numbers, which found that “After 10 years, the suicide rate for those who had therapy was 229 per 100,000 compared to 314 per 100,000 in the group that did not get the treatment.” Very very naively, then, that’s one life saved via averted suicide per 1,000 treated, or about $150k to save a life via therapy (vs $3-5k for AMF), so probably wouldn’t make much difference. But that is just looking at suicide. We could look at the all-cause mortality effects on treating depression (mental and physical health are often comorbid, etc.).
And read this as you planning to continue evaluating everything in WELLBYs, which in turn I thought meant ruling out evaluating research—because it isn’t clear to me how you evaluate something like psychedelics research using WELLBYs.
If we said we plan to evaluate projects in terms of their ability to save lives, would that rule out us evaluating something like research? I don’t see how it would. You’d simply need to think about the effect that doing some research would have on the number of lives that are saved.
I feel very uncomfortable mutiplying, dividing adding up these wellbys as if they are interchangeable numbers.
Yeah, it is a bit weird, but you get used to it! No weirder than using QALYs, DALYs etc. which people have been doing for 30+ years.
Re grief, here’s what we say in section A.2 of the other report
We do a shallow calculation for grief in the same way we did in Donaldson et al. (2020). The best estimate we found is from Oswald and Powdthavee (2008): a panel study in the UK which finds the effect on life satisfaction due to the death of a child in the last year as being −0.72 (adjusted for a 0-10 scale). According to Clark et al. (2018), the duration of grief is ~5 years. Based on data from the UNDP, we calculate that the average household size across the beneficiary countries (excluding the recipient of the nets) is 4.03 people (row 16). Hence, an overall effect of grief per death prevented is (0.72 x 5 x 0.5) x 4.03 = 7.26 WELLBYs. However, we think this is an upper bound because it doesn’t account for the counterfactual grief averted. If you avert the death of someone, they will still die at some point in the future, and the people who love them will still grieve.
I’m not sure what you find weird about those numbers.
Do you mean a pure time discount rate, or something else?
I think a pure-time discount rate would actually boost the case for StrongMinds, right? Regarding cash vs therapy, the benefits from therapy happen more so at the start. Regarding saving lives vs improving lives, the benefit of a saved life presumably applies over the many extra years the person lives for.
Hello Henry. It may look like we’re just leaning on 2 RCTs, but we’re not! If you read further down in the ‘cash transfers vs treating depression’ section, we mention that we compared cash transfers to talk therapy on the basis of a meta-analysis of each.
The evidence base for therapy is explained in full in Section 4 of our StrongMinds cost-effectiveness analysis. We use four direct studies and a meta-analysis of 39 indirect studies (n > 38,000). You can see how much weight we give to each source of evidence in Table 2, reproduced below. To be clear, we don’t take the results from StrongMinds’ own trials at face value. We basically use an average figure for their effect size, even though they find a high figure themselves.
Also, what’s wrong with the self-reports? People are self-reporting how they feel. How else should we determine how people feel? Should we just ignore them and assume that we know best? Also, we’re comparing self-reports to other self-reports, so it’s unclear what bias we need to worry about.Regarding the issues of comparing saving lives to improving lives, we’ve just written a whole report on how to think about that. We’re hoping that, by bringing these difficult issues to the surface—rather than glossing over them, which is what normally happens—people can make better-informed decisions. We’re very much on your side: we think people should be thinking harder about which does more good.
Thanks for the feedback Julian. I’ve changed the title and added a ‘just’ that was supposed to have been added to final version but somehow got lost when we copied the text across. I don’t know how much that mollifies you...
We really ummed and erred about the title. We concluded that it was cheeky, but not unnecessarily adversarial. Ultimately, it does encapsulate the message we want to convey: that we should compare charities by their overall impact on wellbeing, which is what we think the WELLBY captures.
I don’t think many people really understand how GiveWell compares charities, they just trust GiveWell because that’s what other people seem to do. HLI’s whole vibe is that, if we think hard about what matters, and how we measure that, we can find even more impactful opportunities. We think that’s exactly what we’ve been able to do—the, admittedly kinda lame, slogan we sometimes use is ‘doing good better by measuring good better’.
To press the point, I wouldn’t even know how to calculate the cost-effectiveness of StrongMinds on GiveWell’s framework. It has two inputs: (1) income and (2) additional years of life. Is treating depression good just because it makes you richer? Because it helps you life longer? That really seems to miss the point. Hence, the WELLBY.
Unless GiveWell adopt the WELLBY, we will inevitably be competing with them to some extent. The question is not whether we compete—the only way we could not compete would be by shutting down—it’s how best to do it. Needless antagonism is obviously bad. We plan to poke and prod GiveWell in a professional and humourful way when we think they could do better—something we’ve been doing for several years already—and we hope they’ll do the same to us. Increased competition in the charity evaluation space will be better for everyone (see GWWC’s recent post).
Don’t just give well, give WELLBYs: HLI’s 2022 charity recommendation
I am super interested in this problem, but finding it a bit tricky to follow your analysis—in part because it’s so spread out and lots of the information seems buried seem in large paragraphs. Is there any chance you could put in it a spreadsheet?
For instance, I’m still unsure what your hypothetical intervention is—providing training to clinics to better diagnose mental health issues and then prescribe SSRIs, is that right?
Hi! Thanks for engaging. Yes, the issue you raise does get discussed quite a bit, and is much worried about (by effective altruists but not in general). I’ve got a working paper here where I review the theory and evidence and tentatively conclude people probably do interpret the difference between each unit as having the same value (i.e. the scales are interpreted linearly).
My colleague, Casper Kaiser, who is also an HLI trustee, has a more recent paper which shows an approximately linear relationship between reports and later behaviour.
Internally at HLI we’re working on doing our own survey on this stuff too!
Why AMF is the best under deprivationism followed by TRIA (AC=5), then TRIA (AC=25), then Epicureanism.
Um, because these are literally the results these views are structured to give! To me, your question is akin to asking “why does consequentialism care more about consequences than deontology?” Sorry, maybe I’ve misunderstood.
Why StrongMinds is generally better than AMF (almost irregardless of what your philosophical view is).
To be clear, there is no intuition here! These are the outputs of an empirical analysis. There’s absolutely no reason it has to be true that the purported best life-extending intervention is better, under a range of different philosophical assumptions, than the purported best life-improving one. In a nearby possible world, AMF* could have been very many times more cost-effective on the assumptions most generous to saving lives.
It is worth noting the portfolio approach corresponding to worldview diversification applies to the allocation of resources of the community as a whole, as far as I understand.
It does? Says who? And why does it? Given that attempts there have been, as far as I can tell, almost nil attempts to think through the worldview diversification approach—despite it being appealed to in decision-making for many years—it strikes me as an open question about how it should be understood. I see moral uncertainty as asking a first-personal question—what should I do, given my beliefs about morality?
I also wonder whether GiveWell’s moral weights being majorly determined by its donors (60 %) has the intention of capturing other effects besides those directly related to the death of the person
Ah, I too used to spend many hours wondering what GiveWell really thought about things. But now I am a man, I have put away such childish things.
Well, if we think feelings matter, we should try to quantify them in a sensible way. That’s what we try to do.
But I share the sentiment that you really do miss something if you try to quantify feelings by measuring something other than feelings, such as income.
The elephant in the bednet: the importance of philosophy when choosing between extending and improving lives
Just skimmed this, but note that there is an existing literature in philosophy on whether non-consequentialist theories can or should be ‘consequentialised’, that is, turned into consequentialist theories, e.g. Portmore (2009), Brown (2011), Sinnott-Armstrong (2019), Shroeder (2019), Muñoz (2021). I found these in 5 minutes, so there’s probably loads more.
A very general problem with the move, as Sinnott-Armstrong (2019) points out, is that if all theories can be re-presented as consequentialist, then it means little to label a theory as consequentialist. Even if successful, we would then have many different ‘consequentialisms’ that suggest, in practice, different things: should you kill the one and harvest their organs to save the five?
Of course, all minimally plausible versions of deontology and virtue ethics must be concerned in part with promoting the good. As John Rawls, not a consequentialist himself, famously put in A Theory of Justice: “All ethical doctrines worth our attention take consequences into account in judging rightness. One which did not would simply be irrational, crazy.” That does not, however, make everyone a consequentialist.
I don’t understand this conversation. The common take is that SBF has been caught admitting he’s a Machiavellian nihilist but then … he did contact a journalist and choose to say this. He cannot have reasonably thought it wouldn’t be shared. What was he hoping to achieve? Seems like bizarre self-sabotage. I wondered if it was some contorted attempt to make himself the fall guy rather than have EA thinking take the blame.
Hello William,
Thanks for saying that. Yeah, I couldn’t really understand where you were coming from (and honestly ended up spending 2+ hours drafting a reply).
On reflection, we should probably have done more WELLBY-related referencing in the post, but we were trying to keep the academic side light. In fact, we probably need to recombine our various scratching on the WELLBY and put them onto a single page on our website—it’s been a lower priority than doing the object-level charity analysis work.
If you’re doing the independent impression thing again, then, as a recipient, it would have been really helpful to know that. Then I would have read it more as a friendly “I’m new to this and sceptical and X and Y—what’s going on with those?” and less as a “I’m sceptical, you clearly have no idea what you’re talking about” (which was more-or-less how I initially interpreted it… :) )