I think I’m more skeptical than you that reasonable alternative assumptions make StrongMinds look more cost effective than AMF. But I agree that StrongMinds seems like it could be a good fit for some donors.
Interested if you could elaborate here. I’m not sure which intuitions you consider ‘reasonable’ and why. As Joel’s figure 4 above indicates, for either HLI’s or GiveWell’s estimates of StrongMinds, talk therapy can be more cost-effective than bednets, and vice versa, but which is more effective depends on the philosophical assumptions—so that ought to be the debate we’re having, but aren’t. Perhaps we have a much more capacious concept of which assumptions are reasonable and you’d want to rule some of those out? If so, which ones?
I’m not sure if this is what you meant, but if we’re talking about a raw difference in philosophical/ethical intuitions, I am very reluctant to say that some of those are unreasonable—in case any reader isn’t aware of this, philosophy seminars often end up with the discussants realising they just have different intuitions. To say, at this point, your intuitions are reasonable and the other person’s are unreasonable is the rhetorical equivalent of banging your fist on the table—you’re not making a new argument, you’re just hoping the other person will agree with you anyway!
I think each part of this chart has some assumptions I don’t think are defensible.
1. I don’t think a neutral point higher than 2 is defensible.
You cite three studies in this report.[1] My read on what to conclude about the neutral point from those is:
i) IDinsight 2019 (n=70; representative of GW recipients): you highlight the average answer of 0.56, but this is excluding the 1⁄3 of people who say it’s not possible to have a life worse than death.[2] I think including those as 0 more accurately reflects their preferences, so 2/3*0.56=0.37/10
ii) Peasgood et al, unpublished (n=75; UK): you say 2⁄10 and I can’t find the study so I’m taking that at face value.
iii) Jamison and Shukla, unpublished (n=600, US, Brazil, China): you highlight the average answer of 25⁄100. In private communication with the author, I got the impression that 1.8/10 was probably more appropriate because the scale used in this study isn’t comparable to typical life satisfaction scales.[3]
So what to make of this? I think you could reasonably put weight in the largest study (1.8/10). Or you could put weight on the most representative study (0.37). I lean towards the latter, because I intuitively find it quite likely that less well off people will report lower neutral points (I don’t feel certain about this, and hoping Jamison & Shukla will have enough sample to test it). But either way, I don’t see any way of combining these studies to get an answer higher than 2.
In addition, a neutral point of 5 implies the average person in over 40 countries would be better off dead. A neutral point of 2.5 implies the average person in Afghanistan would be better off dead. I find these both jarring implications.
HLI’s belief that a neutral point of 5 is within a reasonable range seems to come from Diener et al. 2018.[4] But that article’s not explicit about what it means by “neutral point”. As far as I can tell from a quick skim, it seems to be defining “neutral” as halfway between 0 and 10.
2. I don’t think 38% is a defensible estimate for spillovers, which puts me closer to GiveWell’s estimate of StrongMinds than HLI’s estimate of StrongMinds.
I wrote this critique of your estimate that household spillovers was 52%. That critique had three parts. The third part was an error, which you corrected and brought the answer down to 38%. But I think the first two are actually more important: you’re deriving a general household spillover effect from studies specifically designed to help household members, which would lead to an overestimate.
I thought you agreed with that from your response here, so I’m confused as to why you’re still defending 38%. Flagging that I’m not saying the studies themselves are weak (though it’s true that they’re not very highly powered). I’m saying they’re estimating a different thing from what you’re trying to estimate, and there are good reasons to think the thing they’re trying to estimate is higher. So I think your estimate should be lower.
3. I don’t think strong epicureanism is a defensible position
Strong epicureanism (the red line) is the view that death isn’t bad for the person who dies. I think it’s logically possible to hold this position as a thought experiment in a philosophy seminar, but I’ve never met anyone who actually believes it and I’d be deeply troubled if decisionmakers took action on the basis of it. You seem to agree to some extent,[5] but by elevating it to this chart, and putting it alongside the claim that “Against Malaria Foundation is less cost-effective than StrongMinds under almost all assumptions” I think you’re implying this is a reasonable position to take action on, and I don’t think it is.
So I think my version of this chart looks quite different: the x-axis is between 0.4 and 2, the StrongMinds estimate’s quite a bit closer to GiveWell than HLI, and there’s no “epicureanism” line.
What does HLI actually believe?
More broadly, I’m quite confused about how strongly HLI is recommending StrongMinds. In this post, you say (emphasis mine)
We conclude that the Against Malaria Foundation is less cost-effective than StrongMinds under almost all assumptions. We expect this conclusion will similarly apply to the other life-extending charities recommended by GiveWell.
And
We’re now in a position to confidently recommend StrongMinds as the most effective way we know of to help other people with your money.
But you’ve said elsewhere:
HLI does not (yet) take a stance on these different philosophical views.
That strikes me as inconsistent. You’ve defined a range of assumptions you believe are reasonable, then claimed that StrongMinds > AMF on almost all of those assumptions. And then said you don’t take a stance on these assumptions. But you have to actually defend the range of assumptions you’ve defined as reasonable. And in my view, they’re not.
“Empirical work on how individuals interpret the scale could be helpful but is extremely limited. A small (n = 75) survey in the UK found that respondents would choose death over life at a life satisfaction level of about 2⁄10 (Peasgood et al., unpublished, as referenced in Krekel & Frijters, 2021). A survey of people living in poverty in Ghana and Kenya estimated the neutral point as 0.56 (IDinsight, 2019, p. 92; n = 70). There are also preliminary results from a sample of 600 in the USA, Brazil, and China that finds a neutral point of 25⁄100 (Jamison & Shukla, private communication). At the Happier Lives Institute, we are currently working on our own survey to explore this topic further and hope to share our results soon.” Elephant in the bednet
“Approximately one third of respondents stated that it’s not possible to have a life that’s worse than death. These respondents cited deontological frameworks such as the inherent and immeasurable value of life regardless of other factors. The remaining respondents (close to two thirds) indicate that there are points on the ladder where life is worse than death. For these respondents, this point is substantially lower than their current life satisfaction scores –the average point identified was 0.56 on a ladder from 0 to 10, compared to their current average life satisfaction score of 2.21” IDInsight 2019, pg 94
“Other wellbeing researchers, such as Diener et al. (2018), appear to treat the midway point on the scale as the neutral point (i.e., 5 on a 0-10 scale).” Elephant in the bednet
“Although what we might call strong Epicureanism, the view that death is not bad at all, has few takers, there may be more sympathy for weak Epicureanism, where death can be bad, but relatively more weight is given to living well than living long” Elephant in the bednet
Epicureanism is discussed in almost every philosophy course on the badness of death. It’s taken seriously, rather than treated as an absurd position, a non-starter, and whilst not that many philosophers end up as Epicureans, I’ve met some that are very sympathetic. I find critics dismiss the view too quickly and I’ve not seen anything that’s convinced me the view has no merit. I don’t think we should have zero credence in it, and it seems reasonable to point out that it is one of the options. Again, I’m inclined to let donors make up their own minds.
On what HLI actually believes
HLI is currently trying not to have a view on these issues, but point out to donors how having different views would change the priorities so they can form their own view. We may have to develop a ‘house view’ but none of the options for doing this seem particularly appealing (they include: we use my view, we use a staff aggregate, we poll donors, we poll the public, some combo of the previous options).
You bring up this quote
We’re now in a position to confidently recommend StrongMinds as the most effective way we know of to help other people with your money.
I regret this sentence, which is insufficiently nuanced and I wouldn’t use it again (you and I have discussed this privately). That said, I think we’re quite well-caveated elsewhere. You quote this bullet point:
We conclude that the Against Malaria Foundation is less cost-effective than StrongMinds under almost all assumptions. We expect this conclusion will similarly apply to the other life-extending charities recommended by GiveWell.
But you didn’t quote the bullet point directly before it (emphasis added):
In our new report, The Elephant in the Bednet, we show that the relative value of life-extending and life-improving interventions depends very heavily on the philosophical assumptions you make. This issue is usually glossed over and there is no simple answer.
The backstory to the “we confidently recommend StrongMinds” bit is that, when we did the analysis, StrongMinds looked better under almost all assumptions and, even where AMF was better, it was only slightly better (1.3x). We thought donors would want an overall recommendation, and hence StrongMinds seemed like the safe choice (given some intuitions about donors’ intuitions and moral uncertainty). You’re right that we’ll have to rethink what our overall recommendations are, and how to frame them, once the dust has settled on this debate.
Finally, whilst you say
But you have to actually defend the range of assumptions you’ve defined as reasonable. And in my view, they’re not.
This feels uneasily like a double standard. As I’ve pointed out before, neither GiveWell nor Open Philanthropy really defends their views in general (asserting a view isn’t the same as defending it). In this report, GiveWell doesn’t defend its assumptions, point out what other assumptions one might (reasonably) take, or say how this would change the result. Part of what we have tried to highlight in our work is that these issues have been mostly ignored and can really matter.
Our aim was more to cover the range of views we think some reasonable people would believe, not to restrict it to what we think they should believe. We motivated our choices in the original report and will restate that briefly here. For the badness of death, we give the three standard views in the literature. At one end, deprivationism gives ‘full value’ to saving lives. On the other, Epicurianism gives no weight to saving lives. TRIA offers something in between. For the neutral point, we used a range that included what we saw as the minimum and maximum possible values. Including a range of values is not equivalent to saying they are all equally probable. We encourage donors and decision-makers to use values they think are most plausible (for example, by using this interactive chart).
In our new report, The Elephant in the Bednet, we show that the relative value of life-extending and life-improving interventions depends very heavily on the philosophical assumptions you make. This issue is usually glossed over and there is no simple answer.
We conclude that the Against Malaria Foundation is less cost-effective than StrongMinds under almost all assumptions. We expect this conclusion will similarly apply to the other life-extending charities recommended by GiveWell.
In suggesting James quote these together, it sounds like you’re saying something like “this is a clear caveat to the strength of recommendation behind StrongMinds, HLI doesn’t recommend StrongMinds as strongly as the individual bullet implies, it’s misleading for you to not include this”.
As a result of this framing, despite the caveat that HLI “[does] not advocate for any particular view”, I think it’s reasonable to interpret this as being strongly supportive of StrongMinds, which can be true even if HLI does not have a formed view on the exact philosophical view to take.[2]
If you did mean the former (that the bullet about philosophical assumptions is primarily included as a caveat to the strength of recommendation behind StrongMinds), then there is probably some tension here between (emphasis added):
-”the relative value of life-extending and life-improving interventions depends very heavily on the philosophical assumptions you make...there is no simple answer”, and
-”We conclude StrongMinds > AMF under almost all assumptions”
Additionally I think some weak evidence to suggest that HLI is not as well-caveated as it could be is that many people (mistakenly) viewed HLI as an advocacy organisation for mental health interventions. I do think this is a reasonable outside interpretation based on HLI’s communications, even though this is not HLI’s stated intent. For example, I don’t think it would be unreasonable for an outsider to read your current pinned thread and come away with conclusions like:
“StrongMinds is the best place to donate”,
“StrongMinds is better than AMF”,
“Mental health is a very good place to donate if you want to do the most good”,
“Happiness is what ultimately matters for wellbeing and what should be measured”.
If these are not what you want people to take away, then I think pointing to this bullet point caveat doesn’t really meaningfully address this concern—the response kind of feels something like “you should have read the fine print”. While I don’t think it’s not necessary for HLI to take a stance on specific philosophical views, I do think it becomes an issue if people are (mis)interpreting HLI’s stance based on its published statements.
-We show how much cost-effectiveness changes by shifting from one extreme of (reasonable) opinion to the other. At one end, AMF is 1.3x better than StrongMinds. At the other, StrongMinds is 12x better than AMF.
-StrongMinds and GiveDirectly are represented with flat, dashed lines because their cost-effectiveness does not change under the different assumptions.
-As you can see, AMF’s cost-effectiveness changes a lot. It is only more cost-effective than StrongMinds if you adopt deprivationism and place the neutral point below 1.
As you’ve acknowledged, comments like “We’re now in a position to confidently recommend StrongMinds as the most effective way we know of to help other people with your money.” perhaps add to the confusion.
Do you think the neutral point and basic philosophical perspective (e.g., deprivationism vs. epicureanism) are empirical questions, or are they matters on which the donor has to exercise their own moral and philosophical judgment (after considering what the somewhat limited survey data have to say on the topic)?
I would graph the neutral point from 0 to 3. I think very few donors would set the neutral point above 3, and I’d start with the presumption that the most balanced way to present the chart is probably to center it fairly near the best guess from the survey data. On the other hand, if you have most of the surveys reporting “about 2,” then it’s hard to characterize 3 as an outlier view—presumably, a good fraction of the respondents picked a value near, at, or even over 3.
Although I don’t think HLI puts it this way, it doesn’t strike me as implausible to view human suffering as a more severe problem than lost human happiness. As I noted in a different comment, I think of that chart as a starting point from which a donor can apply various discounts and bonuses on a number of potentially relevant factors. But another way to account for this would be to give partial weight to strong epicureanism as a means of discounting the value of lost human happiness vis-a-vis suffering.
Given that your critique was published after HLI’s 2022 charity recommendation, I think it’s fair to ask HLI whether it would reaffirm those characterizations today. I would agree that the appropriate conclusion, on HLI’s current state of analysis, is that the recommendation is either SM or GiveWell’s top charities depending on the donor’s philosophical assumptions. I don’t think it’s inappropriate to make a recommendation based on the charity evaluator’s own philosophical judgment, but unless HLI has changed its stance it has taken no position. I don’t think it is appropriate to merely assume equal credence for each of the philosophical views and neutral points under consideration.
One could also defensibly make a summary recommendation on stated assumptions about donor values or on receipient values. But the best information I’ve seen on those points—the donor and beneficiary surveys as reflected in GiveWell’s moral weights—seemingly points to a predominately deprivationist approach with a pretty low neutral point (otherwise the extremely high value on saving the lives of young children wouldn’t compute).
Thanks Jason, mostly agree with paras 4-5, and think para 2 is a good point as well.
Do you think the neutral point and basic philosophical perspective (e.g., deprivationism vs. epicureanism) are empirical questions, or are they matters on which the donor has to exercise their own moral and philosophical judgment (after considering what the somewhat limited survey data have to say on the topic)?
I think the basic philosophical perspective is a moral/philosophical judgement. But the neutral point combines that moral judgement with empirical models of what peoples’ lives are actually like, and empirical beliefs about how people respond to surveys.
I wonder if, insofar as we do have different perspectives on this (and I don’t think we’re particularly far apart, particularly on the object level question), the crux is around how much weight to put in individual donor judgement? Or even how much individual donors have those judgements?
My experience of even EA-minded (or at least GiveWell) donors is that ~none of them have a position on these kinds of questions, and they actively want to defer. My (less confident but based on quite a few conversations) model of EA-minded StrongMinds donors is they want to give to mental health and see an EA-approved charity so give there, rather than because of a quantitative belief on foundational questions like the neutral point. As an aside, I believe that was how StrongMinds first got on EA’s radar—as a recommendation for Founders Pledge donors who specifically wanted to give to mental health in an evidence-based way.
It does seem plausible to me that donors who follow HLI recommendations (who I expect are particularly philosophically minded) would be more willing to change their decisions based on these kinds of questions than donors I’ve talked to.
I’d be interested if someone wanted to stick up for a neutral point of 3 as something they actually believe and a crux for where they give, rather than something someone could believe, or is plausible. I could be wrong, but I’m starting out skeptical that belief would survive contact with “But that implies the world would be better if everyone in Afghanistan died” and “a representative survey of people whose deaths you’d be preventing think their lives are more valuable than that”
From HLI’s perspective, it makes sense to describe how the moral/philosophical views one assumes affect the relative effectiveness of charities. They are, after all, a charity recommender, and donors are their “clients” in a sense. GiveWell doesn’t really do this, which makes sense—GiveWell’s moral weights are so weighted toward saving lives that it doesn’t really make sense for them to investigate charities with other modes of action. I think it’s fine to provide a bottom-line recommendation on whatever moral/philosophical view a recommender feels is best-supported, but it’s hardly obligatory.
We recognize donor preferences in that we don’t create a grand theory of effectiveness and push everyone to donate to longtermist organizations, or animal-welfare organizations, or global health organizations depending on the grand theory’s output. Donors choose among these for their own idiosyncratic reasons, but moral/philosophical views are certainly among the critical criteria for many donors. I don’t see why that shouldn’t be the case for interventions within a cause area that produce different kinds of outputs as well.
Here, I doubt most global-health donors—either those who take advice from GiveWell or from HLI—have finely-tuned views on deprivationism, neutral points, and so on. However, I think many donors do have preferences that indirectly track on some of those issues. For instance, you describe a class of donors who “want to give to mental health.” While there could be various reasons for that, it’s plausible to me that these donors place more of an emphasis on improving experience for those who are alive (e.g., they give partial credence to epicureanism) and/or on alleviating suffering. If they did assess and chart their views on neutral point and philosophical view, I would expect them to end up more often at points where SM is ranked relatively higher than the average global-health donor would. But that is just conjecture on my part.
One interesting aspect of thinking from the donor perspective is the possibility that survey results could be significantly affected by religious beliefs. If many respondents chose a 0 neutral point because their religious tradition led them to that conclusion, and you are quite convinced that the religious tradition is just wrong in general, do you adjust for that? Does not adjusting allow the religious tradition to indirectly influence where you spend your charitable dollar?
To me, the most important thing a charity evaluator/recommender does is clearly communicate what the donation accomplishes (on average) if given to various organizations they identify—X lives saved (and smaller benefits), or Y number of people’s well-being improved by Z amount. That’s the part the donor can’t do themselves (without investing a ton of time and resources).
I don’t think the neutral point is as high as 3. But I think it’s fine for HLI to offer recommendations for people who do.
Hi James, thanks for elaborating, that’s really useful! We’ll reply to your points in separate comments.
Your statement, 1. I don’t think a neutral point higher than 2 is defensible
Reply: I don’t think we have enough evidence or theory to be confident about where to put the neutral point.
Your response about where to put the neutral point involves taking answers to survey questions where people are asked something like “where on a 0-10 scale would you choose not to keep living?” and assuming we should take those answers at face value for where to locate the neutral point. However, this conclusion strikes me as too fast; I don’t think we have enough theory or evidence on this issue. Are we definitely asking the right questions? Do we understand people’s responses? Should we agree with them even if we understand them?
I’m not sure if I told you about this, but we’re working on a pilot survey for this and other wellbeing measuring issues. The relevant sections for neutrality are 1.3 and 6. I’ll try to put the main bits here, to make life easier (link to initial part of report on EA forum; link to full report):
The neutral point refers to the level on a SWB scale at which existence has a neutral value, compared to non-existence for that person (assuming this state is perpetual and considering only the effects on that person). Above this point life is ‘better than death’; below it life is ‘worse than death’. This is conceptually distinct, but possibly closely related, to what we call the zero point: the level on a SWB scale at which that type of SWB is overall neither positive nor negative (e.g., someone is neither overall satisfied or dissatisfied). A natural thought is that the zero point and the neutral point coincide: if life is good(/bad) for us when it has positive(/negative) wellbeing, so a life has neutral value if it has zero wellbeing.
As we say in footnote 5, this aligned is straightforwardly entailed by the standard formulation of utilitarianism.
That’s why,
[In our pilot survey, w]e test whether respondents put the neutral point and zero point for life satisfaction in the same place, and whether respondents interpret the zero point for life satisfaction as somewhere between 0 and 5. If respondents do both, that would provide an explanation of previous works’ findings. However, there are several reasons why respondents might not believe that the neutral point and the zero point coincide. Some of these issues are discussed in Section 6.2.
Unfortunately, we find that, on a life satisfaction scale, participants put the zero point at 5⁄10, and the neutral point at 1.3/10. We’re not really sure what to make of this. Here’s what we say in section 6.2 in full.
Survey questions about the neutral point cannot settle the debate about when a life is not worth living on their own. At most, we can elicit respondents’ beliefs and preferences about their own lives, which, at least on some moral theories, will be an important determinant of the neutral point.
As noted in Section 1.3, an intuitive thought about the value of extending lives is that living longer is good for you if you would have positive wellbeing, and bad for you if you would have negative wellbeing. Yet, the above result seems in tension with this. We might expect someone who is overall dissatisfied with their life would say that they have negative wellbeing. However, if you have negative wellbeing, shouldn’t you also believe that living longer would be bad for you? Note that we have specifically asked respondents to disregard the effects of their existence on others. This should rule out participants thinking their life is “worth living” merely because their existence is good for other people (e.g., their family). In light of these arguments, how might we explain the divergences between the reported neutral point and the reported zero point?
One reason participants might not align the neutral point with the zero point on a life satisfaction scale is that participants are not endorsing a life satisfaction theory of wellbeing. That is, respondents do not believe that their wellbeing is wholly represented by their overall satisfaction with life. If, for instance, participants ultimately valued their happiness, and they expect that they would be happy even if they were very dissatisfied, then they would justifiably conclude that even if they were dissatisfied with their life, it would be worth it, for them, to keep living.
A distinct possibility is that there are special theoretical considerations connected to the life satisfaction theory of wellbeing. Part of the motivation for such a theory is that individuals get to choose what makes their lives go well (Sumner 1999, Plant 2020). Hence, perhaps individuals could coherently distinguish between the level of satisfaction at which they would rather stop existing (the neutral point) and the level at which they are neither overall satisfied or dissatisfied (the zero point).13
What the previous two comments reveal is that investigations into the neutral point may substantially turn on philosophical assumptions. We may need a ‘theory-led’ approach, where we decide what theory of wellbeing we think is correct, and then consider how, given a particular theory of wellbeing, the location of the neutral point would be determined. This would contrast with a ‘data-led’ approach where we strive to be theory agnostic.
Another concern is that our survey questions are too cognitively demanding for participants. Perhaps respondents do not, in fact, account for the stipulation that they should discount the effects on others. Alternatively, respondents might answer our questions on the assumption that their life would only temporarily, rather than permanently, be at that level of life satisfaction. In future iterations of our survey, we may try to get a better understanding of this possibility.
Finally, there may be experimenter demand effects. Respondents may think it is important to express the value of human life, and therefore it is wrong to say a life is ‘not worth living’ unless a life is truly miserable.
For these reasons, we remain unsure whether a level of 1.3/10, which is the sample mean on our neutrality question, indeed gives a valid estimate of the ‘true’ neutral point.
We plan to think about this more and test our hypotheses in the full version of the survey. If you have ideas for what we should test, now would be a great time to share them!
Given the methodological challenges in measuring the neutral point, I would have some hesitation to credit any conclusions that diverged too much from what revealed preferences imply. A high neutral point implies that many people in developing countries believe their lives are not worth living. So I’d look for evidence of behavior (either in respondents or in the population more generally) that corroborated whether people acted in a way that was consistent with the candidate neutral point.
For instance, although money, family, and other considerations doubtless affect it, studying individuals who are faced with serious and permanent (or terminal) medical conditions might be helpful. At what expected life satisfaction score do they decline treatment? If the neutral point is relatively close to the median point in a country, one would expect to see a lot of people decide to not obtain curative treatment if the results would leave them 1-2 points less satisfied than their baseline.
You might be able to approximate that by asking hypothetical questions about specific situations that you believe respondents would assess as reducing life satisfaction by a specified amount (disability, imprisonment, social stigma, etc.), and then ask whether the respondent believes they would find life still worth living if that happened. I don’t think that approach works to establish a neutral point, but I think having something more concrete would be an important cross-check on what may otherwise come across as an academic, conjectural exercise to many respondents.
A high neutral point implies that many people in developing countries believe their lives are not worth living.
This isn’t necessarily the case. I assume that if people described their lives as having negative wellbeing, this wouldn’t imply they thought their life was not worth continuing.
People can have negative wellbeing and still want to live for the sake of others or causes greater than themselves.
Life satisfaction appears to be increasing over time in low income countries. I think this progress is such that many people who may have negative wellbeing at present, will not have negative wellbeing their whole lives.
Edit: To expand a little, for these reasons, as well as the very reasonable drive to survive (regardless of wellbeing), I find it difficult to interpret revealed preferences and it’s unclear they’re a bastion of clarity in this confusing debate.
Anectdotally, I’ve clearly had periods of negative wellbeing before (sometimes starkly), but never wanted to die during those periods. If I knew that such periods were permanent, I’d probably think it was good for me to not-exist, but I’d still hesitate to say I’d prefer to not-exist, because I don’t just care about my wellbeing. As Tyrion said “Death is so final, and life is so full of possibilities.”
I think these difficulties should highlight that the difficulties here aren’t just localized to this area of the topic.
Thanks for these points! The idea that people care about more than their wellbeing may be critical here. I’m thinking of a simplified model with the following assumptions: a mean lifetime wellbeing of 5, SD 2, normal distribution, wellbeing is constant through the lifespan, with a neutral point of 4 (which is shared by everyone).
Under these assumptions, AMF gets no “credit” (except for grief avoidance) for saving the life of a hypothetical person with wellbeing of 4. I’m really hesitant to say that saving that person’s life doesn’t morally “count” as a good because they are at the neutral point. On the one hand, the model tells me that saving this person’s life doesn’t improve total wellbeing. On the other hand, suppose I (figuratively) asked the person whose life was saved, and he said that he preferred his existence to non-existence and appreciated AMF saving his life.
At that point, I think the WELLBY-based model might not be incorporating some important data—the person telling us that he prefers his existence to non-existence would strongly suggest that saving his life had moral value that should indeed “count” as a moral good in the AMF column. His answers may not be fully consistent, but it’s not obvious to me why I should fully credit his self-reported wellbeing but give zero credence to his view on the desirability of his continued existence. I guess he could be wrong to prefer his continued existence, but he is uniquely qualified to answer that question and so I think I should be really hesitant to completely discount what he says. And a full 30% of the population would have wellbeing of 4 or less under the assumptions.
Even more concerning, AMF gets significantly “penalized” for saving the life of a hypothetical person with wellbeing of 3 who also prefers existence to non-existence. And almost 16% of the population would score at least that low.
Of course, the real world is messier than a quick model. But if you have a population where the neutral point is close enough to the population average, but almost everyone prefers continued existence, it seems that you are going to have a meaningful number of cases where AMF gets very little / no / negative moral “credit” for saving the lives of people who want (or would want) their lives saved. That seems like a weakness, not a feature, of the WELLBY-based model to me.
2. I don’t think 38% is a defensible estimate for spillovers, which puts me closer to GiveWell’s estimate of StrongMinds than HLI’s estimate of StrongMinds.
I wrote this critique of your estimate that household spillovers was 52%. That critique had three parts. The third part was an error, which you corrected and brought the answer down to 38%. But I think the first two are actually more important: you’re deriving a general household spillover effect from studies specifically designed to help household members, which would lead to an overestimate.
I thought you agreed with that from your response here, so I’m confused as to why you’re still defending 38%. Flagging that I’m not saying the studies themselves are weak (though it’s true that they’re not very highly powered). I’m saying they’re estimating a different thing from what you’re trying to estimate, and there are good reasons to think the thing they’re trying to estimate is higher. So I think your estimate should be lower.
I could have been clearer, the 38% is a placeholder while I do the Barker et al. 2022 analysis. You did update me about the previous studies’ relevance. My arguments are less supporting the 38% figure—which I expect to update with more data and more about explaining why I think that I have a higher prior for household spillovers from psychotherapy than you and Alex seem to. But really, the hope is that we can soon be discussing more and better evidence.
Interested if you could elaborate here. I’m not sure which intuitions you consider ‘reasonable’ and why. As Joel’s figure 4 above indicates, for either HLI’s or GiveWell’s estimates of StrongMinds, talk therapy can be more cost-effective than bednets, and vice versa, but which is more effective depends on the philosophical assumptions—so that ought to be the debate we’re having, but aren’t. Perhaps we have a much more capacious concept of which assumptions are reasonable and you’d want to rule some of those out? If so, which ones?
I’m not sure if this is what you meant, but if we’re talking about a raw difference in philosophical/ethical intuitions, I am very reluctant to say that some of those are unreasonable—in case any reader isn’t aware of this, philosophy seminars often end up with the discussants realising they just have different intuitions. To say, at this point, your intuitions are reasonable and the other person’s are unreasonable is the rhetorical equivalent of banging your fist on the table—you’re not making a new argument, you’re just hoping the other person will agree with you anyway!
Sure, happy to elaborate.
Here’s figure 4 for reference:
I think each part of this chart has some assumptions I don’t think are defensible.
1. I don’t think a neutral point higher than 2 is defensible.
You cite three studies in this report.[1] My read on what to conclude about the neutral point from those is:
i) IDinsight 2019 (n=70; representative of GW recipients): you highlight the average answer of 0.56, but this is excluding the 1⁄3 of people who say it’s not possible to have a life worse than death.[2] I think including those as 0 more accurately reflects their preferences, so 2/3*0.56=0.37/10
ii) Peasgood et al, unpublished (n=75; UK): you say 2⁄10 and I can’t find the study so I’m taking that at face value.
iii) Jamison and Shukla, unpublished (n=600, US, Brazil, China): you highlight the average answer of 25⁄100. In private communication with the author, I got the impression that 1.8/10 was probably more appropriate because the scale used in this study isn’t comparable to typical life satisfaction scales.[3]
So what to make of this? I think you could reasonably put weight in the largest study (1.8/10). Or you could put weight on the most representative study (0.37). I lean towards the latter, because I intuitively find it quite likely that less well off people will report lower neutral points (I don’t feel certain about this, and hoping Jamison & Shukla will have enough sample to test it). But either way, I don’t see any way of combining these studies to get an answer higher than 2.
In addition, a neutral point of 5 implies the average person in over 40 countries would be better off dead. A neutral point of 2.5 implies the average person in Afghanistan would be better off dead. I find these both jarring implications.
HLI’s belief that a neutral point of 5 is within a reasonable range seems to come from Diener et al. 2018.[4] But that article’s not explicit about what it means by “neutral point”. As far as I can tell from a quick skim, it seems to be defining “neutral” as halfway between 0 and 10.
2. I don’t think 38% is a defensible estimate for spillovers, which puts me closer to GiveWell’s estimate of StrongMinds than HLI’s estimate of StrongMinds.
I wrote this critique of your estimate that household spillovers was 52%. That critique had three parts. The third part was an error, which you corrected and brought the answer down to 38%. But I think the first two are actually more important: you’re deriving a general household spillover effect from studies specifically designed to help household members, which would lead to an overestimate.
I thought you agreed with that from your response here, so I’m confused as to why you’re still defending 38%. Flagging that I’m not saying the studies themselves are weak (though it’s true that they’re not very highly powered). I’m saying they’re estimating a different thing from what you’re trying to estimate, and there are good reasons to think the thing they’re trying to estimate is higher. So I think your estimate should be lower.
3. I don’t think strong epicureanism is a defensible position
Strong epicureanism (the red line) is the view that death isn’t bad for the person who dies. I think it’s logically possible to hold this position as a thought experiment in a philosophy seminar, but I’ve never met anyone who actually believes it and I’d be deeply troubled if decisionmakers took action on the basis of it. You seem to agree to some extent,[5] but by elevating it to this chart, and putting it alongside the claim that “Against Malaria Foundation is less cost-effective than StrongMinds under almost all assumptions” I think you’re implying this is a reasonable position to take action on, and I don’t think it is.
So I think my version of this chart looks quite different: the x-axis is between 0.4 and 2, the StrongMinds estimate’s quite a bit closer to GiveWell than HLI, and there’s no “epicureanism” line.
What does HLI actually believe?
More broadly, I’m quite confused about how strongly HLI is recommending StrongMinds. In this post, you say (emphasis mine)
And
But you’ve said elsewhere:
That strikes me as inconsistent. You’ve defined a range of assumptions you believe are reasonable, then claimed that StrongMinds > AMF on almost all of those assumptions. And then said you don’t take a stance on these assumptions. But you have to actually defend the range of assumptions you’ve defined as reasonable. And in my view, they’re not.
“Empirical work on how individuals interpret the scale could be helpful but is extremely limited. A small (n = 75) survey in the UK found that respondents would choose death over life at a life satisfaction level of about 2⁄10 (Peasgood et al., unpublished, as referenced in Krekel & Frijters, 2021). A survey of people living in poverty in Ghana and Kenya estimated the neutral point as 0.56 (IDinsight, 2019, p. 92; n = 70). There are also preliminary results from a sample of 600 in the USA, Brazil, and China that finds a neutral point of 25⁄100 (Jamison & Shukla, private communication). At the Happier Lives Institute, we are currently working on our own survey to explore this topic further and hope to share our results soon.” Elephant in the bednet
“Approximately one third of respondents stated that it’s not possible to have a life that’s worse than death. These respondents cited deontological frameworks such as the inherent and immeasurable value of life regardless of other factors. The remaining respondents (close to two thirds) indicate that there are points on the ladder where life is worse than death. For these respondents, this point is substantially lower than their current life satisfaction scores –the average point identified was 0.56 on a ladder from 0 to 10, compared to their current average life satisfaction score of 2.21” IDInsight 2019, pg 94
I’m not sharing the full reasoning because it was private correspondence and I haven’t asked the authors if they’d be comfortable with me sharing.
“Other wellbeing researchers, such as Diener et al. (2018), appear to treat the midway point on the scale as the neutral point (i.e., 5 on a 0-10 scale).” Elephant in the bednet
“Although what we might call strong Epicureanism, the view that death is not bad at all, has few takers, there may be more sympathy for weak Epicureanism, where death can be bad, but relatively more weight is given to living well than living long” Elephant in the bednet
On 3. Epicureanism being a defensible position
Epicureanism is discussed in almost every philosophy course on the badness of death. It’s taken seriously, rather than treated as an absurd position, a non-starter, and whilst not that many philosophers end up as Epicureans, I’ve met some that are very sympathetic. I find critics dismiss the view too quickly and I’ve not seen anything that’s convinced me the view has no merit. I don’t think we should have zero credence in it, and it seems reasonable to point out that it is one of the options. Again, I’m inclined to let donors make up their own minds.
On what HLI actually believes
HLI is currently trying not to have a view on these issues, but point out to donors how having different views would change the priorities so they can form their own view. We may have to develop a ‘house view’ but none of the options for doing this seem particularly appealing (they include: we use my view, we use a staff aggregate, we poll donors, we poll the public, some combo of the previous options).
You bring up this quote
I regret this sentence, which is insufficiently nuanced and I wouldn’t use it again (you and I have discussed this privately). That said, I think we’re quite well-caveated elsewhere. You quote this bullet point:
But you didn’t quote the bullet point directly before it (emphasis added):
The backstory to the “we confidently recommend StrongMinds” bit is that, when we did the analysis, StrongMinds looked better under almost all assumptions and, even where AMF was better, it was only slightly better (1.3x). We thought donors would want an overall recommendation, and hence StrongMinds seemed like the safe choice (given some intuitions about donors’ intuitions and moral uncertainty). You’re right that we’ll have to rethink what our overall recommendations are, and how to frame them, once the dust has settled on this debate.
Finally, whilst you say
This feels uneasily like a double standard. As I’ve pointed out before, neither GiveWell nor Open Philanthropy really defends their views in general (asserting a view isn’t the same as defending it). In this report, GiveWell doesn’t defend its assumptions, point out what other assumptions one might (reasonably) take, or say how this would change the result. Part of what we have tried to highlight in our work is that these issues have been mostly ignored and can really matter.
Our aim was more to cover the range of views we think some reasonable people would believe, not to restrict it to what we think they should believe. We motivated our choices in the original report and will restate that briefly here. For the badness of death, we give the three standard views in the literature. At one end, deprivationism gives ‘full value’ to saving lives. On the other, Epicurianism gives no weight to saving lives. TRIA offers something in between. For the neutral point, we used a range that included what we saw as the minimum and maximum possible values. Including a range of values is not equivalent to saying they are all equally probable. We encourage donors and decision-makers to use values they think are most plausible (for example, by using this interactive chart).
In suggesting James quote these together, it sounds like you’re saying something like “this is a clear caveat to the strength of recommendation behind StrongMinds, HLI doesn’t recommend StrongMinds as strongly as the individual bullet implies, it’s misleading for you to not include this”.
But in other places HLI’s communication around this takes on a framing of something closer to “The cost effectiveness of AMF, (but not StrongMinds) varies greatly under these assumptions. But the vast majority of this large range falls below the cost effectiveness of StrongMinds”. (extracted quotes in footnote)[1]
As a result of this framing, despite the caveat that HLI “[does] not advocate for any particular view”, I think it’s reasonable to interpret this as being strongly supportive of StrongMinds, which can be true even if HLI does not have a formed view on the exact philosophical view to take.[2]
If you did mean the former (that the bullet about philosophical assumptions is primarily included as a caveat to the strength of recommendation behind StrongMinds), then there is probably some tension here between (emphasis added):
-”the relative value of life-extending and life-improving interventions depends very heavily on the philosophical assumptions you make...there is no simple answer”, and
-”We conclude StrongMinds > AMF under almost all assumptions”
Additionally I think some weak evidence to suggest that HLI is not as well-caveated as it could be is that many people (mistakenly) viewed HLI as an advocacy organisation for mental health interventions. I do think this is a reasonable outside interpretation based on HLI’s communications, even though this is not HLI’s stated intent. For example, I don’t think it would be unreasonable for an outsider to read your current pinned thread and come away with conclusions like:
“StrongMinds is the best place to donate”,
“StrongMinds is better than AMF”,
“Mental health is a very good place to donate if you want to do the most good”,
“Happiness is what ultimately matters for wellbeing and what should be measured”.
If these are not what you want people to take away, then I think pointing to this bullet point caveat doesn’t really meaningfully address this concern—the response kind of feels something like “you should have read the fine print”. While I don’t think it’s not necessary for HLI to take a stance on specific philosophical views, I do think it becomes an issue if people are (mis)interpreting HLI’s stance based on its published statements.
(commenting in personal capacity etc)
As you’ve acknowledged, comments like “We’re now in a position to confidently recommend StrongMinds as the most effective way we know of to help other people with your money.” perhaps add to the confusion.
Do you think the neutral point and basic philosophical perspective (e.g., deprivationism vs. epicureanism) are empirical questions, or are they matters on which the donor has to exercise their own moral and philosophical judgment (after considering what the somewhat limited survey data have to say on the topic)?
I would graph the neutral point from 0 to 3. I think very few donors would set the neutral point above 3, and I’d start with the presumption that the most balanced way to present the chart is probably to center it fairly near the best guess from the survey data. On the other hand, if you have most of the surveys reporting “about 2,” then it’s hard to characterize 3 as an outlier view—presumably, a good fraction of the respondents picked a value near, at, or even over 3.
Although I don’t think HLI puts it this way, it doesn’t strike me as implausible to view human suffering as a more severe problem than lost human happiness. As I noted in a different comment, I think of that chart as a starting point from which a donor can apply various discounts and bonuses on a number of potentially relevant factors. But another way to account for this would be to give partial weight to strong epicureanism as a means of discounting the value of lost human happiness vis-a-vis suffering.
Given that your critique was published after HLI’s 2022 charity recommendation, I think it’s fair to ask HLI whether it would reaffirm those characterizations today. I would agree that the appropriate conclusion, on HLI’s current state of analysis, is that the recommendation is either SM or GiveWell’s top charities depending on the donor’s philosophical assumptions. I don’t think it’s inappropriate to make a recommendation based on the charity evaluator’s own philosophical judgment, but unless HLI has changed its stance it has taken no position. I don’t think it is appropriate to merely assume equal credence for each of the philosophical views and neutral points under consideration.
One could also defensibly make a summary recommendation on stated assumptions about donor values or on receipient values. But the best information I’ve seen on those points—the donor and beneficiary surveys as reflected in GiveWell’s moral weights—seemingly points to a predominately deprivationist approach with a pretty low neutral point (otherwise the extremely high value on saving the lives of young children wouldn’t compute).
Thanks Jason, mostly agree with paras 4-5, and think para 2 is a good point as well.
I think the basic philosophical perspective is a moral/philosophical judgement. But the neutral point combines that moral judgement with empirical models of what peoples’ lives are actually like, and empirical beliefs about how people respond to surveys.
I wonder if, insofar as we do have different perspectives on this (and I don’t think we’re particularly far apart, particularly on the object level question), the crux is around how much weight to put in individual donor judgement? Or even how much individual donors have those judgements?
My experience of even EA-minded (or at least GiveWell) donors is that ~none of them have a position on these kinds of questions, and they actively want to defer. My (less confident but based on quite a few conversations) model of EA-minded StrongMinds donors is they want to give to mental health and see an EA-approved charity so give there, rather than because of a quantitative belief on foundational questions like the neutral point. As an aside, I believe that was how StrongMinds first got on EA’s radar—as a recommendation for Founders Pledge donors who specifically wanted to give to mental health in an evidence-based way.
It does seem plausible to me that donors who follow HLI recommendations (who I expect are particularly philosophically minded) would be more willing to change their decisions based on these kinds of questions than donors I’ve talked to.
I’d be interested if someone wanted to stick up for a neutral point of 3 as something they actually believe and a crux for where they give, rather than something someone could believe, or is plausible. I could be wrong, but I’m starting out skeptical that belief would survive contact with “But that implies the world would be better if everyone in Afghanistan died” and “a representative survey of people whose deaths you’d be preventing think their lives are more valuable than that”
What do you think?
From HLI’s perspective, it makes sense to describe how the moral/philosophical views one assumes affect the relative effectiveness of charities. They are, after all, a charity recommender, and donors are their “clients” in a sense. GiveWell doesn’t really do this, which makes sense—GiveWell’s moral weights are so weighted toward saving lives that it doesn’t really make sense for them to investigate charities with other modes of action. I think it’s fine to provide a bottom-line recommendation on whatever moral/philosophical view a recommender feels is best-supported, but it’s hardly obligatory.
We recognize donor preferences in that we don’t create a grand theory of effectiveness and push everyone to donate to longtermist organizations, or animal-welfare organizations, or global health organizations depending on the grand theory’s output. Donors choose among these for their own idiosyncratic reasons, but moral/philosophical views are certainly among the critical criteria for many donors. I don’t see why that shouldn’t be the case for interventions within a cause area that produce different kinds of outputs as well.
Here, I doubt most global-health donors—either those who take advice from GiveWell or from HLI—have finely-tuned views on deprivationism, neutral points, and so on. However, I think many donors do have preferences that indirectly track on some of those issues. For instance, you describe a class of donors who “want to give to mental health.” While there could be various reasons for that, it’s plausible to me that these donors place more of an emphasis on improving experience for those who are alive (e.g., they give partial credence to epicureanism) and/or on alleviating suffering. If they did assess and chart their views on neutral point and philosophical view, I would expect them to end up more often at points where SM is ranked relatively higher than the average global-health donor would. But that is just conjecture on my part.
One interesting aspect of thinking from the donor perspective is the possibility that survey results could be significantly affected by religious beliefs. If many respondents chose a 0 neutral point because their religious tradition led them to that conclusion, and you are quite convinced that the religious tradition is just wrong in general, do you adjust for that? Does not adjusting allow the religious tradition to indirectly influence where you spend your charitable dollar?
To me, the most important thing a charity evaluator/recommender does is clearly communicate what the donation accomplishes (on average) if given to various organizations they identify—X lives saved (and smaller benefits), or Y number of people’s well-being improved by Z amount. That’s the part the donor can’t do themselves (without investing a ton of time and resources).
I don’t think the neutral point is as high as 3. But I think it’s fine for HLI to offer recommendations for people who do.
Hi James, thanks for elaborating, that’s really useful! We’ll reply to your points in separate comments.
Your statement, 1. I don’t think a neutral point higher than 2 is defensible
Reply: I don’t think we have enough evidence or theory to be confident about where to put the neutral point.
Your response about where to put the neutral point involves taking answers to survey questions where people are asked something like “where on a 0-10 scale would you choose not to keep living?” and assuming we should take those answers at face value for where to locate the neutral point. However, this conclusion strikes me as too fast; I don’t think we have enough theory or evidence on this issue. Are we definitely asking the right questions? Do we understand people’s responses? Should we agree with them even if we understand them?
I’m not sure if I told you about this, but we’re working on a pilot survey for this and other wellbeing measuring issues. The relevant sections for neutrality are 1.3 and 6. I’ll try to put the main bits here, to make life easier (link to initial part of report on EA forum; link to full report):
As we say in footnote 5, this aligned is straightforwardly entailed by the standard formulation of utilitarianism.
That’s why,
Unfortunately, we find that, on a life satisfaction scale, participants put the zero point at 5⁄10, and the neutral point at 1.3/10. We’re not really sure what to make of this. Here’s what we say in section 6.2 in full.
We plan to think about this more and test our hypotheses in the full version of the survey. If you have ideas for what we should test, now would be a great time to share them!
Given the methodological challenges in measuring the neutral point, I would have some hesitation to credit any conclusions that diverged too much from what revealed preferences imply. A high neutral point implies that many people in developing countries believe their lives are not worth living. So I’d look for evidence of behavior (either in respondents or in the population more generally) that corroborated whether people acted in a way that was consistent with the candidate neutral point.
For instance, although money, family, and other considerations doubtless affect it, studying individuals who are faced with serious and permanent (or terminal) medical conditions might be helpful. At what expected life satisfaction score do they decline treatment? If the neutral point is relatively close to the median point in a country, one would expect to see a lot of people decide to not obtain curative treatment if the results would leave them 1-2 points less satisfied than their baseline.
You might be able to approximate that by asking hypothetical questions about specific situations that you believe respondents would assess as reducing life satisfaction by a specified amount (disability, imprisonment, social stigma, etc.), and then ask whether the respondent believes they would find life still worth living if that happened. I don’t think that approach works to establish a neutral point, but I think having something more concrete would be an important cross-check on what may otherwise come across as an academic, conjectural exercise to many respondents.
This isn’t necessarily the case. I assume that if people described their lives as having negative wellbeing, this wouldn’t imply they thought their life was not worth continuing.
People can have negative wellbeing and still want to live for the sake of others or causes greater than themselves.
Life satisfaction appears to be increasing over time in low income countries. I think this progress is such that many people who may have negative wellbeing at present, will not have negative wellbeing their whole lives.
Edit: To expand a little, for these reasons, as well as the very reasonable drive to survive (regardless of wellbeing), I find it difficult to interpret revealed preferences and it’s unclear they’re a bastion of clarity in this confusing debate.
Anectdotally, I’ve clearly had periods of negative wellbeing before (sometimes starkly), but never wanted to die during those periods. If I knew that such periods were permanent, I’d probably think it was good for me to not-exist, but I’d still hesitate to say I’d prefer to not-exist, because I don’t just care about my wellbeing. As Tyrion said “Death is so final, and life is so full of possibilities.”
I think these difficulties should highlight that the difficulties here aren’t just localized to this area of the topic.
Thanks for these points! The idea that people care about more than their wellbeing may be critical here. I’m thinking of a simplified model with the following assumptions: a mean lifetime wellbeing of 5, SD 2, normal distribution, wellbeing is constant through the lifespan, with a neutral point of 4 (which is shared by everyone).
Under these assumptions, AMF gets no “credit” (except for grief avoidance) for saving the life of a hypothetical person with wellbeing of 4. I’m really hesitant to say that saving that person’s life doesn’t morally “count” as a good because they are at the neutral point. On the one hand, the model tells me that saving this person’s life doesn’t improve total wellbeing. On the other hand, suppose I (figuratively) asked the person whose life was saved, and he said that he preferred his existence to non-existence and appreciated AMF saving his life.
At that point, I think the WELLBY-based model might not be incorporating some important data—the person telling us that he prefers his existence to non-existence would strongly suggest that saving his life had moral value that should indeed “count” as a moral good in the AMF column. His answers may not be fully consistent, but it’s not obvious to me why I should fully credit his self-reported wellbeing but give zero credence to his view on the desirability of his continued existence. I guess he could be wrong to prefer his continued existence, but he is uniquely qualified to answer that question and so I think I should be really hesitant to completely discount what he says. And a full 30% of the population would have wellbeing of 4 or less under the assumptions.
Even more concerning, AMF gets significantly “penalized” for saving the life of a hypothetical person with wellbeing of 3 who also prefers existence to non-existence. And almost 16% of the population would score at least that low.
Of course, the real world is messier than a quick model. But if you have a population where the neutral point is close enough to the population average, but almost everyone prefers continued existence, it seems that you are going to have a meaningful number of cases where AMF gets very little / no / negative moral “credit” for saving the lives of people who want (or would want) their lives saved. That seems like a weakness, not a feature, of the WELLBY-based model to me.
I could have been clearer, the 38% is a placeholder while I do the Barker et al. 2022 analysis. You did update me about the previous studies’ relevance. My arguments are less supporting the 38% figure—which I expect to update with more data and more about explaining why I think that I have a higher prior for household spillovers from psychotherapy than you and Alex seem to. But really, the hope is that we can soon be discussing more and better evidence.