Hi All
My first post here—please be kind!
I’ve recently been listening to and reading various arguments around EA topics that rely heavily on reasoning about a loosely defined notion of “expected value”. Coming from a mathematics background, and having no grounding at all in utilitarianism, this has left me wondering why expected value is used as the default decision rule in evaluating (speculative, future) effects of EA actions even when the probabilities involved are small (and uncertain).
I have seen a few people here refer to Pascal’s mugging, and I agree with that critique of EV. But it doesn’t seem to me you need to go anywhere near that far before things break down. Naively (?), I’d say that if you invest your resources in an action with a 1 in a million chance of saving a billion lives, and a 999,999 in a million chance of having no effect, then (the overwhelming majority of the time) you haven’t done anything at all. It only works if you can do the action many many times, and get an independent roll of your million-sided dice each time. To take an uncontroversial example, if the lottery was super-generous and paid out £100 trillion, but I had to work for 40 years to get one ticket, the EV of doing that might be, say, £10 million. But I still wouldn’t win. So I’d actually get nothing if I devoted my life to that. Right...?
I’m hoping the community here might be able to point me to something I could read, or just tell me why this isn’t a problem, and why I should be motivated by things with low probabilities of ever happening but high expected values.
If anyone feels like humoring me further, I also have some more basic/fundamental doubts which I expect are just things I don’t know about utilitarianism, but which often seem to be taken for granted. At the risk of looking very stupid, here are those:
Why does anyone think human happiness/wellbeing/flourishing/worth-living-ness is a numerical quantity? Based on subjective introspection about my own “utility” I can identify some different states I might be in, and a partial order of preference (I prefer to feel contented than to be in pain but I can’t rank all my possible states), but the idea that I could be, say 50% happier seems to have no meaning at all.
If we grant the first thing—that we have numerical values associated with our wellbeing—why does anyone expect there to be a single summary statistic (like a sum total, or an average) that can be used to combine everyone’s individual values to decide which of two possible worlds is better? There seems to be debate about whether “total utilitarianism” is right, or whether some other measure is better. But why should there be such a measure at all?
In practice, even if there is such a statistic, how does one use it? It’s hard to deny the obviousness of “two lives saved is better than one”, but as soon as you move to trying to compare unlike things it immediately feels much harder and non-obvious. How am I supposed to use “expected value” to actually compare, in the real world, certainly saving 3 lives with a 40% change of hugely improving the educational level of ten children (assuming these are the end outcomes—I’m not talking about whether educating the kids saves more lives later or something)? And then people want to talk about and compare values for whole future worlds many years hence—wow.
I have a few more I’m less certain about, but I’ll stop for now and see how this lands. Cheers for reading the above. I’ll be very grateful for explanations of why I’m talking nonsense, if you have the time and inclination!
I think NunoSempere’s answer is good and looking vNM utility should give you a clearer idea of where people are coming from in these discussions. I would also recommend the Stanford Encyclopedia of Philosophy’s article on expected utility theory. https://plato.stanford.edu/entries/rationality-normative-utility/
You make an important and often overlooked point about the Long-Run Arguments for expected utility theory (described in the article above). You might find Christian Tarsney’s paper, Exceeding Expectations, interesting and relevant. https://globalprioritiesinstitute.org/christian-tarsney-exceeding-expectations-stochastic-dominance-as-a-general-decision-theory/
On 3, this is a super hard practical difficulty that doesn’t have a satisfactory answer in many cases. Very relevant is Hilary Greaves’ Cluelessness. https://issuu.com/aristotelian.society/docs/greaves
As NunoSempere suggests, GiveWell is a good place to look for some tricky comparisons. My colleague, Stephen Clare, and I made this (very primitive!) attempt to compare saving human lives with sparing chickens from factory farms, which you might find interesting. https://forum.effectivealtruism.org/posts/ahr8k42ZMTvTmTdwm/how-good-is-the-humane-league-compared-to-the-against
Thank you very much—I’m part way through Christian Tarsney’s paper and definitely am finding it interesting. I’ll also have a go at Hilary Greaves piece. Listening to her on 80,000 hours’ podcast was one thing that contributed to asking this question. She seems (at least there) to accept EV as the obviously right decision criterion, but a podcast probably necessitates simplifying her views!
1 & 2 might be normally be answered by the Von Neumann–Morgenstern utility theorem*
In the case you mentioned, you can try to calculate the impact of an education throughout the beneficiaries’ lives. In this case, I’d expect it to mostly be an increase in future wages, but also some other positive externalities. Then you look at the willingness to trade time for money, or the willingness to trade years of life for money, or the goodness and badness of life at different earning levels, and you come up with a (very uncertain) comparison.
If you want to look at an example of this, you might want to look at GiveWell’s evaluations in general, or at their evaluation of deworming charities in particular.
I hope that’s enough to point you to some directions which might answer your questions.
* But e.g., for negative utilitarians, axiom’s 3 and 3′ wouldn’t apply in general (because they prefer to avoid suffering infinitely more than promoting happiness, i.e. consider L=some suffering, M=non-existence, N=some happiness) but they would still apply for the particular case where they’re trading-off between different quantities of suffering. In any case, even if negative utilitarians would represent the world with two points (total suffering, total happiness), they still have a way of comparing between possible worlds (choose the one with the least suffering, then the one with the most happiness if suffering is equal).
Thanks very much. I am going to spend some time thinking about the von-Neumann-Mortgenstern theorem. Despite my huge in-built bias towards believing things labelled “von-Neumann”, at an initial scan I found only one of the axioms (transitivity) felt obviously “true” to me about things like “how good is the whole world?”. They all seem true if actually playing games of chance for money of course, which seems to often be the model. But I intend to think about that harder.
On GiveWell, I think they’re doing an excellent job of trying to answer these questions. I guess I tend to get a bit stuck at the value-judgement level (e.g. how to decide what fraction of a human life a chicken life is worth). But it doesn’t matter much in practice because I can then fall back on a gut-level view and yet still choose a charity from their menu and be confident it’ll be pretty damn good.
Here is one answer to one part of your question.
I think the idea you are getting at here is you’d rather have a 10% chance of saving 10 people than a 1/million chance of saving a billion lives. I can definitely see why you would feel this way (and there are prudential reasons for it—e.g. worrying about being tricked), but if you were sure this was actually the maths, I think you should definitely go for the 1/million shot of saving the billion.
To see why, imagine yourself in the shoes of one of the 1,000,000,010 people whose lives are at risk in this scenario. If you go for the ‘safe’ option, you have a 10%*1/1,000,000,010 ~= one in a billion chance of being saved. In contrast, if you go for the ‘riskier’ option, you actually have a one in a million chance—over a thousand times better!
Hi Larks. Thanks for raising this way of re-framing the point. I think I still disagree, but it’s helpful to see this way of looking at it which I really hadn’t thought of. I still disagree because I am assuming I only get one chance at doing the action and personally I don’t value a 1 in a million chance of being saved higher than zero. I think if I know I’m not going to be faced with the same choice many times, it is better to save 10 people, than to let everyone die and then go around telling people I chose the higher expected value!
Would you be interested in selling me a lottery ticket? We can use an online random number generator. I will win with a one-in-a-million chance, in which case you will give me all your worldly possessions, including all your future income, and you swear to do my wishes in all things. I will pay you $0.01 for this lottery ticket.
If you really believed that one-in-a-million was the same as zero, this should be an attractive deal for you. But my guess is that actually you would not want to take it!
So here is something which sometimes breaks people: You’re saying that you prefer A = 10% chance of saving 10 people over B = 1 in a million chance of saving a billion lives. Do you still prefer a 10% chance of A over a 10% chance of B?
If you are, note how you can be Dutch-booked.
I won’t try to answer your three numbered points since they are more than a bit outside my wheelhouse + other people have already started to address them, but I will mention a few things about your preface to that (e.g., Pascal’s mugging).
I was a bit surprised to not see a mention of the so-called Petersburg Paradox, since that posed the most longstanding challenge to my understanding of expected value. The major takeaways I’ve had for dealing with both the Petersburg Paradox and Pascal’s mugging (more specifically, “why is it that this supposedly accurate decision theory rule seems to lead me to make a clearly bad decision?”) are somewhat-interrelated and are as follows:
1. Non-linear valuation/utility: money should not be assumed to linearly translate to utility, meaning that as your numerical winnings reach massive numbers you typically will see massive drops in marginal utility. This by itself should mostly address the issue with the lottery choice you mentioned: the “expected payoff/winnings” (in currency terms) is almost meaningless because it totally fails to reflect the expected value, which is probably miniscule/negative since getting $100 trillion likely does not make you that much happier than getting $1 trillion (for numerical illustration, let’s suppose 1000 utils vs. 995u), which itself likely is only slightly better than winning $100 billion (say, 950u) … and so on whereas it costs you 40 years if you don’t win (let’s suppose that’s like −100u).
2. Bounded bankrolling: with things like the Petersburg Paradox, my understanding is that the longer you play, the higher your average payoff tends to be. However, that might still be -$99 by the time you go bankrupt and literally starve to death, after which point you no longer can play.
3. Bounded payoff: in reality, you would expect that payoffs to be limited to some reasonable, finite amount. If we suppose that they are for whatever reason not limited, then that essentially “breaks the barrier” for other outcomes, which are the next point:
4. Countervailing cases: This is really crucial for bringing things together, yet I feel like it is consistently underappreciated. Take for example classic Pascal’s mugging-type situations, like “A strange-looking man in a suit walks up to you and says that he will warp up to his spaceship and detonate a super-mega nuke that will eradicate all life on earth if and only if you do not give him $50 (which you have in your wallet), but he will give you $3^^^3 tomorrow if and only if you give him $50.” We could technically/formally suppose the chance he is being honest is nonzero (e.g., 0.0000000001%), but still abide by rational expectation theory if you suppose that there are indistinguishably likely cases that cause the opposite expected value—for example, the possibility that he is telling you the exact opposite of what he will do if you give him the money (for comparison, see the philosopher God response to Pascal’s wager), or the possibility that the “true” mega-punisher/rewarder is actually just a block down the street and if you give your money to this random lunatic you won’t have the $50 to give to the true one (for comparison, see the “other religions” response to the narrow/Christianity-specific Pascal’s wager). Ultimately, this is the concept of fighting (imaginary) fire with (imaginary) fire, occasionally shows up in realms like competitive policy debate (where people make absurd arguments about how some random policy may lead to extinction), and is a major reason why I have a probability-trimming heuristic for these kinds of situations/hypotheticals.
Hi Harrison. I think I agree strongly with (2) and (3) here. I’d argue Infinite expected values that depend on (very) large numbers of trials / bankrolls etc. can and should be ignored. With the Petersburg Paradox as state in the link you included, making any vaguely reasonable assumption about the wealth of the casino, or lifetime of the player, the expected value falls to something much less appealing! This is kind of related to my “saving lives” example in my question—if you only get to play once, the expected value becomes basically irrelevant because the good outcome just actually doesn’t happen. It only starts to be worthwhile when you get to play many times. And hey, maybe you do. If there are 10,000 EAs all doing totally (probabilistically) independent things that each have a 1 in a million chance of some huge payoff, we start to get into realms worth thinking about.
Actually, I think it’s worth being a bit more careful about treating low-likelihood outcomes as irrelevant simply because you aren’t able to attempt to get that outcome more often: your intuition might be right, but you would likely be wrong in then concluding “expected utility/value theory is bunk.” Rather than throw out EV, you should figure out whether your intuition is recognizing something that your EV model is ignoring, and if so, figure out what that is. I listed a few example points above, to give another illustration:
Suppose you have a case where you have the chance to push button X or button Y once: if you push button X, there is a 1⁄10,000 chance that you will save 10,000,000 people from certain death (but a 9,999⁄10,000 chance that they will all still die); if you push button Y there is a 100% chance that 1 person will be saved (but 9,999,999 people will die). There are definitely some selfish reasons to choose button Y (e.g., you won’t feel guilty like if you pressed button X and everyone still died), and there may also be some aspect of non-linearity in the impact of how many people are dying (refer back to (1) in my original answer). However, if we assume away those other details (e.g., you won’t feel guilty, the deaths to utility loss is relatively linear) -- if we just assume the situation is “press button X for a 1⁄10,000 chance of 10,000,000 utils; press button Y for a 100% chance of 1 util” the answer is perhaps counterintuitive but still reasonable: without having a crystal ball that perfectly tells the future, the optimal strategy is to press button X.
Hey Ben, I think these are pretty reasonable questions and do not make you look stupid.
On Pascal’s mugging in particular, I would consider this somewhat informal answer: https://nintil.com/pascals-mugging/ Though honestly, I don’t find this super satisfactory, and it is something and still bugs me.
Having said that, I don’t think this line of reasoning is necessary for answering your more practical questions 1-3.
Utilitarianism (and Effective Altruism) don’t require that there’s some specific metaphysical construct that is numerical and corresponds to human happiness. The utilitarian claim is just that some degree of quantification is, in principle, possible. The EA claim is that attempting to carry out this quantification leads to good outcomes, even if it’s not an exact science.
GiveWell painstakingly compiles cost-effectiveness estimates numerical, but goes on to state that they don’t view these as being “literally true”. These estimates still end up being useful for comparing one charity relative to another. You can read more about this thinking here: https://blog.givewell.org/2017/06/01/how-givewell-uses-cost-effectiveness-analyses/
In practice, GiveWell makes all sorts of tradeoffs to attempt to compare goods like “improving education”, “lives saved” or “increasing income”. Sometimes this involves directly asking the targeted populations about their preferences. You can read more about their approach here: https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/2019-moral-weights-research
Finally, in the case of existential-risk, it’s often not necessary to make these kinds of specific calculations at all. By one estimate, the Earth alone could support something like 10^16 human lives, and the universe could support somewhere something like 10^34 human life-years, or up to 10^56 “cybernetic human life-years”. This is all very speculative, but the potential gains are so large that it doesn’t matter if we’re off by 40%, or 40x. https://en.wikipedia.org/wiki/Human_extinction#Ethics
Returning to the original point, you might ask if work on x-risk is then a case of Pascal’s Mugging? Toby Ord gives the odds of human extinction in the next century at around 1⁄6. That’s a pretty huge chance. We’re much less confident what the odds are of EA preventing this risk, but it seems reasonable to think that it’s some normal number. I.e. much higher than 10^-10. In that case, EA has huge expected value. Of course that might all seem like fuzzy reasoning, but I think there’s a pretty good case to be made that our odds are not astronomically low. You can see one version of this argument here: https://slatestarcodex.com/2015/08/12/stop-adding-zeroes/
On Pascal’s mugging specifically, Robert Miles has an interesting youtube video arguing that AI Safety is not a Pascal mugging, which the OP might be interested in:
Thank you—this is all very interesting. I won’t try to reply to all of it, but just thought I would respond to agree on your last point. I think x-risk is worth caring about precisely because the probability seems to be in the “actually might happen” range. (I don’t believe at all that anyone knows it’s 1⁄6 vs. 1⁄10 or 1⁄2, but Toby Ord doesn’t claim to either does he?) It’s when you get to the “1 in a million but with a billion payoff” range I start to get skeptical, because then the thing in question actually just won’t happen, barring many plays of the game.
The problem is real. Though for ‘normal’ low probabilities I suggest biting the bullet. A practical example is the question of whether to found a company. If you found a startup you will probably fail and make very little or no money. However, right now a majority of effective altruist funding comes from Facebook co-founder Dustin Moskovich. The tails are very thick.
If you have a high-risk plan with a sufficiently large reward I suggest going for it even if you are overwhelming likely to fail. Taking the risk is the most altruistic thing you can do. Most effective altruists are unwilling to take on the personal risk.