Pascal’s mugging

TagLast edit: 26 Oct 2022 22:05 UTC by Pablo

Pascal’s mugging is a thought experiment intended to raise a problem for expected value theory. Unlike Pascal’s wager, Pascal’s mugging does not involve infinite utilities or probabilities, so the problem it raises is separate from any of the known paradoxes of infinity.

The thought experiment

The thought experiment and its name first appeared in a blog post by Eliezer Yudkowsky.^[1] Nick Bostrom later elaborated it in the form of a fictional dialogue.^[2]

In Yudkowsky’s original formulation, a person is approached by a mugger who threatens to kill an astronomical number of people unless the person agrees to give them five dollars. Even a tiny probability assigned to the hypothesis that the mugger will deliver on his promise seems sufficient to make the prospect of giving the mugger five dollars better than the alternative, in expectation. The minuscule chance that the mugger is willing and able to save astronomically many people is more than compensated by the enormous value of what is at stake. (If one thinks the probability too low, the number of lives the mugger threatens to kill could be arbitrarily increased.) The thought experiment supposedly raises a problem for expected value theory because it seems intuitively absurd that we should give money to the mugger, yet this is what the theory apparently implies.

Responses

A variety of responses have been developed. One common response is to revise or reject expected value theory. A frequent revision is to ignore scenarios whose probability is below a certain threshold.

This response, however, has a number of problems. One problem is that the threshold seems arbitrary, regardless of where it is set. A critic could always say: “Why do you set the threshold at that value, rather than e.g. one order of magnitude higher or lower?” A more fundamental problem is that it seems that whether a scenario falls below or above a certain threshold is contingent on how the space of possibilities is carved up. For example, an existential risk of 1-in-100 per century can be redescribed as an existential risk of 1-in-5.2 billion per minute. If the threshold is set to a value between those two numbers, whether one should or should not ignore the risk will depend merely on how one describes it.

Another response is to adopt a prior that penalizes hypotheses in proportion to the number of people they imply we can affect. That is, one could adopt a view in which there is roughly a 1 in 10ⁿ chance that someone will have the power to affect 10ⁿ people. Given this penalty, the mugger can no longer resort to the expedient of increasing the number of people they threaten to kill in order to make the offer sufficiently attractive. As the number of people increases, the probability that they will be killed by the mugger decreases commensurately, and the expected value of their successive proposals remains the same.

A final response is to just “bite the bullet” and accept that if the mugger’s proposal is better in expectation, one should indeed give them the five dollars. This approach becomes more plausible when combined with a debunking explanation of the intuition that paying the mugger would be absurd. For example, one can argue that human brains cannot adequately represent very large or very small numbers, and that therefore intuitions triggered by thought experiments making use of such quantities are unreliable and should not be given much evidential weight.

Implications

Regardless of how one responds to Pascal’s mugging, it is important to note that it does not appear to affect the value assigned to “high-stakes” causes or interventions prioritized within the effective altruism community, such as AI safety research or other forms of existential risk mitigation. The case for working on these causes is not fundamentally different from more mundane arguments which do not plausibly fall under the scope of Pascal’s mugging, such as voting in an election.^[3]^[4]

It is also worth stressing that Pascal’s mugging involves both very high stakes and very small probabilities, but the term is sometimes incorrectly applied to cases involving high stakes, regardless of their probability.^[5]