At the time I thought I was explaining [Pascal’s mugging] badly but reading more on this topic I think it is just a non-problem: it only appears to be a problem to those whose only decision making tool is an expected value calculation.
This is quite a strong claim IMO. Could you explain exactly which other decision making tool(s) you would apply to Pascal’s mugging that makes it not a problem? The descriptions of the tools in stories 1 and 2 are too vague for me to clearly see how they’d apply here.
Indeed, if anything, some of those tools strengthen the case for giving into Pascal’s mugging. E.g. “developing a set of internally consistent descriptions of future events based on each uncertainty, then developing plans that are robust to all options”: if you can’t reasonably rule out the possibility that the mugger is telling the truth, paying the mugger seems a lot more robust. Ruling out that possibility in the literal thought experiment doesn’t seem obviously counterintuitive to me, but the standard stories for x- and s-risks don’t seem so absurd that you can treat them as probability 0 (more on this below). Appealing to the possibility that one’s model is just wrong, which does cut against naive EV calculations, doesn’t seem to help here.
I can imagine a few candidates, but none seem satisfactory to me:
“Very small probabilities should just be rounded down to zero.” I can’t think of a principled basis for selecting the threshold for a “very small” probability, at least not one that doesn’t subject us to absurd conclusions like that you shouldn’t wear a seatbelt because probabilities of car crashes are very low. This rule also seems contrary to maximin robustness.
“Very high disutilities are practically impossible.” I simply don’t see sufficiently strong evidence in favor of this to outweigh the high disutility conditional on the mugger telling the truth. If you want to say my reply is just smuggling expected value reasoning in through the backdoor, well, I don’t really consider this a counterargument. Declaring a hard rule like this one, which treats some outcomes as impossible absent a mathematical or logical argument, seems epistemically hubristic and is again contrary to robustness.
“Don’t do anything that extremely violates common sense.” Intuitive, but I don’t think we should expect our common sense to be well-equipped to handle situations involving massive absolute values of (dis)utility.
Let us think through the range of options for addressing Pascal’s mugging. There are basically 3 options:
A: Bite the bullet – if anyone threatens to do cause infinite suffering then do whatever they say.
B: Try to fix your expected value calculations to remove your problem.
C: Take an alternative approach to decision making that does not rely on expected value.
It is also possible that all of A and B and C fail for different reasons.*
Let’s run through.
A:
I think that in practice no one does A. If I email everyone in the EA/longtermism community and say: I am an evil wizard please give me $100 or I will cause infinite suffering! I doubt I will get any takers.
B:
You made three suggestions for addressing Pascal’s mugging. I think I would characterise suggestions 1 and 2 as ways of adjusting your expected value calculations to aim for more accurate expected value estimates (not as using an alternate decision making tool).
I think it would be very difficult to make this work, as it leads to problems such as the ones you highlight.
You could maybe make this work using a high discounting based on the “optimisers curse” type factors to reduce the expected value of high-uncertainty high-value decisions. I am not sure.
(The GPI paper on cluelessness basically says that expected value calculations can never work to solve this problem. It is plausible you could write a similar paper about Pascals mugging. It might be interesting to read the GPI paper and mentally replace “problem of clulessness” with “problem of pascals mugging” and see how it reads).
C:
I do think you could make your third option, the common sense version, work. You just say: if I follow this decision it will lead to very perverse circumstances, such as me having to give everything I own to anyone who claims they will otherwise me cause infinite suffering. It seems so counter-intuitive that I should do this that I will decide not to do this. I think this is roughly the approach that most people follow in practice. This is similar to how you might dismiss this proof that 1+1=3 even if you cannot see the error. It is however a bit of a dissatisfying answer as it is not very rigorous, it is unclear when a conclusion is so absurd as to require outright objection.
It does seem hard to apply most of the DMDU approaches to this problem. An assumption based modeling approach would lead to you writing out all of your assumptions and looking for flaws – I am not sure where it would lead.
if looking for an more rigorous approach the flexible risk planning approach might be useful. Basically make the assumption that: when uncertainty goes up the ability to pinpoint the exact nature of the risk goes down. (I think you can investigate this empirically). So placing a reasonable expected value on a highly uncertain event means that in reality events vaguely of that type are more likely but events specifically as predicted are themselves unlikely. For example you could worry about future weapons technology that could destroy the world and try to explore what this would look like – but you can safely say it is very unlikely to look like your explorations. This might allow you to avoid the pascal mugger and invest appropriate time into more general more flexible evil wizard protection.
Does that help?
* I worry that I have made this work by defining C as everything else and that the above is just saying Paradox → No clear solution → Everything else must be the solutions.
This is true, but we could all be mistaken. This doesn’t seem unlikely to me, considering that our brains simply were not built to handle such incredibly small probabilities and incredibly large magnitudes of disutility. That said, I won’t practically bite the bullet, any more than people who would choose torture over dust specks probably do, or any more than pure impartial consequentialists truly sacrifice all their own frivolities for altruism. (This latter case is often excused as just avoiding burnout, but I seriously doubt the level of self-indulgence of the average consequentialist EA, myself included, is anywhere close to altruistically optimal.)
In general—and this is something I seem to disagree with many in this community about—I think following your ethics or decision theory through to its honest conclusions tends to make more sense than assuming the status quo is probably close to optimal. There is of course some reflective equilibrium involved here; sometimes I do revise my understanding of the ethical/decision theory.
This is similar to how you might dismiss this proof that 1+1=3 even if you cannot see the error.
To the extent that I assign nonzero probability to mathematically absurd statements (based on precedents like these), I don’t think there’s very high disutility in acting as if 1+1=2 in a world where it’s actually true that 1+1=3. But that could be a failure of my imagination.
It is however a bit of a dissatisfying answer as it is not very rigorous, it is unclear when a conclusion is so absurd as to require outright objection.
This is basically my response. I think there’s some meaningful distinction between good applications of reductio ad absurdum and relatively hollow appeals to “common sense,” though, and the dismissal of Pascal’s mugging strikes me as more the latter.
For example you could worry about future weapons technology that could destroy the world and try to explore what this would look like – but you can safely say it is very unlikely to look like your explorations.
I’m not sure I follow how this helps. People who accept giving into Pascal’s mugger don’t dispute that the very bad scenario in question is “very unlikely.”
This might allow you to avoid the pascal mugger and invest appropriate time into more general more flexible evil wizard protection.
I think you might be onto something here, but I’d need the details fleshed out because I don’t quite understand the claim.
This is quite a strong claim IMO. Could you explain exactly which other decision making tool(s) you would apply to Pascal’s mugging that makes it not a problem? The descriptions of the tools in stories 1 and 2 are too vague for me to clearly see how they’d apply here.
Indeed, if anything, some of those tools strengthen the case for giving into Pascal’s mugging. E.g. “developing a set of internally consistent descriptions of future events based on each uncertainty, then developing plans that are robust to all options”: if you can’t reasonably rule out the possibility that the mugger is telling the truth, paying the mugger seems a lot more robust. Ruling out that possibility in the literal thought experiment doesn’t seem obviously counterintuitive to me, but the standard stories for x- and s-risks don’t seem so absurd that you can treat them as probability 0 (more on this below). Appealing to the possibility that one’s model is just wrong, which does cut against naive EV calculations, doesn’t seem to help here.
I can imagine a few candidates, but none seem satisfactory to me:
“Very small probabilities should just be rounded down to zero.” I can’t think of a principled basis for selecting the threshold for a “very small” probability, at least not one that doesn’t subject us to absurd conclusions like that you shouldn’t wear a seatbelt because probabilities of car crashes are very low. This rule also seems contrary to maximin robustness.
“Very high disutilities are practically impossible.” I simply don’t see sufficiently strong evidence in favor of this to outweigh the high disutility conditional on the mugger telling the truth. If you want to say my reply is just smuggling expected value reasoning in through the backdoor, well, I don’t really consider this a counterargument. Declaring a hard rule like this one, which treats some outcomes as impossible absent a mathematical or logical argument, seems epistemically hubristic and is again contrary to robustness.
“Don’t do anything that extremely violates common sense.” Intuitive, but I don’t think we should expect our common sense to be well-equipped to handle situations involving massive absolute values of (dis)utility.
This is a fascinating question – thank you.
Let us think through the range of options for addressing Pascal’s mugging. There are basically 3 options:
A: Bite the bullet – if anyone threatens to do cause infinite suffering then do whatever they say.
B: Try to fix your expected value calculations to remove your problem.
C: Take an alternative approach to decision making that does not rely on expected value.
It is also possible that all of A and B and C fail for different reasons.*
Let’s run through.
A:
I think that in practice no one does A. If I email everyone in the EA/longtermism community and say: I am an evil wizard please give me $100 or I will cause infinite suffering! I doubt I will get any takers.
B:
You made three suggestions for addressing Pascal’s mugging. I think I would characterise suggestions 1 and 2 as ways of adjusting your expected value calculations to aim for more accurate expected value estimates (not as using an alternate decision making tool).
I think it would be very difficult to make this work, as it leads to problems such as the ones you highlight.
You could maybe make this work using a high discounting based on the “optimisers curse” type factors to reduce the expected value of high-uncertainty high-value decisions. I am not sure.
(The GPI paper on cluelessness basically says that expected value calculations can never work to solve this problem. It is plausible you could write a similar paper about Pascals mugging. It might be interesting to read the GPI paper and mentally replace “problem of clulessness” with “problem of pascals mugging” and see how it reads).
C:
I do think you could make your third option, the common sense version, work. You just say: if I follow this decision it will lead to very perverse circumstances, such as me having to give everything I own to anyone who claims they will otherwise me cause infinite suffering. It seems so counter-intuitive that I should do this that I will decide not to do this. I think this is roughly the approach that most people follow in practice. This is similar to how you might dismiss this proof that 1+1=3 even if you cannot see the error. It is however a bit of a dissatisfying answer as it is not very rigorous, it is unclear when a conclusion is so absurd as to require outright objection.
It does seem hard to apply most of the DMDU approaches to this problem. An assumption based modeling approach would lead to you writing out all of your assumptions and looking for flaws – I am not sure where it would lead.
if looking for an more rigorous approach the flexible risk planning approach might be useful. Basically make the assumption that: when uncertainty goes up the ability to pinpoint the exact nature of the risk goes down. (I think you can investigate this empirically). So placing a reasonable expected value on a highly uncertain event means that in reality events vaguely of that type are more likely but events specifically as predicted are themselves unlikely. For example you could worry about future weapons technology that could destroy the world and try to explore what this would look like – but you can safely say it is very unlikely to look like your explorations. This might allow you to avoid the pascal mugger and invest appropriate time into more general more flexible evil wizard protection.
Does that help?
* I worry that I have made this work by defining C as everything else and that the above is just saying Paradox → No clear solution → Everything else must be the solutions.
Thanks for your reply! :)
This is true, but we could all be mistaken. This doesn’t seem unlikely to me, considering that our brains simply were not built to handle such incredibly small probabilities and incredibly large magnitudes of disutility. That said, I won’t practically bite the bullet, any more than people who would choose torture over dust specks probably do, or any more than pure impartial consequentialists truly sacrifice all their own frivolities for altruism. (This latter case is often excused as just avoiding burnout, but I seriously doubt the level of self-indulgence of the average consequentialist EA, myself included, is anywhere close to altruistically optimal.)
In general—and this is something I seem to disagree with many in this community about—I think following your ethics or decision theory through to its honest conclusions tends to make more sense than assuming the status quo is probably close to optimal. There is of course some reflective equilibrium involved here; sometimes I do revise my understanding of the ethical/decision theory.
To the extent that I assign nonzero probability to mathematically absurd statements (based on precedents like these), I don’t think there’s very high disutility in acting as if 1+1=2 in a world where it’s actually true that 1+1=3. But that could be a failure of my imagination.
This is basically my response. I think there’s some meaningful distinction between good applications of reductio ad absurdum and relatively hollow appeals to “common sense,” though, and the dismissal of Pascal’s mugging strikes me as more the latter.
I’m not sure I follow how this helps. People who accept giving into Pascal’s mugger don’t dispute that the very bad scenario in question is “very unlikely.”
I think you might be onto something here, but I’d need the details fleshed out because I don’t quite understand the claim.