I get what you’re saying, but, e.g., in the recent profile of Nick Bostrom in the New Yorker:
No matter how improbable extinction may be, Bostrom argues, its consequences are near-infinitely bad; thus, even the tiniest step toward reducing the chance that it will happen is near-infinitely valuable. At times, he uses arithmetical sketches to illustrate this point. Imagining one of his utopian scenarios—trillions of digital minds thriving across the cosmos—he reasons that, if there is even a one-per-cent chance of this happening, the expected value of reducing an existential threat by a billionth of a billionth of one per cent would be worth a hundred billion times the value of a billion present-day lives. Put more simply: he believes that his work could dwarf the moral importance of anything else.
While the most prominent advocate in the respectable-academic part of that side of the debate is making Pascal-like arguments, there’s going to be some pushback about Pascal’s mugging.
i) I bet Bostrom thinks the odds of a collective AI safety effort of achieving its goal is better than 1%, which would itself be enough to avoid the Pascal’s Mugging situation.
ii) This is a fallback position from which you can defend the work if someone thinks it almost certainly won’t work. I don’t think we should do that, instead we should argue that we can likely solve the problem. But I see the temptation.
iii) I don’t think it’s clear you should always reject a Pascal’s Mugging (or if you should, it may only be because there are more promising options for enormous returns than giving it to the mugger).
I get what you’re saying, but, e.g., in the recent profile of Nick Bostrom in the New Yorker:
While the most prominent advocate in the respectable-academic part of that side of the debate is making Pascal-like arguments, there’s going to be some pushback about Pascal’s mugging.
I’ve also seen Eliezer (the person who came up with the term Pascal’s mugging) give talks where he explicitly disavows this argument.
Two things:
i) I bet Bostrom thinks the odds of a collective AI safety effort of achieving its goal is better than 1%, which would itself be enough to avoid the Pascal’s Mugging situation.
ii) This is a fallback position from which you can defend the work if someone thinks it almost certainly won’t work. I don’t think we should do that, instead we should argue that we can likely solve the problem. But I see the temptation.
iii) I don’t think it’s clear you should always reject a Pascal’s Mugging (or if you should, it may only be because there are more promising options for enormous returns than giving it to the mugger).