I am quite interested in your other arguments for why EV calculations won’t work for pascal’s mugging and why they might extend to x-risks. I would probably have prefered a post already including all the arguments for your case.
About the argument from hypothetical updates: My intuition is, that if you assign a probability of a lot more than 0.1^10^10^10 to the mugger actually being able to follow through this might create other problems (like probabilities of distinct events adding to something higher than 1 or priors inconsistent with occams razor). If that intuition (and your argument) was true (my intuition might very well be wrong and seems at least slightly influenced by motivated reasoning), one would basically have to conclude that bayesian EV reasoning fails as soon as it involves combinations of extreme utilities and miniscule probabilities.
However, i don’t think the credenced for being able to influence x-risks are so low, that updating becomes impossible and therefore i’m not convinced not to use EV to evaluate them by your first argument. I’m quite eager to see the other arguments, though.
Thanks! Yeah, sorry—I was thinking about putting it up all at once but decided against because that would make for a very long post. Maybe I should have anyway, so it’s all in one place.
Well, I don’t share your intuition, but I’d love to see it explored more. Maybe you can get an argument out of it. One way to try would be to try to find a class of at least 10^10^10^10 hypotheses that are at least as plausible as the Mugger’s story.
Interesting post!
I am quite interested in your other arguments for why EV calculations won’t work for pascal’s mugging and why they might extend to x-risks. I would probably have prefered a post already including all the arguments for your case.
About the argument from hypothetical updates: My intuition is, that if you assign a probability of a lot more than 0.1^10^10^10 to the mugger actually being able to follow through this might create other problems (like probabilities of distinct events adding to something higher than 1 or priors inconsistent with occams razor). If that intuition (and your argument) was true (my intuition might very well be wrong and seems at least slightly influenced by motivated reasoning), one would basically have to conclude that bayesian EV reasoning fails as soon as it involves combinations of extreme utilities and miniscule probabilities.
However, i don’t think the credenced for being able to influence x-risks are so low, that updating becomes impossible and therefore i’m not convinced not to use EV to evaluate them by your first argument. I’m quite eager to see the other arguments, though.
Thanks! Yeah, sorry—I was thinking about putting it up all at once but decided against because that would make for a very long post. Maybe I should have anyway, so it’s all in one place.
Well, I don’t share your intuition, but I’d love to see it explored more. Maybe you can get an argument out of it. One way to try would be to try to find a class of at least 10^10^10^10 hypotheses that are at least as plausible as the Mugger’s story.