I think the same sheltering happens if you talk about ignoring small probabilities, even if the probability of the x-risk is in fact extremely small.
The probability that $3000 to AMF saves a life is significant. But the probability that it saves the life of any one particular individual is extremely low. We can divide up the possibility space any number of ways. To me it seems like this is a pretty damning problem for the idea of ignoring small probabilities.
We can say that the outcome of the AMF donation has lower variance than the outcome of an x-risk donation, assuming equal EV. So we could talk about preferring low variance, or being averse to having no impact. But I don’t know if that will seem as intuitively reasonable when we circle our new framework back to more everyday, tangible thought experiments.
Hmmm, good point: If we carve up the space of possibilities finely enough, then every possibility will have a too-low probability. So to make a “ignore small probabilities” solution work, we’d need to include some sort of rule for how to carve up the possibilities. And yeah, this seems like an unpromising way to go…
I think the best way to do it would be to say “We lump all possibilities together that have the same utility.” The resulting profile of dots would be like a hollow bullet or funnel. If we combined that with an “ignore all possibilities below probability p” rule, it would work. It would still have problems, of course.
I think the same sheltering happens if you talk about ignoring small probabilities, even if the probability of the x-risk is in fact extremely small.
The probability that $3000 to AMF saves a life is significant. But the probability that it saves the life of any one particular individual is extremely low. We can divide up the possibility space any number of ways. To me it seems like this is a pretty damning problem for the idea of ignoring small probabilities.
We can say that the outcome of the AMF donation has lower variance than the outcome of an x-risk donation, assuming equal EV. So we could talk about preferring low variance, or being averse to having no impact. But I don’t know if that will seem as intuitively reasonable when we circle our new framework back to more everyday, tangible thought experiments.
Hmmm, good point: If we carve up the space of possibilities finely enough, then every possibility will have a too-low probability. So to make a “ignore small probabilities” solution work, we’d need to include some sort of rule for how to carve up the possibilities. And yeah, this seems like an unpromising way to go…
I think the best way to do it would be to say “We lump all possibilities together that have the same utility.” The resulting profile of dots would be like a hollow bullet or funnel. If we combined that with an “ignore all possibilities below probability p” rule, it would work. It would still have problems, of course.