Agree with the post and the bottom line, though I don’t think it justifies focusing on AI safety, because of a disanalogy.
In your analogy, we assume that when we give the money to the mugger, they either make the coin more likely to land heads, or do nothing.
Meanwhile, in AI safety, small chances of averting doom come with small chances of causing doom—and it seems most of those who work in the field consider that some respected interventions are actually increasing P(Doom). They just disagree on what those doom-increasing interventions are.
Agree with the post and the bottom line, though I don’t think it justifies focusing on AI safety, because of a disanalogy.
In your analogy, we assume that when we give the money to the mugger, they either make the coin more likely to land heads, or do nothing.
Meanwhile, in AI safety, small chances of averting doom come with small chances of causing doom—and it seems most of those who work in the field consider that some respected interventions are actually increasing P(Doom). They just disagree on what those doom-increasing interventions are.
Yep, when going into AI safety you should take into account p(you cause doom) along with p(you avert doom).