(I believe a lot of people get this wrong because they’re not thinking probabilistically. Someone has (say) a 10% P(doom) and a 10% chance of AGI within five years, and they round that off to “it’s not going to happen so we don’t need to worry yet.” A 10% chance is still really really bad.)
Yes! I’ve been saying this for a while, but most EAs still seem to be acting as if the median forecast is what is salient. If your regulation/alignment isn’t likely to be ready until the median forecasted date for AGI/TAI/ASI, then in half of all worlds you (we) don’t make it! When put like that, you can see that what seems like the “moderate” position is anything but—instead it is reckless in the extreme.
Yes! I’ve been saying this for a while, but most EAs still seem to be acting as if the median forecast is what is salient. If your regulation/alignment isn’t likely to be ready until the median forecasted date for AGI/TAI/ASI, then in half of all worlds you (we) don’t make it! When put like that, you can see that what seems like the “moderate” position is anything but—instead it is reckless in the extreme.