So in the multi-agent slowly-replacing case, I’d argue that individual decisions don’t necessarily represent a voluntary decision on behalf of society (I’m imagining something like this scenario). In the misaligned power-seeking case, it seems obvious to me that this is involuntary. I agree that it technically could be a collective voluntary decision to hand over power more quickly, though (and in that case I’d be somewhat less against it).
I think emre’s comment lays out the intuitive case for being careful / taking your time, as does Ryan’s. I think the empirics are a bit messy once you take into account benefits of preventing other risks but I’d guess they come out in favor of delaying by at least a few years.
So in the multi-agent slowly-replacing case, I’d argue that individual decisions don’t necessarily represent a voluntary decision on behalf of society (I’m imagining something like this scenario). In the misaligned power-seeking case, it seems obvious to me that this is involuntary. I agree that it technically could be a collective voluntary decision to hand over power more quickly, though (and in that case I’d be somewhat less against it).
I think emre’s comment lays out the intuitive case for being careful / taking your time, as does Ryan’s. I think the empirics are a bit messy once you take into account benefits of preventing other risks but I’d guess they come out in favor of delaying by at least a few years.