I think there is very likely at some point going to be some sort of transition to a world where AIs are effectively in control. It seems worth it to slow down on the margin to try to shape this transition as best we can, especially slowing it down as we get closer to AGI and ASI. It would be surprising to me if making the transfer of power more voluntary/careful led to worse outcomes (or only led to slightly better outcomes such that the downsides of slowing down a bit made things worse).
Delaying the arrival of AGI by a few years as we get close to it seems good regardless of parameters like the value of involuntary-AI-disempowerment futures. But delaying the arrival by 100s of years seems more likely bad due to the tradeoff with other risks.
It would be surprising to me if making the transfer of power more voluntary/careful led to worse outcomes (or only led to slightly better outcomes such that the downsides of slowing down a bit made things worse).
Two questions here:
Why would accelerating AI make the transition less voluntary? (In my own mind, I’d be inclined to reverse this sentiment a bit: delaying AI by regulation generally involves forcibly stopping people from adopting AI. Force might be justified if it brings about a greater good, but that’s not the argument here.)
I can understand being “careful”. Being careful does seem like a good thing. But “being careful” generally trades off against other values in almost every domain I can think of, and there is such a thing as too much of a good thing. What reason is there to think that pushing for “more caution” is better on the margin compared to acceleration, especially considering society’s default response to AI in the absence of intervention?
So in the multi-agent slowly-replacing case, I’d argue that individual decisions don’t necessarily represent a voluntary decision on behalf of society (I’m imagining something like this scenario). In the misaligned power-seeking case, it seems obvious to me that this is involuntary. I agree that it technically could be a collective voluntary decision to hand over power more quickly, though (and in that case I’d be somewhat less against it).
I think emre’s comment lays out the intuitive case for being careful / taking your time, as does Ryan’s. I think the empirics are a bit messy once you take into account benefits of preventing other risks but I’d guess they come out in favor of delaying by at least a few years.
(edit: my point is basically the same as emre’s)
I think there is very likely at some point going to be some sort of transition to a world where AIs are effectively in control. It seems worth it to slow down on the margin to try to shape this transition as best we can, especially slowing it down as we get closer to AGI and ASI. It would be surprising to me if making the transfer of power more voluntary/careful led to worse outcomes (or only led to slightly better outcomes such that the downsides of slowing down a bit made things worse).
Delaying the arrival of AGI by a few years as we get close to it seems good regardless of parameters like the value of involuntary-AI-disempowerment futures. But delaying the arrival by 100s of years seems more likely bad due to the tradeoff with other risks.
Two questions here:
Why would accelerating AI make the transition less voluntary? (In my own mind, I’d be inclined to reverse this sentiment a bit: delaying AI by regulation generally involves forcibly stopping people from adopting AI. Force might be justified if it brings about a greater good, but that’s not the argument here.)
I can understand being “careful”. Being careful does seem like a good thing. But “being careful” generally trades off against other values in almost every domain I can think of, and there is such a thing as too much of a good thing. What reason is there to think that pushing for “more caution” is better on the margin compared to acceleration, especially considering society’s default response to AI in the absence of intervention?
So in the multi-agent slowly-replacing case, I’d argue that individual decisions don’t necessarily represent a voluntary decision on behalf of society (I’m imagining something like this scenario). In the misaligned power-seeking case, it seems obvious to me that this is involuntary. I agree that it technically could be a collective voluntary decision to hand over power more quickly, though (and in that case I’d be somewhat less against it).
I think emre’s comment lays out the intuitive case for being careful / taking your time, as does Ryan’s. I think the empirics are a bit messy once you take into account benefits of preventing other risks but I’d guess they come out in favor of delaying by at least a few years.