MichaelA🔸 comments on Why those who care about catastrophic and existential risk should care about autonomous weapons

[MichaelA🔸]

And doing so would set a precedent for avoiding a race by recognizing that even each participant’s interests are better served by at least some coordination and cooperation.

I find this sentence confusing. My impression is that a big part of the problem with arms races (or similar things) is precisely that they can occur even if all participants would (in expectation) be better served by everyone coordinating and cooperating, and all participants know this. And the key problem is creating a mechanism by which that coordination/​cooperation can arise and be stable. This would be similar to prisoner’s dilemmas, which I believe arms races are often modelled as. And if that’s the case, then people just recognising that they’ve be better off if everyone cooperated wouldn’t actually fix the problem—they can recognise that and still fail to cooperate.

Or are you suggesting that each participant is better off unilaterally switching into cooperative mode, even if no one else does so? At first glance, that seems like a strong claim? (But this isn’t an area I know a lot about.)

[aaguirre]

Thanks for your comments! I’ve put a few replies, here and elsewhere.

Apologies for writing unclearly here. I did not mean to imply that

each participant is better off unilaterally switching into cooperative mode, even if no one else does so?

Instead I agree that

the key problem is creating a mechanism by which that coordination/​cooperation can arise and be stable.