I’m not sure what you are saying here? Do you think there is a risk of AI companies deliberately causing s-risks (e.g. releasing a basilisk) if we don’t play nice!?
No, I didn’t mean anything like that (although such crazy unlikely risks might also be marginally better reduced through cooperation with these actors). I was simply suggesting that cooperation could be a more effective way to reduce risks of worst-case outcomes that might occur in the absence of cooperative work to prevent them, i.e. work of the directional kind gestured at in my other comment (e.g. because ensuring the inclusion of certain measures to avoid worst-case outcomes has higher EV than does work to slow down AI). Again, I’m not saying that this is definitely the case, but it could well be. It’s fairly unclear, in my view.
Ok. I don’t put much weight on s-risks being a likely outcome. Far more likely seems to be just that the solar system (and beyond) will be arranged in some (to us) arbitrary way, and all carbon-based life will be lost as collateral damage.
Although I guess if you are looking a bit nearer term, then s-risk from misuse could be quite high. But I don’t think any of the major players (OpenAI, Deepmind, Anthropic) are even really working on trying to prevent misuse at all as part of their strategy (their core AI Alignment work is on aligning the AIs, rather than the humans using them!) So actually, this is just another reason to shut it all down.
No, I didn’t mean anything like that (although such crazy unlikely risks might also be marginally better reduced through cooperation with these actors). I was simply suggesting that cooperation could be a more effective way to reduce risks of worst-case outcomes that might occur in the absence of cooperative work to prevent them, i.e. work of the directional kind gestured at in my other comment (e.g. because ensuring the inclusion of certain measures to avoid worst-case outcomes has higher EV than does work to slow down AI). Again, I’m not saying that this is definitely the case, but it could well be. It’s fairly unclear, in my view.
Ok. I don’t put much weight on s-risks being a likely outcome. Far more likely seems to be just that the solar system (and beyond) will be arranged in some (to us) arbitrary way, and all carbon-based life will be lost as collateral damage.
Although I guess if you are looking a bit nearer term, then s-risk from misuse could be quite high. But I don’t think any of the major players (OpenAI, Deepmind, Anthropic) are even really working on trying to prevent misuse at all as part of their strategy (their core AI Alignment work is on aligning the AIs, rather than the humans using them!) So actually, this is just another reason to shut it all down.