“So we are heading to a situation where the labs, governments and even the public understand the danger quite well”, I don’t think this is true? I would suspect even most AI researchers would give a poor answer if asked in detail what it would mean to have superintelligent AI systems, what kinds of behaviours such systems may be expected to show, and what dangers this poses. I could be wrong, but I feel like few people have really envisioned what a worst case scenario actually looks like.
If people did, statements like “strategic competition with China” would appear silly, a bit like saying I’m locked in a strategic competition to shoot myself in the face before my enemy does.
Most of the behaviour in the labs and government is driven by ego and power-seeking heuristics, not rational strategy.
“So we are heading to a situation where the labs, governments and even the public understand the danger quite well”, I don’t think this is true? I would suspect even most AI researchers would give a poor answer if asked in detail what it would mean to have superintelligent AI systems, what kinds of behaviours such systems may be expected to show, and what dangers this poses. I could be wrong, but I feel like few people have really envisioned what a worst case scenario actually looks like.
If people did, statements like “strategic competition with China” would appear silly, a bit like saying I’m locked in a strategic competition to shoot myself in the face before my enemy does.
Most of the behaviour in the labs and government is driven by ego and power-seeking heuristics, not rational strategy.