But I am a bit at loss on why people in the AI safety field think it is possible to build safe AI systems in the first place. I guess as long as it is not proven that the properties of safe AI systems are contradictory with each other, you could assume it is theoretically possible. When it comes to ML, the best performance in practice is sadly often worse than the theoretical best.
To me, this belief that AI safety is hard or impossible would imply that AI x-risk is quite high. Then, I’d think that AI safety is very important but unfortunately intractable. Would you agree? Or maybe I misunderstood what you were trying to say.
I agree that x-risk from AI misuse is quite underexplored.
For context on the field’s current perspectives on these questions, a 2020 survey of AI safety and governance researchers (Clarke et al., 2021) found that, on average [1], researchers currently guess there is: [2]
A 10% chance of existential catastrophe from misaligned, influence-seeking AI [3]
A 6% chance of existential catastrophe from AI-exacerbated war or AI misuse
A 7% chance of existential catastrophe from “other scenarios”
I think you understood me in the same way than my friend did in the second part of the prolog, so I apparently give this impression. But to clarify, I am not certain that AI safety is impossible (I think it is hard, though), and the implications of that depend a lot on how much power the AI systems will be given at the end, and what part of the damage they might cause is due to them being unsafe and what for example misuse, like you said.
To me, this belief that AI safety is hard or impossible would imply that AI x-risk is quite high. Then, I’d think that AI safety is very important but unfortunately intractable. Would you agree? Or maybe I misunderstood what you were trying to say.
I agree that x-risk from AI misuse is quite underexplored.
For what it’s worth, AI safety and governance researchers do assign significant probability to x-risk from AI misuse. AI Governance Week 3 — Effective Altruism Cambridge comments:
I think you understood me in the same way than my friend did in the second part of the prolog, so I apparently give this impression. But to clarify, I am not certain that AI safety is impossible (I think it is hard, though), and the implications of that depend a lot on how much power the AI systems will be given at the end, and what part of the damage they might cause is due to them being unsafe and what for example misuse, like you said.