I guess if I was trying to make an argument that we should be worried with minimal assumptions, I’d argue as follows:
AI will lead to the creation of dangerous capacities
The only possible defence against AI systems will be other AI systems acting with a substantial degree of autonomy. If these systems malfunction or turn against us we will be screwed.
AI arms races will force us to deploy these systems fast and without proper testing. This probably results in us being screwed.
Any given alignment technique is likely to break under a sufficient amount of optimisation pressure. Since we won’t have much time to develop new techniques on the fly, we are likely screwed again if we haven’t developed techniques for aligning powerful systems ahead of time.
Thank you. I quite like the “we don’t have a lot of time” part, both in the fact that we’d need to prepare in advance, and because making decisions under time pressure is almost always worse.
I have a framing that you might find interesting:
I guess if I was trying to make an argument that we should be worried with minimal assumptions, I’d argue as follows:
AI will lead to the creation of dangerous capacities
The only possible defence against AI systems will be other AI systems acting with a substantial degree of autonomy. If these systems malfunction or turn against us we will be screwed.
AI arms races will force us to deploy these systems fast and without proper testing. This probably results in us being screwed.
Any given alignment technique is likely to break under a sufficient amount of optimisation pressure. Since we won’t have much time to develop new techniques on the fly, we are likely screwed again if we haven’t developed techniques for aligning powerful systems ahead of time.
Thank you.
I quite like the “we don’t have a lot of time” part, both in the fact that we’d need to prepare in advance, and because making decisions under time pressure is almost always worse.