Presumably the programmer will make some effort to embed the right set of values in the AI. If this is an easy task, doom is probably not the default outcome.
AI pessimists have argued human values will be difficult to communicate due to their complexity. But as AI capabilities improve, AI systems get better at learning complex things.
Both the instrumental convergence thesis and the complexity of value thesis are key parts of the argument for AI pessimism as it’s commonly presented. Are you claiming that they aren’t actually necessary for the argument to be compelling? (If so, why were they included in the first place? This sounds a bit like justification drift.)
Presumably the programmer will make some effort to embed the right set of values in the AI. If this is an easy task, doom is probably not the default outcome.
AI pessimists have argued human values will be difficult to communicate due to their complexity. But as AI capabilities improve, AI systems get better at learning complex things.
Both the instrumental convergence thesis and the complexity of value thesis are key parts of the argument for AI pessimism as it’s commonly presented. Are you claiming that they aren’t actually necessary for the argument to be compelling? (If so, why were they included in the first place? This sounds a bit like justification drift.)