Do we need to decide on a moral principle(s) first? How would it be possible to develop beneficial AI without first ‘solving’ ethics/morality?
Good question! The answer is no: ‘solving’ ethics/morality first is one thing that we probably eventually need to do, but we could first solve a narrower, simpler form of AI alignment, and use those aligned systems to help us solve ethics/morality and the other trickier problems (like the control problem for more general, capable systems). This is more or less what is discussed in ambitious vs narrow value learning. Narrow value learning is one narrower, simpler form of AI alignment. There are others, discussed here under the heading “Alternative solutions”.
Good question! The answer is no: ‘solving’ ethics/morality first is one thing that we probably eventually need to do, but we could first solve a narrower, simpler form of AI alignment, and use those aligned systems to help us solve ethics/morality and the other trickier problems (like the control problem for more general, capable systems). This is more or less what is discussed in ambitious vs narrow value learning. Narrow value learning is one narrower, simpler form of AI alignment. There are others, discussed here under the heading “Alternative solutions”.