I think this is an important point that’s worth saying.
For what it’s worth, I am not super pessimistic that “solving alignment” is something that can be solved in principle. But I’m quite concerned that the safety-minded AI companies seem to completely ignore the philosophical problems with AI alignment. They all operate under the assumption that alignment is purely an ML problem and they can solve it by basically doing ML research. Which I expect is false (credence: 70%).
Wei Dai has written some good stuff about the problem of “philosophical competence”. See here for a collection of his writings on the topic.
I think this is an important point that’s worth saying.
For what it’s worth, I am not super pessimistic that “solving alignment” is something that can be solved in principle. But I’m quite concerned that the safety-minded AI companies seem to completely ignore the philosophical problems with AI alignment. They all operate under the assumption that alignment is purely an ML problem and they can solve it by basically doing ML research. Which I expect is false (credence: 70%).
Wei Dai has written some good stuff about the problem of “philosophical competence”. See here for a collection of his writings on the topic.
Thanks Michael, I hadn’t seen Wei’s comments before and that was quite a fruitful rabbit hole 😅