Jobst—yes, I think ew need a lot more psych research on how to elicit the human values that AI systems are trying to align with. Especially given that some of our most important values either can’t be articulated very well, or are too ‘obvious’ and ‘common-sensical’ to be discussed much, or are embodied in our physical phenotypes rather than articulated in our brains.
This becomes particularly important in human feedback/input about “higher-level” or more “abstract” questions, as in OpenAI’s deliberative mini-public / citizen assembly idea (https://openai.com/blog/democratic-inputs-to-ai).
Jobst—yes, I think ew need a lot more psych research on how to elicit the human values that AI systems are trying to align with. Especially given that some of our most important values either can’t be articulated very well, or are too ‘obvious’ and ‘common-sensical’ to be discussed much, or are embodied in our physical phenotypes rather than articulated in our brains.
This becomes particularly important in human feedback/input about “higher-level” or more “abstract” questions, as in OpenAI’s deliberative mini-public / citizen assembly idea (https://openai.com/blog/democratic-inputs-to-ai).