Thanks for taking the time to share this, Hayden. It was very useful.
To what extent do behavioural science and systems thinking/āchange matter for AI governance?
To give you my view: I think that nearly all outcomes that EA cares about are mediated by individual and group behaviours and decisions: Who thinks what and does what (e.g., WRT. careers, donations, and advocacy) etc. All of this occurs in a broader context of social norms and laws etc.
Based on all this, I think that it is important to understand what people think and do, why they think and do what they do, and how to change that. Also, to understand how various contextual factors such as social norms and laws affect what people think and do and can be changed.
I notice related work on areas such as climate change, and I project that similar will be needed in AI governance. However, I donāt know the extent to which people working on AI governance share that view or what work, if anything, that has been done. Iād be interested to hear any thoughts that you have time to share.
Also, Iād really appreciate if you can suggest any good literature or people to engage with.
Iām not Hayden but I think behavioural science is useful area for thinking about AI governance, in particular about the design of human-computer interfaces. One example with current widely deployed AI systems is recommender engines (this is not a HCI eg). Iām trying to understand the tendencies of recommenders towards biases like concentration, or contamination problems, and how they impact user behaviour and choice. Additionally, how what they optimise for does/ādoes not capture their values, whether thatās because of a misalignment of values between the user and the company or because itās just really hard to learn human preferences because theyāre complex. In doing this, itās really tricky to actually distinguish in the wild between the choice architecture (behavioural parts) vs the algorithm when it comes to attributing to usersā actions.
Recommender systems are a great example of a broader concern. Another is lethal autonomous weapons, where a big focus is āmeaningful human controlā. Automation bias is an issue even up to the nuclear levelāthe concern is that people will more blindly trust ML systems, and wonāt disbelieve them as people did in several Cold War close calls (eg Petrov not believing his computer warning of an attack). See Autonomy and machine learning at the interface of nuclear weapons, computers and people.
Thanks for taking the time to share this, Hayden. It was very useful.
To what extent do behavioural science and systems thinking/āchange matter for AI governance?
To give you my view: I think that nearly all outcomes that EA cares about are mediated by individual and group behaviours and decisions: Who thinks what and does what (e.g., WRT. careers, donations, and advocacy) etc. All of this occurs in a broader context of social norms and laws etc.
Based on all this, I think that it is important to understand what people think and do, why they think and do what they do, and how to change that. Also, to understand how various contextual factors such as social norms and laws affect what people think and do and can be changed.
I notice related work on areas such as climate change, and I project that similar will be needed in AI governance. However, I donāt know the extent to which people working on AI governance share that view or what work, if anything, that has been done. Iād be interested to hear any thoughts that you have time to share.
Also, Iād really appreciate if you can suggest any good literature or people to engage with.
Iām not Hayden but I think behavioural science is useful area for thinking about AI governance, in particular about the design of human-computer interfaces. One example with current widely deployed AI systems is recommender engines (this is not a HCI eg). Iām trying to understand the tendencies of recommenders towards biases like concentration, or contamination problems, and how they impact user behaviour and choice. Additionally, how what they optimise for does/ādoes not capture their values, whether thatās because of a misalignment of values between the user and the company or because itās just really hard to learn human preferences because theyāre complex. In doing this, itās really tricky to actually distinguish in the wild between the choice architecture (behavioural parts) vs the algorithm when it comes to attributing to usersā actions.
Hi both,
Yes behavioural science isnāt a topic Iām super familiar with, but it seems very important!
I think most of the focus so far has been on shifting norms/ābehaviour at top AI labs, for example nudging Publication and Release Norms for Responsible AI.
Recommender systems are a great example of a broader concern. Another is lethal autonomous weapons, where a big focus is āmeaningful human controlā. Automation bias is an issue even up to the nuclear levelāthe concern is that people will more blindly trust ML systems, and wonāt disbelieve them as people did in several Cold War close calls (eg Petrov not believing his computer warning of an attack). See Autonomy and machine learning at the interface of nuclear weapons, computers and people.
Jess Whittlestoneās PhD was in Behavioural Science, now sheās Head of AI Policy at the Centre for Long-Term Resilience.