I’m a cofounder and the head of AI governance at the Nottingham AI Safety Initiative (NAISI), a student society with faculty advisors and affiliation with UoN’s student union. In this role I’ve taught courses on frontier AI governance, run multiple events on AI safety, and convinced several people to pursue careers in AI safety. Alongside NAISI I’m doing a part-time MA in policy research at Nottingham, focused on catastrophic AI risk.
My current main research is the infrastructure required for unmonitorable multi-agent collusion to become a reality, and what we can do about it. It grew out of my LessWrong work on encrypted agent communication.
I also run BlueDot-informed AI governance courses, help people move into AI safety careers from non-technical backgrounds, and write on the Forum and LessWrong.
Thought-provoking read, thanks for having shared it with me on my post!
I particularly appreciated your ideas around the tension between grounding AI morality in human-like experience—despite AI’s lack of it—while recognising that both continuous change and susceptibility to bias complicate moral reasoning, raising questions about whether brief, strategically timed or partial experiences could instil robust moral understanding without distorting judgment in the way emotionally charged human experience often does.
I also found your reflections on engineering AI with predominantly positive qualia, the challenge to valence as the default moral reward signal, the idea that AI sentience might influence human ethical behaviour in return, and the call for a deeper moral foundation than utility maximisation to all be quite novel and helpful ways of seeing things.
Your reference to Agarwal and Edelman aligns well with David Pearce’s idea of ‘a motivational system based on heritable gradients of bliss’, worth checking out if you’re not familiar—I think it’s a promising model for designing a sentient ASI.
I’m of course not aware of why you chose not to distinguish between consciousness and sentience, but I do find the distinction 80,000 Hours (and no doubt many other sources) makes between them to be useful.