In catastrophic AI risk, extinction is only half the equation. The other half is ensuring a future thatās actually worth saving. I want to help ensure that a post-AGI world is a good one, not marred by moral disaster.
I will be doing my masters thesis on policy/āgovernance addressing catastrophic AI risk. Iām currently hoping to focus on preventing AI from exacerbating or locking in totalitarianism, perhaps particularly fascism.
Iāve also been running my universityās Buddhism and Meditation society for three years.
Best place to reach me is my email, lloydrb100@gmail.com
Thought-provoking read, thanks for having shared it with me on my post!
I particularly appreciated your ideas around the tension between grounding AI morality in human-like experienceādespite AIās lack of itāwhile recognising that both continuous change and susceptibility to bias complicate moral reasoning, raising questions about whether brief, strategically timed or partial experiences could instil robust moral understanding without distorting judgment in the way emotionally charged human experience often does.
I also found your reflections on engineering AI with predominantly positive qualia, the challenge to valence as the default moral reward signal, the idea that AI sentience might influence human ethical behaviour in return, and the call for a deeper moral foundation than utility maximisation to all be quite novel and helpful ways of seeing things.
Your reference to Agarwal and Edelman aligns well with David Pearceās idea of āa motivational system based on heritable gradients of blissā, worth checking out if youāre not familiarāI think itās a promising model for designing a sentient ASI.
Iām of course not aware of why you chose not to distinguish between consciousness and sentience, but I do find the distinction 80,000 Hours (and no doubt many other sources) makes between them to be useful.