Executive summary: This speculative post shares a collection of somewhat novel, mostly unpursued AI safety research and project ideas spanning field infrastructure, intervention prioritisation, recursive self-improvement, governance, robotics, and digital moral patienthood, offered in case they prove helpful or generative for others.
Key points:
The author proposes a live, regularly-updated, highly visual database of AIS research questions with progress tracking, plus a separate database of proposed interventions tracking how many people work on each and roughly how much time, to more quantitatively assess neglectedness.
The author asks whether intervention comparisons should factor in interactions between interventions (synergies, clashes) and viability across broad timelines, noting these factors aren’t often taken into account, with mechanistic interpretability and evals given as a possibly mutually reinforcing example.
The author asks whether recursive self-improvement can be roughly simulated through an LLM repeatedly improving its system prompt as a toy model for alignment dynamics, while noting this would not reproduce full RSI since weights, architecture, training data, and capabilities remain fixed.
The author suggests that if the world is currently getting worse, postponing the singularity may be an active choice to let worse norms and more brittle institutions become the substrate from which superintelligence emerges—framed as the “opposite of a long reflection.”
Drawing on Ilya Sutskever’s November 2025 claim that models lack an emotion-modulated value function and Geoffrey Hinton’s argument that safe superintelligence requires genuine care for us, the author asks whether emotion’s functional benefits can be obtained without sentience—an “unfeeling feeling machine” that stretches the philosophical zombie concept.
The author argues near-future videogames may pose uniquely severe s-risks because many (possibly millions) of NPCs might run on possibly-sentient LLMs and videogames are possibly the only context where AI systems might be deliberately tortured.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.
Executive summary: This speculative post shares a collection of somewhat novel, mostly unpursued AI safety research and project ideas spanning field infrastructure, intervention prioritisation, recursive self-improvement, governance, robotics, and digital moral patienthood, offered in case they prove helpful or generative for others.
Key points:
The author proposes a live, regularly-updated, highly visual database of AIS research questions with progress tracking, plus a separate database of proposed interventions tracking how many people work on each and roughly how much time, to more quantitatively assess neglectedness.
The author asks whether intervention comparisons should factor in interactions between interventions (synergies, clashes) and viability across broad timelines, noting these factors aren’t often taken into account, with mechanistic interpretability and evals given as a possibly mutually reinforcing example.
The author asks whether recursive self-improvement can be roughly simulated through an LLM repeatedly improving its system prompt as a toy model for alignment dynamics, while noting this would not reproduce full RSI since weights, architecture, training data, and capabilities remain fixed.
The author suggests that if the world is currently getting worse, postponing the singularity may be an active choice to let worse norms and more brittle institutions become the substrate from which superintelligence emerges—framed as the “opposite of a long reflection.”
Drawing on Ilya Sutskever’s November 2025 claim that models lack an emotion-modulated value function and Geoffrey Hinton’s argument that safe superintelligence requires genuine care for us, the author asks whether emotion’s functional benefits can be obtained without sentience—an “unfeeling feeling machine” that stretches the philosophical zombie concept.
The author argues near-future videogames may pose uniquely severe s-risks because many (possibly millions) of NPCs might run on possibly-sentient LLMs and videogames are possibly the only context where AI systems might be deliberately tortured.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.