I would be interested to hear the counterpoints from those who have Disagree-voted on this post.
Likewise!
Do you think the âvery particular worldviewâ you describe is found equally among those working on technical AI safety and AI governance/âpolicy? My impression is that policy inherently requires thinking through concrete pathways of how AGI would lead to actual harm as well as greater engagement with people outside of AI safety.
I think theyâre quite prevalent regardless. While some peopleâs roles indeed require them to analyze concrete pathways more than others, the foundation of their analysis is often implicitly built upon this worldview in the first place. The result is that their concrete pathways tend to be centred around some kind of misaligned AGI, just in much more detail. Conversely, someone with a very different worldview who does such an analysis might end up with concrete pathways centred around severe discrimination of marginalized groups.
I have also noticed a split between the âsuperintelligence will kill us allâ worldview (which you seem to be describing) and âregardless of whether superintelligence kills us all, AGI/âTAI will be very disruptive and we need to manage those risksâ (which seemed to be more along the lines of the Will MacAskill post you linked toâespecially as he talks about directing people to causes other than technical safety or safety governance).
There are indeed many different âsub-worldviewsâ, and I was kind of lumping them all under one big umbrella. To me, the most defining characteristic of this worldview is AI-centrism, and treating the impending AGI as an extremely big deal â not just like any other big deals we have seen before, but this will be unprecedented. Those within this overarching worldview would differ in terms of the details, e.g. will it kill everyone? or will it just lead to gradual disempowerment? are LLMs getting us to AGI? or is it some yet-to-be-discovered architecture? should we focus on getting to AGI safely? or start thinking more about the post-AGI world? I think many people move between these âsub-worldviewsâ as they see evidences that update their priors, but way fewer people move out of this overarching worldview entirely.
Likewise!
I think theyâre quite prevalent regardless. While some peopleâs roles indeed require them to analyze concrete pathways more than others, the foundation of their analysis is often implicitly built upon this worldview in the first place. The result is that their concrete pathways tend to be centred around some kind of misaligned AGI, just in much more detail. Conversely, someone with a very different worldview who does such an analysis might end up with concrete pathways centred around severe discrimination of marginalized groups.
There are indeed many different âsub-worldviewsâ, and I was kind of lumping them all under one big umbrella. To me, the most defining characteristic of this worldview is AI-centrism, and treating the impending AGI as an extremely big deal â not just like any other big deals we have seen before, but this will be unprecedented. Those within this overarching worldview would differ in terms of the details, e.g. will it kill everyone? or will it just lead to gradual disempowerment? are LLMs getting us to AGI? or is it some yet-to-be-discovered architecture? should we focus on getting to AGI safely? or start thinking more about the post-AGI world? I think many people move between these âsub-worldviewsâ as they see evidences that update their priors, but way fewer people move out of this overarching worldview entirely.