I fully agree with the shift away from research and toward policy. With how close we are to what you termed smarter-than-human AI (also called AGI or ASI, but your term is much more precise, so I’ll use it going forward), research is not where efforts are best placed. We could be looking at a human extinction scenario (or an equally bad outcome, such as the permanent limiting of human potential) within 5-20 years. That’s an emergency situation as far as I’m considered. Once the necessary laws and procedures are in place, research an continue.
I can’t speak for all EAs, but my ultimate goal is to see a world without smarter-than-human AI until humanity outgrows its tendencies to wage war and seek personal gain over the flourishing of all sentient beings. This would likely place ASI development somewhere between 100 years AP* and never, and probably closer to the “never” end of that timescale. This is something we have to accept—especially those of us with tech-loving tendencies.
I’m under no illusions that Silicon Valley would ever accept this, but in a democratic society they aren’t the ones calling the shots. A democratic government can ban agents / generative / smarter-than-human AI, and the actors I mentioned previously would simply have to accept it. We need the US, EU, Canada, Taiwan, and Japan to adopt MIRI guidelines on AI safety, security, and non-proliferation—and these conversations must begin at the local level.
If we are looking to shift the Overton window, we have to target our communications toward “ordinary people” and policymakers, not tech geeks and data wonks. This will be my top priority going forward, along with animal welfare activism.
I fully agree with the shift away from research and toward policy. With how close we are to what you termed smarter-than-human AI (also called AGI or ASI, but your term is much more precise, so I’ll use it going forward), research is not where efforts are best placed. We could be looking at a human extinction scenario (or an equally bad outcome, such as the permanent limiting of human potential) within 5-20 years. That’s an emergency situation as far as I’m considered. Once the necessary laws and procedures are in place, research an continue.
I can’t speak for all EAs, but my ultimate goal is to see a world without smarter-than-human AI until humanity outgrows its tendencies to wage war and seek personal gain over the flourishing of all sentient beings. This would likely place ASI development somewhere between 100 years AP* and never, and probably closer to the “never” end of that timescale. This is something we have to accept—especially those of us with tech-loving tendencies.
I’m under no illusions that Silicon Valley would ever accept this, but in a democratic society they aren’t the ones calling the shots. A democratic government can ban agents / generative / smarter-than-human AI, and the actors I mentioned previously would simply have to accept it. We need the US, EU, Canada, Taiwan, and Japan to adopt MIRI guidelines on AI safety, security, and non-proliferation—and these conversations must begin at the local level.
If we are looking to shift the Overton window, we have to target our communications toward “ordinary people” and policymakers, not tech geeks and data wonks. This will be my top priority going forward, along with animal welfare activism.
*AP = after present