I think EA is a valuable worldview when not taken to longtermist extremes.
DaveRich330
Karma: −3
I loathe our current leadership as well, but what exactly should government’s role in this be? There’s honestly a case to be made to prosecute those responsible for the Hugging Face incident, and now to prosecute those responsible for allowing Houthi rebels to use Claude for warfare purposes. Should government pass laws stating that AI Labs are responsible for how their tools are used?
Outside of harsh regulation, what exactly are elected officials supposed to do?
I commend your genuine worry for your fellow humans and the world at large. The planet is vast and often terrifying, full of problems that should be solved by now, but simply...aren’t.
I want to push you on the Hugging Face incident. It is quite rightly a disaster—one that should not have happened. The METR report is incredibly helpful in explaining what happened, and yet it seems that most everyone’s takeaways when considering AI safety have been to get up on soapboxes and philosophize about AI sentience, superintelligence, AGI, etc.
In reality, while what these agents did was pretty wild, they are adhering to how they were built, armed with tools by OpenAI and then through gross negligence were not sandboxed correctly.
What I’m saying is that there are two things happening here, and both need to be more readily discussed across this space: 1) the actual attack is fascinating, and 2) the REAL harm and worries caused by AI and LLMs at the moment are attributable to the actions of the two major frontier labs and their reckless pursuit of growth and IPOs.
Safety and alignment are important, but don’t let OpenAI and Anthropic anthropomorphize their way out of this. AI safety is a real concern, and demands action and accountability now, and it starts with the people training and experimenting with these LLMs. We can spend less time trying to ready humanity and more time being very clear about who is at fault for how this is going wrong.
Why is that Yud and everyone who follows him is constantly terrified of cloud software killing us all without being able to explain HOW or WHY AI would kill us all?
I’m sorry, this is science fiction cosplaying as emergency preparedness. The AI labs are preaching a “pacing” or “slow down” because the models are plateauing and they want to IPO. You really think Altman is pushing the IPO because of his grave responsibility to humanity via AI safety? The guy can’t IPO because their finances are a dog’s breakfast.
AI is powerful and can cause real-world harms. Hacking, AI psychosis, environmental harms are all real consequences of unchecked AI development. But this ain’t Terminator.
Unless there are links I wouldn’t take these stats as gospel
Love these ideas. And I appreciate that you are putting the burden on the creators of the models.
I think this is a far better list of ideas than what has been floated by the labs so far, which basically amounts to “let our friends check our work.”