I completely agree.
But others may not, because most humans aren’t longtermists nor utilitarians. So I’m afraid arguments like this won’t sway the public opinion much at all. People like progress because it will get them and their loved ones (children and grandchildren, whose future they can imagine) better lives. They just barely care at all whether humanity ends after their grandchildren’s lives (to the extent they can even think about it).
This is why I believe that most arguents against AGI x-risk are really based on differing timelines. People like to think that humans are so special we won’t surpass them for a long time. And they mostly care about the future for their loved ones.
Copied from my LW comment, since this is probably more of an EAF discussion:
This is really important pushback. This is the discussion we need to be having.
Most people who are trying to track this believe China has not been racing toward AGI up to this point. Whether they embark on that race is probably being determined now—and based in no small part on the US’s perceived attitude and intentions.
Any calls for racing toward AGI should be closely accompanied with “and of course we’d use it to benefit the entire world, sharing the rapidly growing pie”. If our intentions are hostile, foreign powers have little choice but to race us.
And we should not be so confident we will remain ahead if we do race. There are many routes to progress other than sheer scale of pretraining. The release of DeepSeek r1 today indicates that China is not so far behind. Let’s remember that while the US “won” the race for nukes, our primary rival had nukes very soon after—by stealing our advancements. A standoff between AGI-armed US and China could be disastrous—or navigated successfully if we take the right tone and prevent further proliferation (I shudder to think of Putin controlling an AGI, or many potentially unstable actors).
This discussion is important, so it needs to be better. This pushback is itself badly flawed. In calling out the report’s lack of references, it provides almost none itself. Citing a 2017 official statement from China seems utterly irrelevant to guessing their current, privately held position. Almost everyone has updated massively since 2017. (edit: It’s good that this piece does note that public statements are basically meaningless in such matters.) If China is “racing toward AGI” as an internal policy, they probably would’ve adopted that recently. (I doubt that they are racing yet, but it seems entirely possible they’ll start now in response to the US push to do so—and the their perspective on the US as a dangerous aggressor on the world stage. But what do I know—we need real experts on China and international relations.)
Pointing out the technical errors in the report seems irrelevant to harmful. You can understand very little of the details and still understand that AGI would be a big, big deal if true — and the many experts predicting short timelines could be right. Nitpicking the technical expertise of people who are essentially probably correct in their assessment just sets a bad tone of fighting/arguing instead of having a sensible discussion.
And we desperately need a sensible discussion on this topic.