However, on your point 2 about ‘technological determinism’: I worry that way too many EAs have adopted this view that building ASI is ‘inevitable’, and that the only leverage we have over the future of AI X-risk is to join AI companies explicitly trying to build ASI, and try to steer them in benign directions that increase control and alignment.
That seems to be the strategy that 80k Hours has actively pushed for years. It certainly helps EAs find lucrative, high-prestige jobs in the Bay Area, and gives them the illusion that they’re doing good. But to outsiders, it looks like little more than a self-serving jobs program for utilitarians who want a slice of that sweet, sweet, equity in AI companies—without any of the guilt of actually working on ASI capabilities development.
And the weird thing is, this strategy only makes sense if we believe two key things: (1) ASI development is ‘inevitable’—even if virtually all of humanity agrees that it would be suicidal, and (2) ASI alignment is solvable—such that we can keep control of ASIs, and force them to work for humans, generation after generation, forever.
Both of these seem equally and wildly implausible. And the sooner we recognize their implausibility, the faster we can move beyond this rather cynical/self-serving 80k Hours strategy of encouraging young EAs to join the safety-washing, PR-window-dressing ‘technical AI alignment’ groups at frontier AI companies like OpenAI, Anthropic, DeepMind, etc.
Matt—thanks for an insightful post. Mostly agree.
However, on your point 2 about ‘technological determinism’: I worry that way too many EAs have adopted this view that building ASI is ‘inevitable’, and that the only leverage we have over the future of AI X-risk is to join AI companies explicitly trying to build ASI, and try to steer them in benign directions that increase control and alignment.
That seems to be the strategy that 80k Hours has actively pushed for years. It certainly helps EAs find lucrative, high-prestige jobs in the Bay Area, and gives them the illusion that they’re doing good. But to outsiders, it looks like little more than a self-serving jobs program for utilitarians who want a slice of that sweet, sweet, equity in AI companies—without any of the guilt of actually working on ASI capabilities development.
And the weird thing is, this strategy only makes sense if we believe two key things: (1) ASI development is ‘inevitable’—even if virtually all of humanity agrees that it would be suicidal, and (2) ASI alignment is solvable—such that we can keep control of ASIs, and force them to work for humans, generation after generation, forever.
Both of these seem equally and wildly implausible. And the sooner we recognize their implausibility, the faster we can move beyond this rather cynical/self-serving 80k Hours strategy of encouraging young EAs to join the safety-washing, PR-window-dressing ‘technical AI alignment’ groups at frontier AI companies like OpenAI, Anthropic, DeepMind, etc.