Let me add a little global perspective (as a US citizen who’s lived in 4 countries outside the US for a total of 14 years, and who doesn’t always see the US as the ‘good guy’ in geopolitics).
The US is 4% of the world’s population. The American AI industry is (probably) years ahead of any other country, and is pushing ahead with the rationale that ‘if we don’t keep pushing ahead, a bad actor (which usually implies China) will catch up, and that would be bad’. Thus, we impose AI X-risk on the other 96% of humans without the informed consent, support, or oversight.
We used the same arms-race rationale in the 1940s to develop the atomic bomb (‘if we don’t do it, Germany will’) and in the 1950s to develop the hydrogen bomb (‘if we don’t do it, the Soviet Union will’). In both cases, we were the bad actor. The other countries were nowhere close to us. We exaggerated the threat that they would catch up, and we got the American public to buy into that narrative. But we were really the ones pushing ahead into X-risk territory. Now we’re promoting the same narrative for AI development. ‘The AI arms race cannot be stopped’, ‘AGI is inevitable’, ‘the genie is out of the bottle’, ‘if not us, then China’, etc, etc.
We Americans have a very hard time accepting that ‘we might be the baddies’. We are uncomfortable acknowledging any moral obligations to the rest of humanity (if they conflict in any way with our geopolitical interests). We like to impose our values on the world, but we don’t like to submit to any global oversight by others.
I hope that this public discussion about AI risks also includes some soul-searching by Americans—not just the AI industry, but all of us, concerning the way that our country is, yet again, pushing ahead with developing extremely dangerous technology, without any sense of moral obligation to others.
Having taught online courses for CUHK-Shenzhen in China for a year, and discussed quite a bit about EA, AI, and X risk with the very bright young students there, I often imagine how they would view the recent developments in the American AI industry. I think they would be appalled by our American hubris. They know that the American political system is too partisan, fractured, slow, and dysfunctional to impose any effective regulation on Big Tech. They know that American tech companies are legally obligated (by ‘fiduciary duty’ to shareholders) to prioritize quarterly profits over long-term human survival. They know that many Bay Area tech bros supporting AI are transhumanists, extropians, or Singularity-welcomers who look forward to humanity being replaced by machines. They know that many Americans view China as a reckless, irresponsible, totalitarian state that isn’t worth listening to about any AI safety concerns. So, I imagine, any young Chinese students who’s paying attention would take an extremely negative view of the risks that the American AI industry is imposing on the other 7.7 billion people in the world.
They know that American tech companies are legally obligated (by ‘fiduciary duty’ to shareholders) to prioritize quarterly profits over long-term human survival.
Fwiw I read here that there actually is not such a legal duty to prioritize profits.
Not an expert either, but safest to say the corporate-law question is nuanced and not free from doubt. It’s pretty clear there’s no duty to maximize short-term profits, though.
But we can surmise that most boards that allow the corporation to seriously curtail its profits—at least its medium-term profits—will get replaced by shareholders soon enough. So the end result is largely the same.
I signed and strongly support this open letter.
Let me add a little global perspective (as a US citizen who’s lived in 4 countries outside the US for a total of 14 years, and who doesn’t always see the US as the ‘good guy’ in geopolitics).
The US is 4% of the world’s population. The American AI industry is (probably) years ahead of any other country, and is pushing ahead with the rationale that ‘if we don’t keep pushing ahead, a bad actor (which usually implies China) will catch up, and that would be bad’. Thus, we impose AI X-risk on the other 96% of humans without the informed consent, support, or oversight.
We used the same arms-race rationale in the 1940s to develop the atomic bomb (‘if we don’t do it, Germany will’) and in the 1950s to develop the hydrogen bomb (‘if we don’t do it, the Soviet Union will’). In both cases, we were the bad actor. The other countries were nowhere close to us. We exaggerated the threat that they would catch up, and we got the American public to buy into that narrative. But we were really the ones pushing ahead into X-risk territory. Now we’re promoting the same narrative for AI development. ‘The AI arms race cannot be stopped’, ‘AGI is inevitable’, ‘the genie is out of the bottle’, ‘if not us, then China’, etc, etc.
We Americans have a very hard time accepting that ‘we might be the baddies’. We are uncomfortable acknowledging any moral obligations to the rest of humanity (if they conflict in any way with our geopolitical interests). We like to impose our values on the world, but we don’t like to submit to any global oversight by others.
I hope that this public discussion about AI risks also includes some soul-searching by Americans—not just the AI industry, but all of us, concerning the way that our country is, yet again, pushing ahead with developing extremely dangerous technology, without any sense of moral obligation to others.
Having taught online courses for CUHK-Shenzhen in China for a year, and discussed quite a bit about EA, AI, and X risk with the very bright young students there, I often imagine how they would view the recent developments in the American AI industry. I think they would be appalled by our American hubris. They know that the American political system is too partisan, fractured, slow, and dysfunctional to impose any effective regulation on Big Tech. They know that American tech companies are legally obligated (by ‘fiduciary duty’ to shareholders) to prioritize quarterly profits over long-term human survival. They know that many Bay Area tech bros supporting AI are transhumanists, extropians, or Singularity-welcomers who look forward to humanity being replaced by machines. They know that many Americans view China as a reckless, irresponsible, totalitarian state that isn’t worth listening to about any AI safety concerns. So, I imagine, any young Chinese students who’s paying attention would take an extremely negative view of the risks that the American AI industry is imposing on the other 7.7 billion people in the world.
Fwiw I read here that there actually is not such a legal duty to prioritize profits.
MaxRa—I might have been wrong about this; I’m not at all an expert in corporate law. Thanks for the informative link.
A more accurate claim might be ‘American tech companies tend to prioritize short-term profits over long-term human survival’.
Not an expert either, but safest to say the corporate-law question is nuanced and not free from doubt. It’s pretty clear there’s no duty to maximize short-term profits, though.
But we can surmise that most boards that allow the corporation to seriously curtail its profits—at least its medium-term profits—will get replaced by shareholders soon enough. So the end result is largely the same.