While some narratives about AI alignment bear a conspicuous resemblance to the apocalyptic thinking and eschatology of some Christians in history, there isn’t much that fundamentally distinguishes that mindset towards AI alignment from similar mindsets towards other ostensible existential risks.
This has been true at times during the last century and remains true today. It was at times crucial, if not necessary, for some of those involved in other communities similar to long-termist effective altruism to make decisions and take actions that contradicted much of this advice.
---
This advice is in one way also applicable to other potential global catastrophic or existential risks as well but in another way may not be applicable to any of them. Even before the advent of nuclear weapons, World War II (WWII) was feared to potentially destroy civilization. Between the Cold War that began a few years later and different kinds of global ecological catastrophe, there are hundreds of millions of people across several generations who have experienced for more than half a century a life in way that had them convinced they were living at the hinge of history. While such concerns may have been alleged to be fears too similar to religious eschatology, almost all of them were rooted in secular phenomena examined from a naturalistic and materialist perspective.
This isn’t limited to generic populations and includes communities that are so similar to the existential risk (x-risk) reduction community of today that they serve as a direct inspiration for our present efforts. After the Manhattan Project, Albert Einstein and other scientists who contributed to the effort but weren’t aware of the full intentions of the government of the United States for nuclear weapons both wanted to do something about their complicity in such destruction. For the record, while they weren’t certain either way, at the time many of those scientists feared a sufficiently large-scale nuclear war could indeed cause human extinction. Among others, those scientists founded the Bulletin of Atomic Scientists, likely the first ever ‘x-risk reduction’ organization in history.
In both the United States and the Soviet Union, scientists and others well-placed to warn the public about the cataclysmic threat posed by the struggle for more power by both superpowers took personal and professional risks. Some of those who did so were censured, fired and/or permanently lost their careers. Some were even criminally convicted or jailed. Had they not, perhaps none of us would have ever been born to try reducing x-risks or talk about how to think about that today.
To some extent, the same likely remains true in multiple countries today. The same is also true for the climate crisis. Employees of Amazon who have made tweets advocating for greater efforts to combat the climate crisis have been fired because their affiliation with Amazon in that way risks bringing too much attention to how Amazon itself contributes to the crisis. There also more and more people who through civil disobedience have gotten arrested for their participation in civil disobedience to combat the climate crisis or other global catastrophic risks.
I’ve known many in effective altruism who’ve changed their careers so to focus on x-risk reduction not limited to AI alignment. There are millions of young people around the world who are pursuing careers intended to do the same because they both believe it’s more important than anything else they could do and it’s futile to pursue anything else in the face of looming catastrophe. All of this is anticipated to be critical in their lifetimes, often in the next 20-30 years. All of those people have also been presumed to be delusional in a way akin to the apocalyptic delusions of religious fanatics in history.
While for the other risks there isn’t the same expected potential for transhumanism, indefinite life extension and utopian conditions, the future of humankind and perhaps all life is considered to be under threat. Beyond effective altruism, I’ve got more and more friends, and friends of friends, who are embracing a mindset entailing much of the above. Perhaps what should surprise us is that more people we don’t know from in and around effective altruism aren’t doing the same.
I am confused by this comment because I think you’re suggesting the Bulletin of Atomic Scientists didn’t follow the advice above, but it sounds like they followed it to the letter.
I recognize there is some ambiguity in my comment. I also read your article again and I noticed some ambiguity I perceived on my part. That seems to be the source of confusion.
To clarify, it was not only the Bulletin of Atomic Scientists (BAS) who took those personal and professional risks in question. Other scientists and individuals who were not ‘leaders’ took those risks too. Albert Einstein did so personally outside the BAS but he called on any and all scientists to be willing to blow the whistle if necessary, even if they risked going to jail.
For such leading scientists to call on others to also be (tentatively) willing to take such risks if necessary contradicts the advice of early church leaders to the laity to “not quit their day jobs.”
Nobody was advising scientists in positions to reduce x-risks or whatnot to embrace a value system so different they’d personally spurn those who didn’t share it. Yet my impression is that during the Cold War, “common sense morality” would be loyalty to the authorities in the United States or Soviet Union, including to not challenge their Cold War policies. In that case, scientists and other whistleblowers would have been defying commonly accepted public morality.
I think I’ve addressed this under the “Boldly tell the truth” bullet. Early Christians were encouraged to share their beliefs even if it would result in their deaths, which seems much more extreme than potentially losing a job.
If you’re interested in how they balanced these two seemingly contradictory topics, I could write more about that later, but I thought it would be pretty straightforward (speak boldly and honestly about your beliefs, and in all other respects be a good citizen).
Summary: The difference between early Christianity and modern movements focused on reducing prospective existential risks is to that to publicly and boldly speak one’s beliefs that go against the ruling ideology was considered against common sense morality during the Cold War. Modern x-risk movements can’t defend themselves from suppression as well because their small communities subject to severe conditions in modern police/surveillance states.
Some scientists and whistleblowers in the Soviet Union and the United States not only lost their jobs but were imprisoned for a number of years, or were otherwise legally punished or politically persecuted in ways that had severe consequences beyond the professional. As far as I’m aware, none of them were killed and I’d be very surprised if any of them were.
Please don’t concern yourself to write more on this subject on my behalf. I’m satisfied with the conclusion that the difference between early Christians and the modern whistleblowers in question is that for the whistleblowers to publicly and boldly express their honest beliefs was perceived as a betrayal of good citizenship. The two major conditions that come to mind that determined these different outcomes are:
1. The Authoritarianism on Both Sides of the Iron Curtain During the Cold War.
Stalinist Russia is of course recognized as being totalitarian but history has been mythologized to downplay how much liberal democracy in the United States was at risk of failing during the same period. I watched a couple documentaries on that subject produced to clarify the record about the facts of the matter during the McCarthyist era. The anti-communism of the time was becoming extreme in a way well-characterized in a speech Harry S. Truman addressed to Congress. I forget the exact quote but to paraphrase it, it went something like: “we didn’t finish beating fascism only for us to descend into fascism ourselves.”
2. The Absence of an Authoritative Organization on the Part of the Defectors
(Note: In this case, I don’t mean “defector” to be pejorative but only to indicate that members of the respective communities took actions defying rules established by the reigning political authority.)
As I understand it, Christianity began dramatically expanding even within a few years of Jesus’ crucifixion. Over the next few decades, it became a social/religious organization that grew enough that it became harder and harder for the Roman Empire to simply quash. There was not really an organization for Cold War whistleblowers that had enough resources to meaningfully defend its members from being suppressed or persecuted.
Summary:
While some narratives about AI alignment bear a conspicuous resemblance to the apocalyptic thinking and eschatology of some Christians in history, there isn’t much that fundamentally distinguishes that mindset towards AI alignment from similar mindsets towards other ostensible existential risks.
This has been true at times during the last century and remains true today. It was at times crucial, if not necessary, for some of those involved in other communities similar to long-termist effective altruism to make decisions and take actions that contradicted much of this advice.
---This advice is in one way also applicable to other potential global catastrophic or existential risks as well but in another way may not be applicable to any of them. Even before the advent of nuclear weapons, World War II (WWII) was feared to potentially destroy civilization. Between the Cold War that began a few years later and different kinds of global ecological catastrophe, there are hundreds of millions of people across several generations who have experienced for more than half a century a life in way that had them convinced they were living at the hinge of history. While such concerns may have been alleged to be fears too similar to religious eschatology, almost all of them were rooted in secular phenomena examined from a naturalistic and materialist perspective.
This isn’t limited to generic populations and includes communities that are so similar to the existential risk (x-risk) reduction community of today that they serve as a direct inspiration for our present efforts. After the Manhattan Project, Albert Einstein and other scientists who contributed to the effort but weren’t aware of the full intentions of the government of the United States for nuclear weapons both wanted to do something about their complicity in such destruction. For the record, while they weren’t certain either way, at the time many of those scientists feared a sufficiently large-scale nuclear war could indeed cause human extinction. Among others, those scientists founded the Bulletin of Atomic Scientists, likely the first ever ‘x-risk reduction’ organization in history.
In both the United States and the Soviet Union, scientists and others well-placed to warn the public about the cataclysmic threat posed by the struggle for more power by both superpowers took personal and professional risks. Some of those who did so were censured, fired and/or permanently lost their careers. Some were even criminally convicted or jailed. Had they not, perhaps none of us would have ever been born to try reducing x-risks or talk about how to think about that today.
To some extent, the same likely remains true in multiple countries today. The same is also true for the climate crisis. Employees of Amazon who have made tweets advocating for greater efforts to combat the climate crisis have been fired because their affiliation with Amazon in that way risks bringing too much attention to how Amazon itself contributes to the crisis. There also more and more people who through civil disobedience have gotten arrested for their participation in civil disobedience to combat the climate crisis or other global catastrophic risks.
I’ve known many in effective altruism who’ve changed their careers so to focus on x-risk reduction not limited to AI alignment. There are millions of young people around the world who are pursuing careers intended to do the same because they both believe it’s more important than anything else they could do and it’s futile to pursue anything else in the face of looming catastrophe. All of this is anticipated to be critical in their lifetimes, often in the next 20-30 years. All of those people have also been presumed to be delusional in a way akin to the apocalyptic delusions of religious fanatics in history.
While for the other risks there isn’t the same expected potential for transhumanism, indefinite life extension and utopian conditions, the future of humankind and perhaps all life is considered to be under threat. Beyond effective altruism, I’ve got more and more friends, and friends of friends, who are embracing a mindset entailing much of the above. Perhaps what should surprise us is that more people we don’t know from in and around effective altruism aren’t doing the same.
I am confused by this comment because I think you’re suggesting the Bulletin of Atomic Scientists didn’t follow the advice above, but it sounds like they followed it to the letter.
I recognize there is some ambiguity in my comment. I also read your article again and I noticed some ambiguity I perceived on my part. That seems to be the source of confusion.
To clarify, it was not only the Bulletin of Atomic Scientists (BAS) who took those personal and professional risks in question. Other scientists and individuals who were not ‘leaders’ took those risks too. Albert Einstein did so personally outside the BAS but he called on any and all scientists to be willing to blow the whistle if necessary, even if they risked going to jail.
For such leading scientists to call on others to also be (tentatively) willing to take such risks if necessary contradicts the advice of early church leaders to the laity to “not quit their day jobs.”
Nobody was advising scientists in positions to reduce x-risks or whatnot to embrace a value system so different they’d personally spurn those who didn’t share it. Yet my impression is that during the Cold War, “common sense morality” would be loyalty to the authorities in the United States or Soviet Union, including to not challenge their Cold War policies. In that case, scientists and other whistleblowers would have been defying commonly accepted public morality.
I think I’ve addressed this under the “Boldly tell the truth” bullet. Early Christians were encouraged to share their beliefs even if it would result in their deaths, which seems much more extreme than potentially losing a job.
If you’re interested in how they balanced these two seemingly contradictory topics, I could write more about that later, but I thought it would be pretty straightforward (speak boldly and honestly about your beliefs, and in all other respects be a good citizen).
Summary: The difference between early Christianity and modern movements focused on reducing prospective existential risks is to that to publicly and boldly speak one’s beliefs that go against the ruling ideology was considered against common sense morality during the Cold War. Modern x-risk movements can’t defend themselves from suppression as well because their small communities subject to severe conditions in modern police/surveillance states.
Some scientists and whistleblowers in the Soviet Union and the United States not only lost their jobs but were imprisoned for a number of years, or were otherwise legally punished or politically persecuted in ways that had severe consequences beyond the professional. As far as I’m aware, none of them were killed and I’d be very surprised if any of them were.
Please don’t concern yourself to write more on this subject on my behalf. I’m satisfied with the conclusion that the difference between early Christians and the modern whistleblowers in question is that for the whistleblowers to publicly and boldly express their honest beliefs was perceived as a betrayal of good citizenship. The two major conditions that come to mind that determined these different outcomes are:
1. The Authoritarianism on Both Sides of the Iron Curtain During the Cold War.
Stalinist Russia is of course recognized as being totalitarian but history has been mythologized to downplay how much liberal democracy in the United States was at risk of failing during the same period. I watched a couple documentaries on that subject produced to clarify the record about the facts of the matter during the McCarthyist era. The anti-communism of the time was becoming extreme in a way well-characterized in a speech Harry S. Truman addressed to Congress. I forget the exact quote but to paraphrase it, it went something like: “we didn’t finish beating fascism only for us to descend into fascism ourselves.”
2. The Absence of an Authoritative Organization on the Part of the Defectors
(Note: In this case, I don’t mean “defector” to be pejorative but only to indicate that members of the respective communities took actions defying rules established by the reigning political authority.)
As I understand it, Christianity began dramatically expanding even within a few years of Jesus’ crucifixion. Over the next few decades, it became a social/religious organization that grew enough that it became harder and harder for the Roman Empire to simply quash. There was not really an organization for Cold War whistleblowers that had enough resources to meaningfully defend its members from being suppressed or persecuted.