I agree that there are no plausible circumstances in which anyone’s relatives will benefit in a way not shared with a larger class of people. However, I do think groups of people differ in ways that are relevant to how important fast AI development vs. more risk-averse AI development is to their interests. Giving undue weight to the interests of a group of people because one’s friends or family are in that group would still raise the concern I expressed above.
One group that—if they were considering their own interests only—might be rationally expected to accept somewhat more risk than the population as a whole are those who are ~50-55+. As Jaime wrote:
For some of my older relatives, it might make a big difference to their health and wellbeing whether AI-fueled explosive growth happens in 10 vs 20 years.
A similar outcome could also happen if (e.g.) the prior generation of my family has passed on, I had young children, and as a result of prioritizing their interests I didn’t give enough weight to older individuals’ desire to have powerful AI soon enough to improve and/or extend their lives.
the prior generation of my family has passed on, I had young children
This seems to suggest that you think the politicians “making the world better for my children” statement would then also be problematic. Do you agree with that?
I’ll be honest, this argument seems a bit too clever. Is the underlying problem with the statement really that it implies a set of motivations that might slightly up-weight a certain age group? One of the comments speaks of “core values” for EA. Is that really a core value? I’m pretty sure I recall reading an argument by McAskill about how actually we should more heavily weight young people in various ways (I think it was voting), for example. I serious doubt most EAs could claim that they literally are distributionally exact in weighting all morally relevant entities in every decision they make. I think the “core value” that exists probably isn’t really this demanding, although I could be wrong.
Prioritising young people often makes sense from an impartial welfare standpoint, because young people have more years left, so there is more welfare to be affected. With voting in particular, it’s the younger people who have to deal with the longer term consequences of any electoral outcome. You see this in climate change related critiques of the Baby Boomer generation.
See eg
“Effective altruism can be defined by four key values: …
2. Impartial altruism: all people count equally — effective altruism aims to give everyone’s interests equal weight, no matter where or when they live. When combined with prioritisation, this often results in focusing on neglected groups…”
Prioritising young people often makes sense from an impartial welfare standpoint
Sure, I think you can make a reasonable argument for that, but if someone disagreed with that, would you say they lack impartiality? To me it seems like something that is up for debate, within the “margin-of-error” of what is meant by impartiality. Two EAs could come down on different sides of that issue and still be in good standing in the community, and wouldn’t be considered to not believe in the general principle of impartiality. Likewise, I think we can interpret Jeff Kaufman’s argument above as expressing a similar view about an individual’s loved-ones. It is within the “margin-of-error” of impartiality to still have a higher degree of concern for loved-ones, even if that might not be living up to the platonic ideal of impartiality.
My point in bringing this up is, the exact reason why the statement in question is bad seems to be shifting a bit over the conversation. Is the core reason that Sevilla’s statement is objectionable really that it might up-weight people in a certain age group?
TFD, I think your analysis is correct and incisive. I’m grateful to you for writing these comments on this post.
It seems clear that if Jaime had different views about the risk-reward of hypothetical 21st century AGI, nobody would be complaining about him loving his family.
Accusing Jaime of “selfishness”, even though he used that term himself in (what I interpret to be) a self-deprecating way, seems really unfair and unreasonable, and just excessively mean. As you and Jeff Kaufman pointed out, many people who are accepted into the EA movement have the same or similar views as Jaime on who to prioritize and so on. These criticisms would not be levied against Jaime if he were not an AI risk skeptic.
The social norms of EA or at least the EA Forum are different today than they were ten years ago. Ten years ago, if you said you only care about people who are either alive today or who will be born in the next 100 years, and you don’t think much about AGI because global poverty seems a lot more important, then you would be fully qualified to be the president of a university EA group, get a job at a meta-EA organization, or represent the views of the EA movement to a public audience.
Today, it seems like there are a lot more people who self-identify as EAs who see focusing on global poverty as more or less a waste of time relative to the only thing that matters, which is that the Singularity is coming in about 2-5 years (unless we take drastic action), and all our efforts should be focused on making sure the Singularity goes good and not bad — including trying to delay it if that helps. People who disagree with this view have not yet been fully excluded from EA but it seems like some people are pretty mean to people who disagree. (I am one of the people who disagrees.)
As a side note, it’s also strange to me that people are treating the founding of Mechanize as if it has a realistic chance to accelerate AGI progress more than a negligible amount — enough of a chance of enough of an acceleration to be genuinely concerning. AI startups are created all the time. Some of them state wildly ambitious goals, like Mechanize. They typically fail to achieve these goals. The startup Vicarious comes to mind.
There are many startups trying to automate various kinds of physical and non-physical labour. Some larger companies like Tesla and Alphabet are also working on this. Why would Mechanize be particularly concerning or be particularly likely to succeed?
The social norms of EA or at least the EA Forum are different today than they were ten years ago. Ten years ago, if you said you only care about people who are either alive today or who will be born in the next 100 years, and you don’t think much about AGI because global poverty seems a lot more important, then you would be fully qualified to be the president of a university EA group, get a job at a meta-EA organization, or represent the views of the EA movement to a public audience.
This isn’t just a social thing, it’s also response to a lot of changes in AI timelines over the past ten years. Back then a lot of us had views like “most experts think powerful AI is far off, I’m not going to sink a bunch of time into how it might affect my various options for doing good”, but as expert views have shifted that makes less sense. While “don’t think much about AGI because global poverty seems a lot more important” is still a reasonable position to hold (ex: people who think we can’t productively influence how AI goes and so we should focus on doing as much good as we can in areas we can affect), I think it requires a good bit more reasoning and thought than it did ten years ago.
(On the other hand, I see “only care about people who are either alive today or who will be born in the next 100 years” as still within the range of common EA views (ex).)
I see it primarily as a social phenomenon because I think the evidence we have today that AGI will arrive by 2030 is less compelling than the evidence we had in 2015 that AGI would arrive by 2030. In 2015, it was a little more plausible that AGI could arrive by 2030 because that was 15 years away and who knows what can happen in 15 years.
Now that 2030 is a little less than 5 years away, AGI by 2030 is a less plausible prediction than it was in 2015 because there’s less time left and it’s more clear it won’t happen.
I don’t think the reasons people believe AGI will arrive by 2030 are primarily based on evidence but are primarily a sociological phenomenon. People were ready to believe this regardless of the evidence going back to Ray Kurzweil’s The Age of Spiritual Machines in 1999 and Eliezer Yudkowsky’s “End-of-the-World Bet” in 2017. People don’t really pay attention to whether the evidence is good or bad, they ignore obvious evidence and arguments against near-term AGI, and they mostly make a choice to ignore or attack people who express disagreement and instead tune into the relentless drumbeat of people agreeing with them. This is sociology, not epistemology.
Don’t believe me? Talk to me again in 5 years and send me a fruit basket. (Or just kick the can down the road and say AGI is coming in 2035...)
Expert opinion has changed? First, expert opinion is not itself evidence, it’s people’s opinions about evidence. What evidence are the experts basing their beliefs on? That seems way more important than someone just saying a number based on an intuition.
Second, expert opinion does not clearly support the idea of near-term AGI.
As of 2023, the expert opinion on AGI was… well, first of all, really confusing. The AI Impacts survey found that the experts believed there is a 50% chance by 2047 that “unaided machines can accomplish every task better and more cheaply than human workers.” And also that there’s a 50% chance that by 2116 “machines could be built to carry out the task better and more cheaply than human workers.” I don’t know why these predictions are 69 years apart.
Regardless, 2047 is sufficiently far away that it might as well be 2057 or 2067 or 2117. This is just people generating a number using a gut feeling. We don’t know how to build AGI and we have no idea how long it will take to figure out how to. No amount of thinking of numbers or saying numbers can escape this fundamental truth.
We actually won’t have to wait long to see that some of the most attention-catching near-term AI predictions are false. Dario Amodei, the CEO of Anthropic (a company that is said to be “literally creating God”), has predicted that by some point between June 2025 and September 2025, 90% of all code will written by AI rather than humans. In late 2025 and early 2026, when it’s clear Dario was wrong about this (when, not if), maybe some people will start to be more skeptical of attention-grabbing expert predictions. But maybe not.
There are already strong signs of AGI discourse being irrational and absurd. On April 16, 2025, Tyler Cowen claimed that OpenAI’s o3 model is AGI and asked, “is April 16th AGI day?”. In a follow-up post on April 17, seemingly in response to criticism, he said, “I don’t mind if you don’t want to call it AGI”, but seemed to affirm he still thinks o3 is AGI.
On one hand, I hope that in 5 years the people who promoted the idea of AGI by 2030 will lose a lot of credibility and maybe will do some soul-searching to figure out how they could be so wrong. On the other hand, there is nothing preventing people from being irrational indefinitely, such as:
Defining whatever exists in 2030 as AGI (Tyler Cowen already did it in 2025, and Ray Kurzweil innovated the technique years ago).
Kicking the can down the road a few years, and repeat as necessary (similar to how Elon Musk has predicted that the Tesla fleet will achieve Level 4⁄5 autonomy in a year every year from 2015 to 2025 and has not given up the game despite his losing streak).
Telling a story in which AGI didn’t happen only because effective altruists or other good actors successfully delayed AGI development.
I think part of the sociological problem is that people are just way too polite about how crazy this all is and how awful the intellectual practices of effective altruists have been on this topic. (Sorry!) So, I’m being blunt about this to try to change that a little.
I see it primarily as a social phenomenon because I think the evidence we have today that AGI will arrive by 2030 is less compelling than the evidence we had in 2015 that AGI would arrive by 2030.
The evidence we have today that there will be AGI by 2030 is clearly dramatically stronger than the evidence we had in 2015 that there would be AGI by 2020, and that is surely the relevant comparison. This is not EA specific—we have been ahead of the curve in thinking AI would be a big deal, but the whole world has updated in this direction, and it would be strange if we hadn’t as well.
My personal take is that there are pretty reasonable arguments that what we have seen in AI/ML since 2015 suggests AI will be a big deal. I like the way I have seen Yoshua Bengio talk about it “over the next few years, or a few decades”. I share the view that either of those possibilities are reasonable. People who are highly confident that something like AGI is going to arrive over the next few years are more confident in this than I am, but I think that view is within the bounds of reasonable interpretation of the evidence. I think it is also with-in the bounds of reasonable to have the opposite view, that something like AGI is most likely further than a few years away.
Don’t believe me? Talk to me again in 5 years and send me a fruit basket. (Or just kick the can down the road and say AGI is coming in 2035...)
I think this is a healthy attitude and that I think is worth appreciating. We may get answers to these questions over the next few years. That seems pretty positive to me. We will be able to resolve some of these disagreements productively by observing what happens. I hope people who have different views now keep this in mind and that the environment is still in a good place for people who disagree now to work together in the future if some of these disagreements get resolved.
I will offer the ea forum internet-points equivalent of a fruit basket to anyone who would like one in the future if we disagree now and in the future they are proven right and I am proven wrong.
I think part of the sociological problem is that people are just way too polite about how crazy this all is and how awful the intellectual practices of effective altruists have been on this topic.
Can you saw what view it is you think is crazy? It seems quite reasonable to me to think that AI is going to be a massive deal and therefore that it would be highly useful to influence how it goes. On other other hand, I think people often over-estimate the robustness of the arguments for any given strategy for how to actually do that influencing. In other words, its reasonable to prioritize AI, but people’s AI takes are often very over-confident.
For what it’s worth, I basically agree with the view that Mechanize is unlikely to be successful at it’s goals:
As a side note, it’s also strange to me that people are treating the founding of Mechanize as if it has a realistic chance to accelerate AGI progress more than a negligible amount — enough of a chance of enough of an acceleration to be genuinely concerning. AI startups are created all the time. Some of them state wildly ambitious goals, like Mechanize. They typically fail to achieve these goals. The startup Vicarious comes to mind.
There are many startups trying to automate various kinds of physical and non-physical labour. Some larger companies like Tesla and Alphabet are also working on this. Why would Mechanize be particularly concerning or be particularly likely to succeed?
It seems clear that if Jaime had different views about the risk-reward of hypothetical 21st century AGI, nobody would be complaining about him loving his family.
I do think this is substantially correct, but I also want to acknowledge that these can be difficult subjects to navigate. I think anyone has done anything wrong, I’m sure I myself have done something similar to this many times. But I do think its worth trying to understand where the central points of disagreement lie, and I think this really is the central disagreement.
On the question of changing EA attitudes towards AI over the years, although I personally think AI will be a big deal, could be dangerous, and those issues are worth of significant attention, I also can certainly see reasons why people might disagree and why those people would have reasonable grievances with decisions by certain EA people and organizations.
An idea that I have pondered for a while about EA is a theory about which “boundaries” a community emphasizes. Although I’ve only ever interacted with EA by reading related content online, my perception is that EA really emphasizes the boundary around the EA community itself, while de-emphasizing the boundaries around individual people or organizations. The issues around Epoch I think demonstrate this. The feeling of betrayal comes from viewing “the community” as central. I think a lot of other cultures that place more emphasize on those other boundaries might react differently. For example, at most companies I have worked at, although certainly they would never be happy to see an employee leave, they wouldn’t view moving to another job as a betrayal, even if an employee went to work for a direct competitor. I personally think placing more emphasis on orgs/individuals rather than the community as a whole could have some benefits, such as with the issue you raise about how to navigate changing views on AI.
Although emphasizing “the community” might seem like its ideal for cooperation, I think it can actually harm cooperation in the presence of substantial disagreements, because it generates dynamics like what is going on here. People feel like they can’t cooperate with people across the disagreement. We will probably see some of these disagreements resolved over the next few years as AI progresses. I for one hope that even if I am wrong I can take any necessary corrections on-board and still work with people who I disagreed with to make positive contributions. Likewise, I hope that if I am right, people who I disagreed with still feel like they can work with me despite that.
As a side note, it’s also strange to me that people are treating the founding of Mechanize as if it has a realistic chance to accelerate AGI progress more than a negligible amount — enough of a chance of enough of an acceleration to be genuinely concerning. AI startups are created all the time. Some of them state wildly ambitious goals, like Mechanize. They typically fail to achieve these goals. The startup Vicarious comes to mind.
I admit I had a similar thought, but I am of two minds about it. On the one hand, I think intentions do matter. I think it is reasonable to point out if you think someone is making a mistake, even if you think ultimately that mistake is unlikely to have a substantial impact because the person is unlikely to succeed in what they are trying to do.
On the other hand, I do think the degree of the reaction and the way that people are generalizing seems like people are almost pricing in the idea that the actions in question have already had a huge impact. So I do wonder if people are kind of over-updating on this specific case for similar reasons to what you mention.
I agree that there are no plausible circumstances in which anyone’s relatives will benefit in a way not shared with a larger class of people. However, I do think groups of people differ in ways that are relevant to how important fast AI development vs. more risk-averse AI development is to their interests. Giving undue weight to the interests of a group of people because one’s friends or family are in that group would still raise the concern I expressed above.
One group that—if they were considering their own interests only—might be rationally expected to accept somewhat more risk than the population as a whole are those who are ~50-55+. As Jaime wrote:
A similar outcome could also happen if (e.g.) the prior generation of my family has passed on, I had young children, and as a result of prioritizing their interests I didn’t give enough weight to older individuals’ desire to have powerful AI soon enough to improve and/or extend their lives.
This seems to suggest that you think the politicians “making the world better for my children” statement would then also be problematic. Do you agree with that?
I’ll be honest, this argument seems a bit too clever. Is the underlying problem with the statement really that it implies a set of motivations that might slightly up-weight a certain age group? One of the comments speaks of “core values” for EA. Is that really a core value? I’m pretty sure I recall reading an argument by McAskill about how actually we should more heavily weight young people in various ways (I think it was voting), for example. I serious doubt most EAs could claim that they literally are distributionally exact in weighting all morally relevant entities in every decision they make. I think the “core value” that exists probably isn’t really this demanding, although I could be wrong.
Prioritising young people often makes sense from an impartial welfare standpoint, because young people have more years left, so there is more welfare to be affected. With voting in particular, it’s the younger people who have to deal with the longer term consequences of any electoral outcome. You see this in climate change related critiques of the Baby Boomer generation.
See eg
“Effective altruism can be defined by four key values: …
2. Impartial altruism: all people count equally — effective altruism aims to give everyone’s interests equal weight, no matter where or when they live. When combined with prioritisation, this often results in focusing on neglected groups…”
https://80000hours.org/2020/08/misconceptions-effective-altruism/
Sure, I think you can make a reasonable argument for that, but if someone disagreed with that, would you say they lack impartiality? To me it seems like something that is up for debate, within the “margin-of-error” of what is meant by impartiality. Two EAs could come down on different sides of that issue and still be in good standing in the community, and wouldn’t be considered to not believe in the general principle of impartiality. Likewise, I think we can interpret Jeff Kaufman’s argument above as expressing a similar view about an individual’s loved-ones. It is within the “margin-of-error” of impartiality to still have a higher degree of concern for loved-ones, even if that might not be living up to the platonic ideal of impartiality.
My point in bringing this up is, the exact reason why the statement in question is bad seems to be shifting a bit over the conversation. Is the core reason that Sevilla’s statement is objectionable really that it might up-weight people in a certain age group?
TFD, I think your analysis is correct and incisive. I’m grateful to you for writing these comments on this post.
It seems clear that if Jaime had different views about the risk-reward of hypothetical 21st century AGI, nobody would be complaining about him loving his family.
Accusing Jaime of “selfishness”, even though he used that term himself in (what I interpret to be) a self-deprecating way, seems really unfair and unreasonable, and just excessively mean. As you and Jeff Kaufman pointed out, many people who are accepted into the EA movement have the same or similar views as Jaime on who to prioritize and so on. These criticisms would not be levied against Jaime if he were not an AI risk skeptic.
The social norms of EA or at least the EA Forum are different today than they were ten years ago. Ten years ago, if you said you only care about people who are either alive today or who will be born in the next 100 years, and you don’t think much about AGI because global poverty seems a lot more important, then you would be fully qualified to be the president of a university EA group, get a job at a meta-EA organization, or represent the views of the EA movement to a public audience.
Today, it seems like there are a lot more people who self-identify as EAs who see focusing on global poverty as more or less a waste of time relative to the only thing that matters, which is that the Singularity is coming in about 2-5 years (unless we take drastic action), and all our efforts should be focused on making sure the Singularity goes good and not bad — including trying to delay it if that helps. People who disagree with this view have not yet been fully excluded from EA but it seems like some people are pretty mean to people who disagree. (I am one of the people who disagrees.)
As a side note, it’s also strange to me that people are treating the founding of Mechanize as if it has a realistic chance to accelerate AGI progress more than a negligible amount — enough of a chance of enough of an acceleration to be genuinely concerning. AI startups are created all the time. Some of them state wildly ambitious goals, like Mechanize. They typically fail to achieve these goals. The startup Vicarious comes to mind.
There are many startups trying to automate various kinds of physical and non-physical labour. Some larger companies like Tesla and Alphabet are also working on this. Why would Mechanize be particularly concerning or be particularly likely to succeed?
This isn’t just a social thing, it’s also response to a lot of changes in AI timelines over the past ten years. Back then a lot of us had views like “most experts think powerful AI is far off, I’m not going to sink a bunch of time into how it might affect my various options for doing good”, but as expert views have shifted that makes less sense. While “don’t think much about AGI because global poverty seems a lot more important” is still a reasonable position to hold (ex: people who think we can’t productively influence how AI goes and so we should focus on doing as much good as we can in areas we can affect), I think it requires a good bit more reasoning and thought than it did ten years ago.
(On the other hand, I see “only care about people who are either alive today or who will be born in the next 100 years” as still within the range of common EA views (ex).)
I see it primarily as a social phenomenon because I think the evidence we have today that AGI will arrive by 2030 is less compelling than the evidence we had in 2015 that AGI would arrive by 2030. In 2015, it was a little more plausible that AGI could arrive by 2030 because that was 15 years away and who knows what can happen in 15 years.
Now that 2030 is a little less than 5 years away, AGI by 2030 is a less plausible prediction than it was in 2015 because there’s less time left and it’s more clear it won’t happen.
I don’t think the reasons people believe AGI will arrive by 2030 are primarily based on evidence but are primarily a sociological phenomenon. People were ready to believe this regardless of the evidence going back to Ray Kurzweil’s The Age of Spiritual Machines in 1999 and Eliezer Yudkowsky’s “End-of-the-World Bet” in 2017. People don’t really pay attention to whether the evidence is good or bad, they ignore obvious evidence and arguments against near-term AGI, and they mostly make a choice to ignore or attack people who express disagreement and instead tune into the relentless drumbeat of people agreeing with them. This is sociology, not epistemology.
Don’t believe me? Talk to me again in 5 years and send me a fruit basket. (Or just kick the can down the road and say AGI is coming in 2035...)
Expert opinion has changed? First, expert opinion is not itself evidence, it’s people’s opinions about evidence. What evidence are the experts basing their beliefs on? That seems way more important than someone just saying a number based on an intuition.
Second, expert opinion does not clearly support the idea of near-term AGI.
As of 2023, the expert opinion on AGI was… well, first of all, really confusing. The AI Impacts survey found that the experts believed there is a 50% chance by 2047 that “unaided machines can accomplish every task better and more cheaply than human workers.” And also that there’s a 50% chance that by 2116 “machines could be built to carry out the task better and more cheaply than human workers.” I don’t know why these predictions are 69 years apart.
Regardless, 2047 is sufficiently far away that it might as well be 2057 or 2067 or 2117. This is just people generating a number using a gut feeling. We don’t know how to build AGI and we have no idea how long it will take to figure out how to. No amount of thinking of numbers or saying numbers can escape this fundamental truth.
We actually won’t have to wait long to see that some of the most attention-catching near-term AI predictions are false. Dario Amodei, the CEO of Anthropic (a company that is said to be “literally creating God”), has predicted that by some point between June 2025 and September 2025, 90% of all code will written by AI rather than humans. In late 2025 and early 2026, when it’s clear Dario was wrong about this (when, not if), maybe some people will start to be more skeptical of attention-grabbing expert predictions. But maybe not.
There are already strong signs of AGI discourse being irrational and absurd. On April 16, 2025, Tyler Cowen claimed that OpenAI’s o3 model is AGI and asked, “is April 16th AGI day?”. In a follow-up post on April 17, seemingly in response to criticism, he said, “I don’t mind if you don’t want to call it AGI”, but seemed to affirm he still thinks o3 is AGI.
On one hand, I hope that in 5 years the people who promoted the idea of AGI by 2030 will lose a lot of credibility and maybe will do some soul-searching to figure out how they could be so wrong. On the other hand, there is nothing preventing people from being irrational indefinitely, such as:
Defining whatever exists in 2030 as AGI (Tyler Cowen already did it in 2025, and Ray Kurzweil innovated the technique years ago).
Kicking the can down the road a few years, and repeat as necessary (similar to how Elon Musk has predicted that the Tesla fleet will achieve Level 4⁄5 autonomy in a year every year from 2015 to 2025 and has not given up the game despite his losing streak).
Telling a story in which AGI didn’t happen only because effective altruists or other good actors successfully delayed AGI development.
I think part of the sociological problem is that people are just way too polite about how crazy this all is and how awful the intellectual practices of effective altruists have been on this topic. (Sorry!) So, I’m being blunt about this to try to change that a little.
The evidence we have today that there will be AGI by 2030 is clearly dramatically stronger than the evidence we had in 2015 that there would be AGI by 2020, and that is surely the relevant comparison. This is not EA specific—we have been ahead of the curve in thinking AI would be a big deal, but the whole world has updated in this direction, and it would be strange if we hadn’t as well.
My personal take is that there are pretty reasonable arguments that what we have seen in AI/ML since 2015 suggests AI will be a big deal. I like the way I have seen Yoshua Bengio talk about it “over the next few years, or a few decades”. I share the view that either of those possibilities are reasonable. People who are highly confident that something like AGI is going to arrive over the next few years are more confident in this than I am, but I think that view is within the bounds of reasonable interpretation of the evidence. I think it is also with-in the bounds of reasonable to have the opposite view, that something like AGI is most likely further than a few years away.
I think this is a healthy attitude and that I think is worth appreciating. We may get answers to these questions over the next few years. That seems pretty positive to me. We will be able to resolve some of these disagreements productively by observing what happens. I hope people who have different views now keep this in mind and that the environment is still in a good place for people who disagree now to work together in the future if some of these disagreements get resolved.
I will offer the ea forum internet-points equivalent of a fruit basket to anyone who would like one in the future if we disagree now and in the future they are proven right and I am proven wrong.
Can you saw what view it is you think is crazy? It seems quite reasonable to me to think that AI is going to be a massive deal and therefore that it would be highly useful to influence how it goes. On other other hand, I think people often over-estimate the robustness of the arguments for any given strategy for how to actually do that influencing. In other words, its reasonable to prioritize AI, but people’s AI takes are often very over-confident.
For what it’s worth, I basically agree with the view that Mechanize is unlikely to be successful at it’s goals:
I appreciate your comment.
I do think this is substantially correct, but I also want to acknowledge that these can be difficult subjects to navigate. I think anyone has done anything wrong, I’m sure I myself have done something similar to this many times. But I do think its worth trying to understand where the central points of disagreement lie, and I think this really is the central disagreement.
On the question of changing EA attitudes towards AI over the years, although I personally think AI will be a big deal, could be dangerous, and those issues are worth of significant attention, I also can certainly see reasons why people might disagree and why those people would have reasonable grievances with decisions by certain EA people and organizations.
An idea that I have pondered for a while about EA is a theory about which “boundaries” a community emphasizes. Although I’ve only ever interacted with EA by reading related content online, my perception is that EA really emphasizes the boundary around the EA community itself, while de-emphasizing the boundaries around individual people or organizations. The issues around Epoch I think demonstrate this. The feeling of betrayal comes from viewing “the community” as central. I think a lot of other cultures that place more emphasize on those other boundaries might react differently. For example, at most companies I have worked at, although certainly they would never be happy to see an employee leave, they wouldn’t view moving to another job as a betrayal, even if an employee went to work for a direct competitor. I personally think placing more emphasis on orgs/individuals rather than the community as a whole could have some benefits, such as with the issue you raise about how to navigate changing views on AI.
Although emphasizing “the community” might seem like its ideal for cooperation, I think it can actually harm cooperation in the presence of substantial disagreements, because it generates dynamics like what is going on here. People feel like they can’t cooperate with people across the disagreement. We will probably see some of these disagreements resolved over the next few years as AI progresses. I for one hope that even if I am wrong I can take any necessary corrections on-board and still work with people who I disagreed with to make positive contributions. Likewise, I hope that if I am right, people who I disagreed with still feel like they can work with me despite that.
I admit I had a similar thought, but I am of two minds about it. On the one hand, I think intentions do matter. I think it is reasonable to point out if you think someone is making a mistake, even if you think ultimately that mistake is unlikely to have a substantial impact because the person is unlikely to succeed in what they are trying to do.
On the other hand, I do think the degree of the reaction and the way that people are generalizing seems like people are almost pricing in the idea that the actions in question have already had a huge impact. So I do wonder if people are kind of over-updating on this specific case for similar reasons to what you mention.