A couple months ago I remarked that Sam Bankman-Fried’s trial was scheduled to start in October, and people should prepare for EA to be in the headlines. It turned out that his trial did not actually generate much press for EA, but a month later EA is again making news as a result of recent Open AI board decisions.
A couple quick points:
It is often the case that people’s behavior is much more reasonable than what is presented in the media. It is also sometimes the case that the reality is even stupider than what is presented. We currently don’t know what actually happened, and should hold multiple hypotheses simultaneously.[1]
It’s very hard to predict the outcome of media stories. Here are a few takes I’ve heard; we should consider that any of these could become the dominant narrative.
Vinod Khosla (The Information): “OpenAI’s board members’ religion of ‘effective altruism’ and its misapplication could have set back the world’s path to the tremendous benefits of artificial intelligence”
John Thornhill (Financial Times): One entrepreneur who is close to OpenAI says the board was “incredibly principled and brave” to confront Altman, even if it failed to explain its actions in public. “The board is rightly being attacked for incompetence,” the entrepreneur told me. “But if the new board is composed of normal tech people, then I doubt they’ll take safety issues seriously.”
The Economist: “The chief lesson is the folly of policing technologies using corporate structures … Fortunately for humanity, there are bodies that have a much more convincing claim to represent its interests: elected governments”
The previous point notwithstanding, people’s attention spans are extremely short, and the median outcome of a news story is ~nothing. I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA, and we might expect a similar thing to happen here.[2]
Animal welfare has historically been unique amongst EA causes in having a dedicated lobby who is fighting against it. While we don’t yet have a HumaneWatch for AI Safety, we should be aware that people have strong interests in how AI develops, and this means that stories about AI will be treated differently from those about, say, malaria.
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people you’ve never met nor have any strong feelings about. The flipside of this though is that we get to celebrate the victories of people we’ve never met. Here are a few things posted in the last week that I thought were cool:
The Against Malaria Foundation is in the middle of a nine-month bed net distribution which is expected to prevent 20 million cases of malaria, and about 40,000 deaths. (Rob Mather)
The Shrimp Welfare Project signed an agreement to prevent 125 million shrimps per year from having their eyes cut off and other painful farming practices. (Ula Zarosa)
The Belgian Senate voted to add animal welfare to their Constitution. (Bob Jacobs)
Scott Alexander’s recent post also has a nice summary of victories.
Note that the data collected here does not exclude the possibility that perception of EA was affected in some subcommunities, and it might be the case that some subcommunities (e.g. OpenAI staff) do have a changed opinion, even if the average person’s opinion is unchanged
I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA
Just for the record, I think the evidence you cited there was shoddy, and I think we are seeing continued references to FTX in basically all coverage of the OpenAI situation, showing that it did clearly have a lasting effect on the perception of EA.
Reputation is lazily-evaluated. Yes, if you ask a random person on the street what they think of you, they won’t know, but when your decisions start influencing them, they will start getting informed, and we are seeing really very clear evidence that when people start getting informed, FTX is heavily influencing their opinion.
There are a lot of recent edits on that article by a single editor, apparently a former NY Times reporter (the edit log is public). From the edit summaries, those edits look rather unfriendly, and the article as a whole feels negatively slanted to me. So I’m not sure how much weight I’d give that article specifically.
Sure, here are the top hits for “Effective Altruism OpenAI” (I did no cherry-picking, this was the first search term that I came up with, and I am just going top to bottom). Each one mentions FTX in a way that pretty clearly matters for the overall article:
“AI safety was embraced as an important cause by big-name Silicon Valley figures who believe in effective altruism, including Peter Thiel, Elon Musk and Sam Bankman-Fried, the founder of crypto exchange FTX, who was convicted in early November of a massive fraud.”
Top comment: ” I only learned about EA during the FTX debacle. And was unaware until recently of its focus on AI. Since been reading and catching up …”
“Coming just weeks after effective altruism’s most prominent backer, Sam Bankman-Fried, was convicted of fraud, the OpenAI meltdown delivered another blow to the movement, which believes that carefully crafted artificial-intelligence systems, imbued with the correct human values, will yield a Golden Age—and failure to do so could have apocalyptic consequences.”
“EA is currently being scrutinized due to its association with Sam Bankman-Fried’s crypto scandal, but less has been written about how the ideology is now driving the research agenda in the field of artificial intelligence (AI), creating a race to proliferate harmful systems, ironically in the name of “AI safety.”
“The first was caused by the downfall of convicted crypto fraudster Sam Bankman-Fried, who was once among the leading figures of EA, an ideology that emerged in the elite corridors of Silicon Valley and Oxford University in the 2010s offering an alternative, utilitarian-infused approach to charitable giving.”
Ah yeah sorry, the claim of the post you criticized was not that FTX isn’t mentioned in the press, but rather that those mentions don’t seem to actually have impacted sentiment very much.
I thought when you said “FTX is heavily influencing their opinion” you were referring to changes in sentiment, but possibly I misunderstood you – if you just mean “journalists mention it a lot” then I agree.
You are also welcome to check Twitter mentions or do other analysis of people talking publicly about EA. I don’t think this is a “journalist only” thing. I will take bets you will see a similar pattern.
I actually did that earlier, then realized I should clarify what you were trying to claim. I will copy the results in below, but even though they support the view that FTX was not a huge deal I want to disclaim that this methodology doesn’t seem like it actually gets at the important thing.
But anyway, my original comment text:
As a convenience sample I searched twitter for “effective altruism”. The first reference to FTX doesn’t come until tweet 36, which is a link to this. Honestly it seems mostly like a standard anti-utilitarianism complaint; it feels like FTX isn’t actually the crux.
In contrast, I see 3 e/acc-type criticisms before that, two “I like EA but this AI stuff is too weird” things (including one retweeted by Yann LeCun??), two “EA is tech-bro/not diverse” complaints and one thing about Whytham Abbey.
I just tried to reproduce the Twitter datapoint. Here is the first tweet when I sort by most recent:
Most tweets are negative, mostly referring to the OpenAI thing. Among the top 10 I see three references to FTX. This continues to be quite remarkable, especially given that it’s been more than a year, and these tweets are quite short.
I don’t know what search you did to find a different pattern. Maybe it was just random chance that I got many more than you did.
Top was mostly showing me tweets from people that I follow, so my sense is it was filtered in a personalized way. I am not fully sure how it works, but it didn’t seem the right type of filter.
Yeah, makes sense. Although I just tried doing the “latest” sort and went through the top 40 tweets without seeing a reference to FTX/SBF.
My guess is that this filter just (unsurprisingly) shows you whatever random thing people are talking about on twitter at the moment, and it seems like the random EA-related thing of today is this, which doesn’t mention FTX.
Probably you need some longitudinal data to have this be useful.
So I think there is a real jump of notoriety once the journalistic class knows who you are. And they now know who we are. “EA, the social movement involved in the FTX and OpenAI crises” is not a good epithet.
The previous point notwithstanding, people’s attention spans are extremely short, and the median outcome of a news story is ~nothing. I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA, and we might expect a similar thing to happen here.
I think this is an oversimplification. This effect is largely caused by competing messages; the modern internet optimizes information for memetic fitness e.g. by maximizing emotional intensity or persuasive effect, and people have so much routine exposure to stuff that leads their minds around in various directions that they get wary (or see having strong reactions to anything at all as immature, since a large portion of outcries on the internet are disproportionately from teenagers). This is the main reason why people take things with a grain of salt.
However, overton windows can still undergo big and lasting shifts (this process could also be engineered deliberately long before generative AI emerged, e.g. via clown attacks which exploit social status instincts to consistently hijack any person’s impressions of any targeted concept). The 80,000 hours podcast with Cass Sunstein covered how Overton windows are dominated by vague impressions of what ideas are acceptable or unacceptable to talk about (note: this podcast was from 2019). This dynamic could plausibly strangle EA’s access to fresh talent, and AI safety’s access to mission-critical policy influence, for several years (which would be far too long).
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people you’ve never met nor have any strong feelings about.
On the flip side, johnswentworth actually had a pretty good take on this; that the human brain is instinctively predisposed to over-focus on the risk of their in-group becoming unpopular among everyone else:
First, [AI safety being condemned by the public] sure does sound like the sort of thing which the human brain presents to us as a far larger, more important fact than it actually is. Ingroup losing status? Few things are more prone to distorted perception than that.
Thanks for the helpful comment – I had not seen John’s dialogue and I think he is making a valid point.
Fair point that the lack of impact might not be due to attention span but instead things like having competing messages.
In case you missed it: Angelina Li compiled some growth metrics about EA here; they seem to indicate that FTX’s collapse did not “strangle” EA (though it probably wasn’t good).
Thoughts on the OpenAI Board Decisions
A couple months ago I remarked that Sam Bankman-Fried’s trial was scheduled to start in October, and people should prepare for EA to be in the headlines. It turned out that his trial did not actually generate much press for EA, but a month later EA is again making news as a result of recent Open AI board decisions.
A couple quick points:
It is often the case that people’s behavior is much more reasonable than what is presented in the media. It is also sometimes the case that the reality is even stupider than what is presented. We currently don’t know what actually happened, and should hold multiple hypotheses simultaneously.[1]
It’s very hard to predict the outcome of media stories. Here are a few takes I’ve heard; we should consider that any of these could become the dominant narrative.
Vinod Khosla (The Information): “OpenAI’s board members’ religion of ‘effective altruism’ and its misapplication could have set back the world’s path to the tremendous benefits of artificial intelligence”
John Thornhill (Financial Times): One entrepreneur who is close to OpenAI says the board was “incredibly principled and brave” to confront Altman, even if it failed to explain its actions in public. “The board is rightly being attacked for incompetence,” the entrepreneur told me. “But if the new board is composed of normal tech people, then I doubt they’ll take safety issues seriously.”
The Economist: “The chief lesson is the folly of policing technologies using corporate structures … Fortunately for humanity, there are bodies that have a much more convincing claim to represent its interests: elected governments”
The previous point notwithstanding, people’s attention spans are extremely short, and the median outcome of a news story is ~nothing. I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA, and we might expect a similar thing to happen here.[2]
Animal welfare has historically been unique amongst EA causes in having a dedicated lobby who is fighting against it. While we don’t yet have a HumaneWatch for AI Safety, we should be aware that people have strong interests in how AI develops, and this means that stories about AI will be treated differently from those about, say, malaria.
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people you’ve never met nor have any strong feelings about. The flipside of this though is that we get to celebrate the victories of people we’ve never met. Here are a few things posted in the last week that I thought were cool:
The Against Malaria Foundation is in the middle of a nine-month bed net distribution which is expected to prevent 20 million cases of malaria, and about 40,000 deaths. (Rob Mather)
The Shrimp Welfare Project signed an agreement to prevent 125 million shrimps per year from having their eyes cut off and other painful farming practices. (Ula Zarosa)
The Belgian Senate voted to add animal welfare to their Constitution. (Bob Jacobs)
Scott Alexander’s recent post also has a nice summary of victories.
A collection of prediction markets about this event can be found here.
Note that the data collected here does not exclude the possibility that perception of EA was affected in some subcommunities, and it might be the case that some subcommunities (e.g. OpenAI staff) do have a changed opinion, even if the average person’s opinion is unchanged
Just for the record, I think the evidence you cited there was shoddy, and I think we are seeing continued references to FTX in basically all coverage of the OpenAI situation, showing that it did clearly have a lasting effect on the perception of EA.
Reputation is lazily-evaluated. Yes, if you ask a random person on the street what they think of you, they won’t know, but when your decisions start influencing them, they will start getting informed, and we are seeing really very clear evidence that when people start getting informed, FTX is heavily influencing their opinion.
Thanks! Could you share said evidence? The data sources I cited certainly have limitations, having access to more surveys etc. would be valuable.
The Wikipedia page on effective altruism mentions Bankman-Fried 11 times, and after/during the OpenAI story, it was edited to include a lot of criticism, ~half of which was written after FTX (e.g. it quotes this tweet https://twitter.com/sama/status/1593046526284410880 )
It’s the first place I would go to if I wanted an independent take on “what’s effective altruism?” I expect many others to do the same.
There are a lot of recent edits on that article by a single editor, apparently a former NY Times reporter (the edit log is public). From the edit summaries, those edits look rather unfriendly, and the article as a whole feels negatively slanted to me. So I’m not sure how much weight I’d give that article specifically.
Sure, here are the top hits for “Effective Altruism OpenAI” (I did no cherry-picking, this was the first search term that I came up with, and I am just going top to bottom). Each one mentions FTX in a way that pretty clearly matters for the overall article:
Bloomberg: “What is Effective Altruism? What does it mean for AI?”
“AI safety was embraced as an important cause by big-name Silicon Valley figures who believe in effective altruism, including Peter Thiel, Elon Musk and Sam Bankman-Fried, the founder of crypto exchange FTX, who was convicted in early November of a massive fraud.”
Reddit “I think this was an Effective Altruism (EA) takeover by the OpenAI board”
Top comment: ” I only learned about EA during the FTX debacle. And was unaware until recently of its focus on AI. Since been reading and catching up …”
WSJ: “How a Fervent Belief Split Silicon Valley—and Fueled the Blowup at OpenAI”
“Coming just weeks after effective altruism’s most prominent backer, Sam Bankman-Fried, was convicted of fraud, the OpenAI meltdown delivered another blow to the movement, which believes that carefully crafted artificial-intelligence systems, imbued with the correct human values, will yield a Golden Age—and failure to do so could have apocalyptic consequences.”
Wired: “Effective Altruism Is Pushing a Dangerous Brand of ‘AI Safety’”
“EA is currently being scrutinized due to its association with Sam Bankman-Fried’s crypto scandal, but less has been written about how the ideology is now driving the research agenda in the field of artificial intelligence (AI), creating a race to proliferate harmful systems, ironically in the name of “AI safety.”
Semafor: “The AI industry turns against its favorite philosophy”
“The first was caused by the downfall of convicted crypto fraudster Sam Bankman-Fried, who was once among the leading figures of EA, an ideology that emerged in the elite corridors of Silicon Valley and Oxford University in the 2010s offering an alternative, utilitarian-infused approach to charitable giving.”
Ah yeah sorry, the claim of the post you criticized was not that FTX isn’t mentioned in the press, but rather that those mentions don’t seem to actually have impacted sentiment very much.
I thought when you said “FTX is heavily influencing their opinion” you were referring to changes in sentiment, but possibly I misunderstood you – if you just mean “journalists mention it a lot” then I agree.
You are also welcome to check Twitter mentions or do other analysis of people talking publicly about EA. I don’t think this is a “journalist only” thing. I will take bets you will see a similar pattern.
I actually did that earlier, then realized I should clarify what you were trying to claim. I will copy the results in below, but even though they support the view that FTX was not a huge deal I want to disclaim that this methodology doesn’t seem like it actually gets at the important thing.
But anyway, my original comment text:
As a convenience sample I searched twitter for “effective altruism”. The first reference to FTX doesn’t come until tweet 36, which is a link to this. Honestly it seems mostly like a standard anti-utilitarianism complaint; it feels like FTX isn’t actually the crux.
In contrast, I see 3 e/acc-type criticisms before that, two “I like EA but this AI stuff is too weird” things (including one retweeted by Yann LeCun??), two “EA is tech-bro/not diverse” complaints and one thing about Whytham Abbey.
And this (survey discussed/criticized here):
I just tried to reproduce the Twitter datapoint. Here is the first tweet when I sort by most recent:
Most tweets are negative, mostly referring to the OpenAI thing. Among the top 10 I see three references to FTX. This continues to be quite remarkable, especially given that it’s been more than a year, and these tweets are quite short.
I don’t know what search you did to find a different pattern. Maybe it was just random chance that I got many more than you did.
I used the default sort (“Top”).
(No opinion on which is more useful; I don’t use Twitter much.)
Top was mostly showing me tweets from people that I follow, so my sense is it was filtered in a personalized way. I am not fully sure how it works, but it didn’t seem the right type of filter.
Yeah, makes sense. Although I just tried doing the “latest” sort and went through the top 40 tweets without seeing a reference to FTX/SBF.
My guess is that this filter just (unsurprisingly) shows you whatever random thing people are talking about on twitter at the moment, and it seems like the random EA-related thing of today is this, which doesn’t mention FTX.
Probably you need some longitudinal data to have this be useful.
I would guess too that these two events have made it much easier to reference EA in passing. eg I think this article wouldn’t have been written 18 months ago. https://www.politico.com/news/2023/10/13/open-philanthropy-funding-ai-policy-00121362
So I think there is a real jump of notoriety once the journalistic class knows who you are. And they now know who we are. “EA, the social movement involved in the FTX and OpenAI crises” is not a good epithet.
Upvoted, I’m grateful for the sober analysis.
I think this is an oversimplification. This effect is largely caused by competing messages; the modern internet optimizes information for memetic fitness e.g. by maximizing emotional intensity or persuasive effect, and people have so much routine exposure to stuff that leads their minds around in various directions that they get wary (or see having strong reactions to anything at all as immature, since a large portion of outcries on the internet are disproportionately from teenagers). This is the main reason why people take things with a grain of salt.
However, overton windows can still undergo big and lasting shifts (this process could also be engineered deliberately long before generative AI emerged, e.g. via clown attacks which exploit social status instincts to consistently hijack any person’s impressions of any targeted concept). The 80,000 hours podcast with Cass Sunstein covered how Overton windows are dominated by vague impressions of what ideas are acceptable or unacceptable to talk about (note: this podcast was from 2019). This dynamic could plausibly strangle EA’s access to fresh talent, and AI safety’s access to mission-critical policy influence, for several years (which would be far too long).
On the flip side, johnswentworth actually had a pretty good take on this; that the human brain is instinctively predisposed to over-focus on the risk of their in-group becoming unpopular among everyone else:
Thanks for the helpful comment – I had not seen John’s dialogue and I think he is making a valid point.
Fair point that the lack of impact might not be due to attention span but instead things like having competing messages.
In case you missed it: Angelina Li compiled some growth metrics about EA here; they seem to indicate that FTX’s collapse did not “strangle” EA (though it probably wasn’t good).