A couple months ago I remarked that Sam Bankman-Friedâs trial was scheduled to start in October, and people should prepare for EA to be in the headlines. It turned out that his trial did not actually generate much press for EA, but a month later EA is again making news as a result of recent Open AI board decisions.
A couple quick points:
It is often the case that peopleâs behavior is much more reasonable than what is presented in the media. It is also sometimes the case that the reality is even stupider than what is presented. We currently donât know what actually happened, and should hold multiple hypotheses simultaneously.[1]
Itâs very hard to predict the outcome of media stories. Here are a few takes Iâve heard; we should consider that any of these could become the dominant narrative.
Vinod Khosla (The Information): âOpenAIâs board membersâ religion of âeffective altruismâ and its misapplication could have set back the worldâs path to the tremendous benefits of artificial intelligenceâ
John Thornhill (Financial Times): One entrepreneur who is close to OpenAI says the board was âincredibly principled and braveâ to confront Altman, even if it failed to explain its actions in public. âThe board is rightly being attacked for incompetence,â the entrepreneur told me. âBut if the new board is composed of normal tech people, then I doubt theyâll take safety issues seriously.â
The Economist: âThe chief lesson is the folly of policing technologies using corporate structures ⊠Fortunately for humanity, there are bodies that have a much more convincing claim to represent its interests: elected governmentsâ
The previous point notwithstanding, peopleâs attention spans are extremely short, and the median outcome of a news story is ~nothing. Iâve commented before that FTXâs collapse had little effect on the average personâs perception of EA, and we might expect a similar thing to happen here.[2]
Animal welfare has historically been unique amongst EA causes in having a dedicated lobby who is fighting against it. While we donât yet have a HumaneWatch for AI Safety, we should be aware that people have strong interests in how AI develops, and this means that stories about AI will be treated differently from those about, say, malaria.
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people youâve never met nor have any strong feelings about. The flipside of this though is that we get to celebrate the victories of people weâve never met. Here are a few things posted in the last week that I thought were cool:
The Against Malaria Foundation is in the middle of a nine-month bed net distribution which is expected to prevent 20 million cases of malaria, and about 40,000 deaths. (Rob Mather)
The Shrimp Welfare Project signed an agreement to prevent 125 million shrimps per year from having their eyes cut off and other painful farming practices. (Ula Zarosa)
The Belgian Senate voted to add animal welfare to their Constitution. (Bob Jacobs)
Scott Alexanderâs recent post also has a nice summary of victories.
Note that the data collected here does not exclude the possibility that perception of EA was affected in some subcommunities, and it might be the case that some subcommunities (e.g. OpenAI staff) do have a changed opinion, even if the average personâs opinion is unchanged
Iâve commented before that FTXâs collapse had little effect on the average personâs perception of EA
Just for the record, I think the evidence you cited there was shoddy, and I think we are seeing continued references to FTX in basically all coverage of the OpenAI situation, showing that it did clearly have a lasting effect on the perception of EA.
Reputation is lazily-evaluated. Yes, if you ask a random person on the street what they think of you, they wonât know, but when your decisions start influencing them, they will start getting informed, and we are seeing really very clear evidence that when people start getting informed, FTX is heavily influencing their opinion.
There are a lot of recent edits on that article by a single editor, apparently a former NY Times reporter (the edit log is public). From the edit summaries, those edits look rather unfriendly, and the article as a whole feels negatively slanted to me. So Iâm not sure how much weight Iâd give that article specifically.
Sure, here are the top hits for âEffective Altruism OpenAIâ (I did no cherry-picking, this was the first search term that I came up with, and I am just going top to bottom). Each one mentions FTX in a way that pretty clearly matters for the overall article:
âAI safety was embraced as an important cause by big-name Silicon Valley figures who believe in effective altruism, including Peter Thiel, Elon Musk and Sam Bankman-Fried, the founder of crypto exchange FTX, who was convicted in early November of a massive fraud.â
Top comment: â I only learned about EA during the FTX debacle. And was unaware until recently of its focus on AI. Since been reading and catching up âŠâ
âComing just weeks after effective altruismâs most prominent backer, Sam Bankman-Fried, was convicted of fraud, the OpenAI meltdown delivered another blow to the movement, which believes that carefully crafted artificial-intelligence systems, imbued with the correct human values, will yield a Golden Ageâand failure to do so could have apocalyptic consequences.â
âEA is currently being scrutinized due to its association with Sam Bankman-Friedâs crypto scandal, but less has been written about how the ideology is now driving the research agenda in the field of artificial intelligence (AI), creating a race to proliferate harmful systems, ironically in the name of âAI safety.â
âThe first was caused by the downfall of convicted crypto fraudster Sam Bankman-Fried, who was once among the leading figures of EA, an ideology that emerged in the elite corridors of Silicon Valley and Oxford University in the 2010s offering an alternative, utilitarian-infused approach to charitable giving.â
Ah yeah sorry, the claim of the post you criticized was not that FTX isnât mentioned in the press, but rather that those mentions donât seem to actually have impacted sentiment very much.
I thought when you said âFTX is heavily influencing their opinionâ you were referring to changes in sentiment, but possibly I misunderstood you â if you just mean âjournalists mention it a lotâ then I agree.
You are also welcome to check Twitter mentions or do other analysis of people talking publicly about EA. I donât think this is a âjournalist onlyâ thing. I will take bets you will see a similar pattern.
I actually did that earlier, then realized I should clarify what you were trying to claim. I will copy the results in below, but even though they support the view that FTX was not a huge deal I want to disclaim that this methodology doesnât seem like it actually gets at the important thing.
But anyway, my original comment text:
As a convenience sample I searched twitter for âeffective altruismâ. The first reference to FTX doesnât come until tweet 36, which is a link to this. Honestly it seems mostly like a standard anti-utilitarianism complaint; it feels like FTX isnât actually the crux.
In contrast, I see 3 e/âacc-type criticisms before that, two âI like EA but this AI stuff is too weirdâ things (including one retweeted by Yann LeCun??), two âEA is tech-bro/ânot diverseâ complaints and one thing about Whytham Abbey.
I just tried to reproduce the Twitter datapoint. Here is the first tweet when I sort by most recent:
Most tweets are negative, mostly referring to the OpenAI thing. Among the top 10 I see three references to FTX. This continues to be quite remarkable, especially given that itâs been more than a year, and these tweets are quite short.
I donât know what search you did to find a different pattern. Maybe it was just random chance that I got many more than you did.
Top was mostly showing me tweets from people that I follow, so my sense is it was filtered in a personalized way. I am not fully sure how it works, but it didnât seem the right type of filter.
Yeah, makes sense. Although I just tried doing the âlatestâ sort and went through the top 40 tweets without seeing a reference to FTX/âSBF.
My guess is that this filter just (unsurprisingly) shows you whatever random thing people are talking about on twitter at the moment, and it seems like the random EA-related thing of today is this, which doesnât mention FTX.
Probably you need some longitudinal data to have this be useful.
So I think there is a real jump of notoriety once the journalistic class knows who you are. And they now know who we are. âEA, the social movement involved in the FTX and OpenAI crisesâ is not a good epithet.
Thoughts on the OpenAI Board Decisions
A couple months ago I remarked that Sam Bankman-Friedâs trial was scheduled to start in October, and people should prepare for EA to be in the headlines. It turned out that his trial did not actually generate much press for EA, but a month later EA is again making news as a result of recent Open AI board decisions.
A couple quick points:
It is often the case that peopleâs behavior is much more reasonable than what is presented in the media. It is also sometimes the case that the reality is even stupider than what is presented. We currently donât know what actually happened, and should hold multiple hypotheses simultaneously.[1]
Itâs very hard to predict the outcome of media stories. Here are a few takes Iâve heard; we should consider that any of these could become the dominant narrative.
Vinod Khosla (The Information): âOpenAIâs board membersâ religion of âeffective altruismâ and its misapplication could have set back the worldâs path to the tremendous benefits of artificial intelligenceâ
John Thornhill (Financial Times): One entrepreneur who is close to OpenAI says the board was âincredibly principled and braveâ to confront Altman, even if it failed to explain its actions in public. âThe board is rightly being attacked for incompetence,â the entrepreneur told me. âBut if the new board is composed of normal tech people, then I doubt theyâll take safety issues seriously.â
The Economist: âThe chief lesson is the folly of policing technologies using corporate structures ⊠Fortunately for humanity, there are bodies that have a much more convincing claim to represent its interests: elected governmentsâ
The previous point notwithstanding, peopleâs attention spans are extremely short, and the median outcome of a news story is ~nothing. Iâve commented before that FTXâs collapse had little effect on the average personâs perception of EA, and we might expect a similar thing to happen here.[2]
Animal welfare has historically been unique amongst EA causes in having a dedicated lobby who is fighting against it. While we donât yet have a HumaneWatch for AI Safety, we should be aware that people have strong interests in how AI develops, and this means that stories about AI will be treated differently from those about, say, malaria.
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people youâve never met nor have any strong feelings about. The flipside of this though is that we get to celebrate the victories of people weâve never met. Here are a few things posted in the last week that I thought were cool:
The Against Malaria Foundation is in the middle of a nine-month bed net distribution which is expected to prevent 20 million cases of malaria, and about 40,000 deaths. (Rob Mather)
The Shrimp Welfare Project signed an agreement to prevent 125 million shrimps per year from having their eyes cut off and other painful farming practices. (Ula Zarosa)
The Belgian Senate voted to add animal welfare to their Constitution. (Bob Jacobs)
Scott Alexanderâs recent post also has a nice summary of victories.
A collection of prediction markets about this event can be found here.
Note that the data collected here does not exclude the possibility that perception of EA was affected in some subcommunities, and it might be the case that some subcommunities (e.g. OpenAI staff) do have a changed opinion, even if the average personâs opinion is unchanged
Just for the record, I think the evidence you cited there was shoddy, and I think we are seeing continued references to FTX in basically all coverage of the OpenAI situation, showing that it did clearly have a lasting effect on the perception of EA.
Reputation is lazily-evaluated. Yes, if you ask a random person on the street what they think of you, they wonât know, but when your decisions start influencing them, they will start getting informed, and we are seeing really very clear evidence that when people start getting informed, FTX is heavily influencing their opinion.
Thanks! Could you share said evidence? The data sources I cited certainly have limitations, having access to more surveys etc. would be valuable.
The Wikipedia page on effective altruism mentions Bankman-Fried 11 times, and after/âduring the OpenAI story, it was edited to include a lot of criticism, ~half of which was written after FTX (e.g. it quotes this tweet https://ââtwitter.com/ââsama/ââstatus/ââ1593046526284410880 )
Itâs the first place I would go to if I wanted an independent take on âwhatâs effective altruism?â I expect many others to do the same.
There are a lot of recent edits on that article by a single editor, apparently a former NY Times reporter (the edit log is public). From the edit summaries, those edits look rather unfriendly, and the article as a whole feels negatively slanted to me. So Iâm not sure how much weight Iâd give that article specifically.
Sure, here are the top hits for âEffective Altruism OpenAIâ (I did no cherry-picking, this was the first search term that I came up with, and I am just going top to bottom). Each one mentions FTX in a way that pretty clearly matters for the overall article:
Bloomberg: âWhat is Effective Altruism? What does it mean for AI?â
âAI safety was embraced as an important cause by big-name Silicon Valley figures who believe in effective altruism, including Peter Thiel, Elon Musk and Sam Bankman-Fried, the founder of crypto exchange FTX, who was convicted in early November of a massive fraud.â
Reddit âI think this was an Effective Altruism (EA) takeover by the OpenAI boardâ
Top comment: â I only learned about EA during the FTX debacle. And was unaware until recently of its focus on AI. Since been reading and catching up âŠâ
WSJ: âHow a Fervent Belief Split Silicon Valleyâand Fueled the Blowup at OpenAIâ
âComing just weeks after effective altruismâs most prominent backer, Sam Bankman-Fried, was convicted of fraud, the OpenAI meltdown delivered another blow to the movement, which believes that carefully crafted artificial-intelligence systems, imbued with the correct human values, will yield a Golden Ageâand failure to do so could have apocalyptic consequences.â
Wired: âEffective Altruism Is Pushing a Dangerous Brand of âAI Safetyââ
âEA is currently being scrutinized due to its association with Sam Bankman-Friedâs crypto scandal, but less has been written about how the ideology is now driving the research agenda in the field of artificial intelligence (AI), creating a race to proliferate harmful systems, ironically in the name of âAI safety.â
Semafor: âThe AI industry turns against its favorite philosophyâ
âThe first was caused by the downfall of convicted crypto fraudster Sam Bankman-Fried, who was once among the leading figures of EA, an ideology that emerged in the elite corridors of Silicon Valley and Oxford University in the 2010s offering an alternative, utilitarian-infused approach to charitable giving.â
Ah yeah sorry, the claim of the post you criticized was not that FTX isnât mentioned in the press, but rather that those mentions donât seem to actually have impacted sentiment very much.
I thought when you said âFTX is heavily influencing their opinionâ you were referring to changes in sentiment, but possibly I misunderstood you â if you just mean âjournalists mention it a lotâ then I agree.
You are also welcome to check Twitter mentions or do other analysis of people talking publicly about EA. I donât think this is a âjournalist onlyâ thing. I will take bets you will see a similar pattern.
I actually did that earlier, then realized I should clarify what you were trying to claim. I will copy the results in below, but even though they support the view that FTX was not a huge deal I want to disclaim that this methodology doesnât seem like it actually gets at the important thing.
But anyway, my original comment text:
As a convenience sample I searched twitter for âeffective altruismâ. The first reference to FTX doesnât come until tweet 36, which is a link to this. Honestly it seems mostly like a standard anti-utilitarianism complaint; it feels like FTX isnât actually the crux.
In contrast, I see 3 e/âacc-type criticisms before that, two âI like EA but this AI stuff is too weirdâ things (including one retweeted by Yann LeCun??), two âEA is tech-bro/ânot diverseâ complaints and one thing about Whytham Abbey.
And this (survey discussed/âcriticized here):
I just tried to reproduce the Twitter datapoint. Here is the first tweet when I sort by most recent:
Most tweets are negative, mostly referring to the OpenAI thing. Among the top 10 I see three references to FTX. This continues to be quite remarkable, especially given that itâs been more than a year, and these tweets are quite short.
I donât know what search you did to find a different pattern. Maybe it was just random chance that I got many more than you did.
I used the default sort (âTopâ).
(No opinion on which is more useful; I donât use Twitter much.)
Top was mostly showing me tweets from people that I follow, so my sense is it was filtered in a personalized way. I am not fully sure how it works, but it didnât seem the right type of filter.
Yeah, makes sense. Although I just tried doing the âlatestâ sort and went through the top 40 tweets without seeing a reference to FTX/âSBF.
My guess is that this filter just (unsurprisingly) shows you whatever random thing people are talking about on twitter at the moment, and it seems like the random EA-related thing of today is this, which doesnât mention FTX.
Probably you need some longitudinal data to have this be useful.
I would guess too that these two events have made it much easier to reference EA in passing. eg I think this article wouldnât have been written 18 months ago. https://ââwww.politico.com/âânews/ââ2023/ââ10/ââ13/ââopen-philanthropy-funding-ai-policy-00121362
So I think there is a real jump of notoriety once the journalistic class knows who you are. And they now know who we are. âEA, the social movement involved in the FTX and OpenAI crisesâ is not a good epithet.