Now: TYPE III AUDIO.
Previously: 80,000 Hours (2014-15; 2017-2021) Worked on web development, product management, strategy, internal systems, IT security, etc.
Before that: My CV.
Side-projects: Inbox When Ready; Radio Bostrom; The Valmy; Comment Helper for Google Docs.
Cool. I’m interested to read some of these.
Hot take: I think you should change the name.
Current name has several issues:
(1) Confusing: based on the name alone, I’d expect the blog to contain very fundamental criticism of EA ideas or community, rather than criticism that is pretty well in the spirit of the enterprise. I’d also expect it to be more hostile than is justified by values or epistemic norms I share.
(2) Bad associations: I’ve seen the phrase “ineffective altruism” a few times before. All the examples I can remember were in the context of low-quality criticism and vibe-based sniping on Twitter.
(3) Hostile to a key audience: one of your most important audiences probably won’t like the name much. If you’re trying to have a good discussion, it’s usually a bad idea to open by suggesting your interlocutor is misguided or “ineffective”. [1]
The combination of (1)-(3) probably explain why this post doesn’t already have 100 karma (83 at the time of writing). I’d guess the name will reduce engagement and sharing going forwards by at least 10% compared to a neutral name like “Thorstad’s blog”.
Thanks again for starting this. I will follow along with interest.
[1] Unfortunately the name “effective altruism” also does this, because it’s dunking on the foil (regular altruism).
I’m not going to respond to the “show me the evidence” requests for now because I’m short on time and it’s hard to do this well. Also: I think you and most readers can probably identify a bunch of evidence in favour of these takes if you take a while to look.
I’m sorry to hear you’re finding this frustrating. Personally I’m enjoying our exchange because it’s giving me a reason to clarify and write down a bunch of things I’ve been thinking about for a long time, and I’m interested to hear what you and others make of them.
On Twitter I suggested we arrange a time to call. Would you be up for this? If yes, send me a DM.
it wasn’t until philosophers raised the stakes to salience that x-risk started to be taken even close to sufficiently seriously.
I agree that philosophers, especially Derek Parfit, Nick Bostrom and Tyler Cowen*, have helped get this up the agenda. So too have many economists, astronomers, futurists, etc. Philosophers don’t have a monopoly on identifying what matters in practice—in fact they’re usually pretty bad at this.
Same thing goes if we look at social movements instead of individuals: the anti-nuclear bomb and environmental folks may have done more for getting catastrophic risk up the agenda than effective altruism has so far—especially in terms of generating a widespread culture concern and sense of unease, which certainly warmed up the audience for Bostrom, Parfit, and so on.
Effective altruism movement is only just getting started (hopefully), and it has achieved remarkable successes already. So I do think we’re on track to play a critical role, and we have Bostrom and Parfit and Ord and Sidgwick and Cowen to thank for that—along with many, many others.
*Those who don’t see Tyler Cowen as fundamentally a philosopher—perhaps one of the greats, certainly better than Parfit (with whom he collaborated early on)—are not following carefully.
There is no sense to the idea that philosophically-informed decision-making is inherently more risky than philosophically ignorant decision-making. [Quite the opposite: it wasn’t until philosophers raised the stakes to salience that x-risk started to be taken even close to sufficiently seriously.]
I strongly disagree with this. The key reason is: most of the time, norms that have been exposed to evolutionary selection pressures beat explicit “rational reflection” by individual humans. One of the major mistakes of Enlightenment philosophers was to think it is usually the other way around. These mistakes were plausibly a necessary condition for some of the horrific violence that’s taken place since they started trending.
I often run into philosophy graduates who tell me that relying on intuitive moral judgements about particular cases is “arrogant”. I reply by asking “where do these intuitions come from?” The metaphysical realists say “they are truths of reason, underwritten by the non-natural essence of rationality itself”. The naturalists say: “these intuitions were transmitted to you via culture and genetics, itself subject to aeons of evolutionary pressure”. I side with the naturalists, despite all the best arguments for non-naturalism (to my mind, they’re mostly bad!).
One way to think about the 21st century predicament is that we usually learn via trial and error and selection pressures, but this dynamic in a world with modern technology seems unlikely to go well.
I also don’t see any evidence for the claim of EA philosophers having “eroded the boundary between this kind of philosophizing and real-world decision-making”.
Have you visited the 80,000 Hours website recently?
I think that effective altruism centrally involves taking the ideas of philosophers and using them to inform real-world decision-making. I am very glad we’re attempting this, but we must recognise that this is an extraordinarily risky business. Even the wisest humans are unqualified for this role. Many of our attempts are 51:49 bets at best—sometimes worth trying, rarely without grave downside risk, never without an accompanying imperative to listen carefully for feedback from the world. And yes—diverse, hedged experiments in overconfidence also make sense. And no, SBF was not hedged anything like enough to take his 51:49 bets—to the point of blameworthy, perhaps criminal negligence.
A notable exception to the “we’re mostly clueless” situation is: catastrophes are bad. This view passes the “common sense” test, and the “nearly all the reasonable takes on moral philosophy” test too (negative utilitarianism is the notable exception). But our global resource allocation mechanisms are not taking “catastrophes are bad” seriously enough. So, EA—along with other groups and individuals—has a role to play in pushing sensible measures to reduce catastrophic risks up the agenda (as well as the sensible disaster mitigation prep).
(Derek Parfit’s “extinction is much worse than 99.9% wipeout” claim is far more questionable—I put some of my chips on this, but not the majority.)
As you suggest, the transform function from “abstract philosophical idea” to “what do” is complicated and messy, and involves a lot of deference to existing norms and customs. Sadly, I think that many people with a “physics and philosophy” sensibility underrate just how complicated and messy the transform function really has to be. So they sometimes make bad decisions on principle instead of good decisions grounded in messy common sense.
I’m glad you shared the J.S. Mill quote.
…the beliefs which have thus come down are the rules of morality for the multitude, and for the philosopher until he has succeeded in finding better
EAs should not be encouraged to grant themselves practical exception from “the rules of morality for the multitude” if they think of themselves as philosophers. Genius, wise philosophers are extremely rare (cold take: Parfit wasn’t one of them).
To be clear: I am strongly in favour of attempts to act on important insights from philosophy. I just think that this is hard to do well. One reason is that there is a notable minority of “physics and philosophy” folks who should not be made kings, because their “need for systematisation” is so dominant as to be a disastrous impediment for that role.
In my other comment, I shared links to Karnofsky, Beckstead and Cowen expressing views in the spirit of the above. From memory, Carl Shuman is in a similar place, and so are Alexander Berger and Ajeya Cotra.
My impression is that more than half of the most influential people in effective altruism are roughly where they should be on these topics, but some of the top “influencers”, and many of the ”second tier”, are not.
(Views my own. Sword meme credit: the artist currently known as John Stewart Chill.)
There’s literally no possible way to systematize ethics without ending up somewhere puzzling.
Central plank of this perspective: systematizing ethics may not be the best idea, but some kinds of folks have a hard time recognising this. Systematising has its merits but if you find ideological mess hard to tolerate, you shouldn’t be a king.
Related reading:
Karnofsky on worldview diversification.
Karnofsky on sequence vs cluster thinking.
Possibly the most underrated criticism of EA on the EA Forum.
Richard: you wrote…
I agree we should be wary of philosopher-kings. But that’s mostly just because we should be wary of “kings” (or immature dictators) in general.
Two options:
(1) An immature 30 year old king.
(2) An immature 30 year old king who has chosen to spend the last 10 years in a bubble of consequentialist-flavoured philosophy. [1]
Knowing nothing else, I’d pick (1). You?
Option (2) seems much higher variance. Most likely, this guy is a midwit, or perhaps a high IQ idiot. In either case, electing him king probably means serious trouble ahead.
Given more information I can imagine picking (2). I’d be looking for evidence of practical wisdom, whether that’s taking context seriously, Karnofsky-style “cluster thinking”, Burkean worldliness, or the pragmatism of Fat Tony.
See also: Byrne Hobart and Dwarkesh Patel on hardcore believers & monasteries.
[1] I’m drawing a caricature here, to make the point clearer. I’m not sure how accurate (1) would be as a description of SBF, but it seems roughly right.
The key passages (my emphasis):
Longtermism seems weird not because of its critics but because of its proponents: it’s expressed mainly by philosophers, and there are strong incentives in academic philosophy to carry out thought experiments to increasingly bizarre (and thus more interesting) conclusions.
This means that longtermism as a concept has been defined not by run-of-the-mill stuff like donating to nuclear nonproliferation groups, but by the philosophical writings of figures like Nick Bostrom, MacAskill, Greaves, and Nick Beckstead, figures who have risen to prominence in part because of their willingness to expound on extreme ideas.
These are all smart people, but they are philosophers, which means their entire job is to test out theories and frameworks for understanding the world, and try to sort through what those theories and frameworks imply. There are professional incentives to defend surprising or counterintuitive positions, to poke at widely held pieties and components of “common sense morality,” and to develop thought experiments that are memorable and powerful (and because of that, pretty weird).
This isn’t a knock on philosophy; it’s what I studied in college and a field from which I have learned a tremendous amount. It’s good for society to have a space for people to test out strange and surprising concepts. But whatever the boundary-pushing concepts being explored, it’s important not to mistake that exploration for practical decision-making.
[…]
The dominance of academic philosophers in EA, and those philosophers’ increasing attempts to apply these kinds of thought experiments to real life — aided and abetted by the sudden burst of billions into EA, due in large part to figures like Bankman-Fried — has eroded the boundary between this kind of philosophizing and real-world decision-making. Poets, as Percy Shelley wrote, may be the unacknowledged legislators of the world, but EA made the mistake of trying to turn philosophers into the actual legislators of the future. A good start would be more clearly stating that funding priorities, for now, are less “longtermist” in this galaxy-brained Bostrom sense and more about fighting specific existential risks — which is exactly what EA funders are doing in most cases. The philosophers can trod the cosmos, but the funders and advocates should be tethered closer to Earth.
[…]
The problem is utilitarianism free from any guardrails …
Sam Bankman-Fried is a hardcore, pure, uncut Benthamite utilitarian. His mother, Barbara Fried, is an influential philosopher known for her arguments that consequentialist moral theories like utilitarianism that focus on the actual results of individual actions are better suited for the difficult real-world trade-offs one faces in a complex society. Her son apparently took that insight very, very seriously.
Effective altruists aren’t all utilitarians, but the core idea of EA — that you should attempt to act in such a way to promote the greatest human and animal happiness and flourishing achievable — is shot through with consequentialist reasoning. The whole project of trying to do the most good you can implies maximizing, and maximizing of “the good,” and that is the literal definition of consequentialism.
It’s not hard to see the problem here: If you’re intent on maximizing the good, you better know what the good is — and that isn’t easy. “EA is about maximizing a property of the world that we’re conceptually confused about, can’t reliably define or measure, and have massive disagreements about even within EA,” Holden Karnofsky, the co-CEO of Open Philanthropy and a leading figure in the development of effective altruism, wrote in September. “By default, that seems like a recipe for trouble.”
Indeed it was. It looks increasingly likely that Sam Bankman-Fried appears to have engaged in extreme misconduct precisely because he believed in utilitarianism and effective altruism, and that his mostly EA-affiliated colleagues at FTX and Alameda Research went along with the plan for the same reasons.
[…]
I think taking a high-earning job with the explicit aim of donating the money still makes a lot of sense for most big-money options.
But what SBF did was not just quantitatively but qualitatively different from classic “earn to give.” You can make seven figures a year as a trader in a hedge fund, but unless you manage the whole fund, you probably won’t become a billionaire. Bankman-Fried very much wanted to be a billionaire — so he could have more resources to devote to EA giving, if we take him at his word — and to do that, he set up whole new corporations that never would’ve existed without him. Those corporations then engaged in incredibly risky business practices that never would’ve occurred if he and his team hadn’t entered the field. He was not one-for-one replacing another finance bro who would have used the earnings on sushi and strippers rather than altruistic causes. He was building a whole new financial world, with consequences that would be much grander in scale.
And in building this world, he acted like a vulgar utilitarian. Philosophers like to talk about “biting the bullet”: accepting an unsavory implication of a theory you’ve adopted, and arguing that this implication really isn’t that bad.
[…]
Bankman-Fried’s error was an extreme hubris that led him to bite bullets he never should have bitten. He famously told economist Tyler Cowen in a podcast interview that if faced with a game where “51 percent [of the time], you double the Earth out somewhere else; 49 percent, it all disappears,” he’d keep playing the game continually.
This is known as the St. Petersburg paradox, and it’s a confounding problem in probability theory, because it’s true that playing the game creates more happy human lives in expectation (that is, adjusting for probabilities) than not playing. But if you keep playing, you’ll almost certainly wipe out humankind. It’s an example of where normal rules of rationality seem to break down.
But Bankman-Fried was not interested in playing by the normal rules of rationality. Cowen notes that if Bankman-Fried kept this up, he’d almost certainly wipe out the Earth eventually. Bankman-Fried replied, “Well, not necessarily. Maybe you St. Petersburg paradox into an enormously valuable existence. That’s the other option.”
These are fun dorm room arguments. They should not guide the decision-making of an actual financial company, yet there is some evidence they did. An as-yet-unconfirmed account of an Alameda all-hands meeting describes CEO Caroline Ellison explaining to staff that she and Bankman-Fried faced a choice in early summer 2022: either to let Alameda default after some catastrophic losses, or to raid consumer funds at FTX to bolster Alameda. As the researcher David Dalrymple has noted, this was basically her and Bankman-Fried making a “double or nothing” coin flip: By taking this step, they reasoned they could either save Alameda and FTX or lose both (as wound up happening), rather than keep just FTX, as in a scenario where the consumer funds were not raided.
You wrote “while I wait for your response”. What are you referring to? I’m not aware of any comms with you aside from this comment.
Good idea, thanks. I’ve shared to the EA Forum Team.
Some of the winning entries from the EA Criticism and Red Teaming Contest have now been narrated:
Are you really in a race? The cautionary tales of Szilárd and Ellsberg by Haydn Belfield
Does economic growth meaningfully improve well-being? by Vadim Albinsky
Effective altruism in the garden of ends by Tyler Alterman
Notes on effective altruism by Michael Nielsen
Biological Anchors external review by Jennifer Lin
You can listen via the [EA Forum Podcast](https://forum.effectivealtruism.org/posts/K5Snxo5EhgmwJJjR2/announcing-audio-narrations-of-ea-forum-posts-1) or on the individual post pages themselves.
Narrating the top comments for some posts is a good idea. I’ll discuss this with the team.
My quick personal take is that we should start experimenting with this immediately. We should also bear in mind that not all posts will have comments that are above the bar for narration.
I actually suggested we narrate a couple of the replies to the Michael Nielsen post, but I lost track of that thread and I can see that this didn’t happen. I still think we should do this—I’ll discuss with the team.
Great to hear, thanks David!
I share your enthusiasm for author narration, but I think it only makes sense when:
(a) the author has a strong natural talent and motivation for narrating
(b) the author’s time would not be better spent doing something else
Some people love the idea of narrating their own writing, to the point where I slightly worry that some will fail to realise that (b) is true of them. But this is probably overthinking it.
My general take is: if an author is keen to narrate, we’ll briefly assess (a) and (b), and if there’s not an obvious issue there, we’ll gladly help them make a great narration.
The steps you suggested for the EA Forum Podcast sound good to me. I’ll send you an email later today.
Again—thanks for all the work you’ve put into the “OG” EA Forum Podcast B)
Sure, I’ll post back.
I submitted a week ago and they’re taking ages. It’s known that they sometimes take a while but this case seems unusual. Dashboard shows no errors but if it’s not there by Friday I will try to find a Googler to speak to...
In the meantime: TYPE III AUDIO has some advice on how to make a great narration: https://blog.type3.audio/how-to-record-a-great-narration/
Irena: good question!
Personally I’m keen on the idea of post authors narrating their own writing. But I need to check with the EA Forum Team before I share a proper reply. I’ve sent them a note, and I’ll post back here on Friday if not before.
Error: the “See here for audio version” link points to a different Cold Takes post.
The correct link is: https://www.cold-takes.com/high-level-hopes-for-ai-alignment/