I am the director of Tlön, a small org that translates content related to effective altruism, existential risk, and global priorities research into multiple languages.
After living nomadically for many years, I recently moved back to my native Buenos Aires. Feel free to get in touch if you are visiting BA and would like to grab a coffee or need a place to stay.
Every post, comment, or wiki edit I authored is hereby licensed under a Creative Commons Attribution 4.0 International License.
If you are like me, this comment will leave you perplexed. After a while, I realized that it should not be read as
Plausibly, best people who have ever lived is a much lower bar than best people who have ever lived.
but as
‘Plausibly best people who have ever lived’ is a much lower bar than ‘best people who have ever lived’.
Scott Alexander introduces the ‘noncentral fallacy’ as follows: “X is in a category whose archetypal member gives us a certain emotional reaction. Therefore, we should apply that emotional reaction to X, even though it is not a central category member.”
This post seems like an archetypal instance of the noncentral fallacy.
Intuitively, it seems we should respond differently depending on which of these three possibilities is true:
They think that what they are doing is negative for the world, but do it anyway, because it is good for themselves personally.
They do not think that what they are doing is negative for the world, but they believe this due to motivated cognition.
They do not think that what they are doing is negative for the world, and this belief was not formed in a way that seems suspect.
From an act consequentialist perspective, these differences do not matter intrinsically, but they are still instrumentally relevant.[1]
I don’t mean to suggest that any one of these possibilities is particularly likely, or they they are all plausible. I haven’t followed this incident closely. FWIW, my vague sense is that the Mechanize founders had all expressed skepticism about the standard AI safety arguments for a while, in a way that seems hard to reconcile with (1) or (2).
By the way, the name is ‘Jaime’, not ‘Jamie’. The latter doesn’t exist in Spanish and the two are pronounced completely differently (they share one phoneme out of five, when aligned phoneme by phoneme).
(I thought I should mention it since the two names often look indistinguishable in written form to people who are not aware that they differ.)
Thanks for the clarification. I didn’t mean to be pedantic: I think these discussions are often unclear about the relevant time horizon. Even Bostrom admits (somewhere) that his earlier writing about existential risk left the timeframe unspecified (vaguely talking about “premature” extinction).
On the substantive question, I’m interested in learning more about your reasoning. To me, it seems much more likely that Earth-originating intelligence will go extinct this century than, say, in the 8973th century AD (conditional on survival up to that century). This is because it seems plausible that humanity (or its descendants) will soon develop technology with enough destructive potential to actually kill all intelligence. Then the question becomes whether they will also successfully develop the technology to protect intelligence from being so destroyed. But I don’t think there are decisive arguments for expecting the offense-defense balance to favor either defense or offense (the strongest argument for pessimism, in my view, is stated in the first paragraph of this book review). Do you deny that this technology will be developed “over time horizons that are brief by cosmological standards”? Or are you confident that our capacity to destroy will be outpaced by our capacity to prevent from destruction?
The extinction of all Earth-originating intelligent life (including AIs) seems extremely unlikely.
Extremely unlikely to happen… when? Surely all Earth-originating intelligent life will eventually go extinct, because the universe’s resources are finite.
Here’s another summary. I used Gemini 2.0 Flash (via the API), and this prompt:
The following is a series of comments by Habryka, in which he makes a bunch of criticisms of the effective altruism (EA) movement. Please look at these comments and provide a summary of Habryka’s main criticisms.
Lack of leadership and accountability: He believes EA leadership is causing harm and lacks mechanisms for correcting course.
Emphasis on PR and narrative control: He condemns EA organizations’ risk aversion, guardedness, and attempts to control the narrative around FTX, prioritizing public image over transparency.
Inadequate community health: He laments conformity pressures, fears of reprisal for dissent, and insufficient efforts to cultivate a culture of open disagreement.
Entanglement with FTX: He faults EA leadership, particularly Will MacAskill, for endorsing Sam Bankman-Fried and entangling the movement with FTX despite warnings about SBF’s character.
Hero worship and lack of respect for intellectual leaders: He criticizes the hero worship of MacAskill, contrasting it with MacAskill’s perceived lack of engagement with other intellectual leaders in the community. He sees this as part of a pattern of MacAskill prioritizing popularity and prestige over community health and epistemic integrity.
Misleading communications and lack of transparency: He criticizes CEA for making inaccurate and misleading statements, for omitting crucial context in communications, and for concealing information about funding decisions.
Scaling too quickly and attracting grifters: He worries that EA’s rapid growth and increased funding attract deceptive actors and create perverse incentives.
Overreliance on potentially compromised institutions: He expresses concerns about EA’s deep ties to institutions like Oxford University, which may stifle intellectual exploration and operational capacity.
Ignoring internal warnings about FTX: He reveals that he and others warned EA leadership about Sam Bankman-Fried’s reputation for dishonesty, but those warnings went unheeded. He suggests he personally observed potentially illegal activities by SBF but chose not to share this information more widely.
Flawed due diligence and poor judgment in grantmaking: He feels EA leadership’s due diligence on SBF was inadequate and that they made poor judgments in providing him with substantial resources. He extends this criticism to grantmaking practices more generally.
Unfair distribution of resources: He argues that the current distribution of funds within EA doesn’t adequately compensate those doing object-level work and undervalues their contributions relative to donors. He argues for a system that recognizes the implicit tradeoff many have made in pursuing lower-paying EA-aligned careers.
Centralized media policy and negative experiences with journalists: While supporting a less centralized media policy, he also cautions against interacting with journalists, as they frequently misrepresent interviewees and create negative experiences.
I tried to tone down the recent hostile edit, but I agree the current version still leaves much to be desired. Unfortunately, editing the wiki is not a priority for me right now, so somebody else will have to take care of it. As a low-effort alternative, one could revert the page to the last good version: it was just a stub, but it was probably better than nothing. Note that the history page is currently not working well, so I’m not sure how to do the reversion. Tagging @Will Aldred, @Ollie Etherington.
80k has made important contributions to our thinking about career choice, as seen e.g. in their work on replaceability, career capital, personal fit, and the ITN framework. This work does not assume a position on the neartermism vs. longtermism debate, so I think the author’s neartermist sympathies can’t fully explain or justify the omission.
Hello. As it happens, right now I’m editing an interview I conducted with @Jaime Sevilla two months ago. Things got delayed for a variety of reasons, but this episode should be out soon.
Methionine restriction has been shown to increase mean and maximum lifespan in various organisms, particularly rodents. Studies show it can increase lifespan by 30-40% in rats and mice, with effect sizes similar to those of calorie restriction. The lower methionine content of plant-based diets should be seen as a plus rather than a minus, I think.
Thanks for the useful exchange.
It may be useful to consider whether you think your comment would pass a reversal test: if the roles were reversed and it was an EA criticizing another movement, but the criticism was otherwise comparable (e.g. in tone and content), would you also have expressed a broadly positive opinion about it? If yes, that would suggest we are disagreeing about the merits of the letter. If no, it seems it’s primarily a disagreement about the standards we should adopt when evaluating external criticism.
(Note that minutes after posting my comment, I replaced “quality” with “merit”, because it better expresses what I was trying to communicate. However, I don’t think this makes a substantial difference to the point you are raising.)
I think that you managed to articulate clearly what I think is the strongest reason for a certain attitude I see among some EAs, which involves applying different standards to external criticism than to stuff written by members of our community.
Empirically, however, what you say doesn’t ring true to me. My impression is that EA has made progress over time primarily by a process of collective discussion with other EAs, as well as “EA-adjacent” folk like rationalists and forecasters, rather than external critics in the reference class Wenar instantiates. In my opinion, the net effect of such external criticism has, in fact, probably been negative: it has often created polarization and tribalism within the EA community, of the sort that makes it more difficult for us to make intellectual progress, and has misallocated precious community attention, which has in turn slowed down that progress.
So, I’d summarize my position as follows: “Yes, it may be reasonable on priors to expect there to exist critics who can see important problems with EA but who may not be able to articulate that criticism in a way that resonates with us. But our posterior, when we also factor in the evidence supplied by the social and intellectual history of EA, is that there is not much to be gained from engaging with that criticism (criticism that doesn’t seem valuable on its merits), and there is in fact a risk of harm in the form of wasting time and energy on unproductive and acrimonious discussion.”
[I accidentally pressed ‘Comment’ before a previous version of this comment was finished; I have deleted that comment.]
Please remember that the downvote button is not a disagreevote button. Under the circumstances here, it is mainly a reduce visibility / push off the homepage button
I would encourage voters to vote based on your views about the merits of the letter, rather than the effects on its visibility. In general, I think voting based on effects is a kind of “naive consequentialism”, which has worse consequences than voting based on merit when the effects of voting are properly accounted for.
What you call performative open-mindedness (I have been internally referring to it as epistemic virtue signaling) is a very real and important phenomenon, and one I wish people wrote about more and were more aware of.
Do you truly not care that people are accidentally spreading misinformation here?
Why do you attribute to me a view I never stated and do not hold? If I say that one cost is greater than another, it doesn’t mean that I do not care about the lesser cost.
In your original shortform, you listed three separate criticisms, but your reply now focuses on just one of those criticisms, in a way that makes it look that my concerns would be invalidated if one granted the validity of that specific criticism. This is the sort of subtle goalpost moving that makes it difficult to have a productive discussion.
Why are you not more concerned about flawed calculations being spread than about me pointing out that flawed calculations are being spread?
Because there is an asymmetry in the costs of waiting. Waiting a week or so to better understand the alleged problems of a tool that will likely be used for years is a very minor cost, compared to the expected improvement in that understanding that will occur over that period.
(ETA: I didn’t downvote any of your comments, in accordance with my policy of never downvoting comments I reply to, even if I believe I would normally have downvoted them. I mention this only because your most recent comment was downvoted just as I posted this one.)
Yeah, sorry: it was obvious to me that this was the intended meaning, after I realized it could be interpreted this way. I noted it because I found the syntactic ambiguity mildly interesting/amusing.