Buck

• Buck 9 Aug 2026 1:02 UTC

  8 points

  0 ∶ 0

  on: Gen­eral ca­pa­bil­ity—and ca­pa­bil­ities gen­er­ally—have no good y-axis

  Thanks for this post. For the record, basically everyone who works on the metrics you’re talking about agrees with you on this.

  I agree that “revenue” is a pretty nice number. Notably, this metric predicts that Anthropic will have revenue equal to the rest of gross world product in ~three years.

• Buck 30 May 2026 17:39 UTC

  4 points

  10 ∶ 38

  on: My dis­agree­ments with CEA’s ap­proach to stew­ard­ing EA

  Longer-standing EAs also don’t seem to engage much with EA infrastructure. Of the EAs I know who’ve been around for a while, I don’t know any who are part of a local group, and only a small handful actively engage on the Forum.

  As a long-time EA, I don’t engage here because the EA Forum is awful and corrosive: it’s full of dumb criticisms and bitter, jealous people, and it’s hard to imagine anything good coming from engaging with it; I wish that this forum would just shut down. This is maybe interesting as a datapoint on how EA infrastructure can be pathological. It’s not a consequence of me disliking EA.

• Buck 27 Mar 2026 23:15 UTC

  20 points

  1 ∶ 0

  on: Buck’s Shortform

  There’s been some discussion here of the claim that AI capabilities improvements have been a consequence of unsustainable increases in inference compute. Redwood Research Astra fellow Anders Cairns Woodruff has written a great post analyzing the data and disputing this.

• Buck 27 Mar 2026 17:58 UTC

  7 points

  0 ∶ 0

  in reply to: Vasco Grilo🔸’s comment on: Sur­vey of AI safety lead­ers on x-risk, AGI timelines, and re­source al­lo­ca­tion (Feb 2026)

  I would be curious to know what is your median time from weak AGI to artificial superintelligence (ASI) in this question from Metaculus

  I don’t want to take the time to really understand the operationalizations used in that question, sorry—I’m worried that the answer might depend substantially on finicky details. In general I think that the AI Futures Model is pretty similar to my views on how long it will take to go from “weak AGI” to “ASI”, for various definitions of those terms.

  I think AI takeover in the next ten years is like 35% (😔), and conditional on takeover, I mostly agree with this; AI takeover is the main mechanism that causes me to expect massive human fatalities in the next decade.

• Buck 26 Mar 2026 20:44 UTC

  12 points

  3 ∶ 0

  in reply to: Vasco Grilo🔸’s comment on: Sur­vey of AI safety lead­ers on x-risk, AGI timelines, and re­source al­lo­ca­tion (Feb 2026)

  I agree the survey underestimates the variance in AI timelines and risk.

  The AI Futures, which is know for AI 2027, had super broad timelines for artificial superintelligence (ASI) timelines on January 26. The difference between the 90th and 10th percentile was 168 years for Daniel Kokotajlo (2027 to 2195), and 137 years for Eli Lifland (2028 to 2165).

  Note that that is a different kind of distribution (one person’s beliefs) than the one reported here (many people’s medians)

• Buck 1 Feb 2026 3:05 UTC

  29 points

  0 ∶ 0

  on: Buck’s Shortform

  @Ryan Greenblatt and I are going to record another podcast together (see the previous one here). We’d love to hear topics that you’d like us to discuss. (The questions people proposed last time are here, for reference.) We’re most likely to discuss issues related to AI, but a broad set of topics other than “preventing AI takeover” are on topic. E.g. last time we talked about the cost to the far future of humans making bad decisions about what to do with AI, and the risk of galactic scale wild animal suffering.

• Buck 17 Jan 2026 23:40 UTC

  3 points

  0 ∶ 0

  in reply to: zeshen🔸’s comment on: Should the AI Safety Com­mu­nity Pri­ori­tize Safety Cases?

  There are so many other risk assessment techniques out there, for reference ISO31010 lists 30 of them (see here) and they’re far from exhaustive.

  Almost nothing on the list you’ve linked is an alternative approach to the same problem safety cases try to solve. E.g. “brainstorming” is obviously not a competitor to safety cases. And safety cases are not even an item in that list!

• Buck 17 Jan 2026 23:38 UTC

  9 points

  1 ∶ 0

  on: Should the AI Safety Com­mu­nity Pri­ori­tize Safety Cases?

  I think EAs are put way too much effort into thinking about safety cases compared to thinking about reducing risks on the margin in cases where risk is much higher (and willingness-to-pay for safety is much lower), because it seems unlikely that willingness-to-pay will be high enough that we’ll have low risk at the relevant point. See e.g. here.

• Buck 10 Nov 2025 21:55 UTC

  22 points

  6 ∶ 1

  in reply to: groundsloth’s comment on: Lack of Diver­sity Within EA

  There’s a social and professional community of Bay Area EAs who work on issues related to transformative AI. People in this cluster tend to have median timelines to transformative AI of 5 to 15 years, tend to think that AI takeover is 5-70% likely, tend to think that we should be fairly cosmopolitan in our altruism.

  People in this cluster mostly don’t post on the EA Forum for a variety of reasons:

  • Many users here don’t seem very well-informed.

  • Lots of users here disagree with me on some of the opinions about AI that I stated above. Obviously it’s totally reasonable for people to disagree on those points, at least before people tell them arguments about them. But it usually doesn’t feel worth my time to argue about those points. I want to spend much more of my time discussing the implications of these basic beliefs than arguing about their probabilities. LessWrong is a much better place for this.

  • The culture here seems pretty toxic. I don’t really feel welcome here. I expect people to treat me with hostility as a result of being moderately influential as an AI safety researcher and executive.

  To be clear, I think it’s a shame that the EA Forum isn’t a better place for people like me to post and comment.

  You can check for yourself that the Bay Area EAs don’t really want to post here by looking up examples of prominent Bay Area EAs and noting that they commented here much more several years ago than they do today.

• Buck 10 Nov 2025 1:08 UTC

  13 points

  9 ∶ 0

  in reply to: huw’s comment on: Lack of Diver­sity Within EA

  Anecdotally, the EA forum skews [...] more Bay Area.

  For what it’s worth, this is not my impression at all. Bay Area EAs (e.g. me) mostly consider the EA Forum to be very unrepresentative of their perspective, to the extent that it’s very rarely worthwhile to post here (which is why they often post on LessWrong instead).

• Buck 10 Nov 2025 1:04 UTC

  7 points

  5 ∶ 1

  in reply to: Yarrow Bouchard 🔸’s comment on: Lack of Diver­sity Within EA

  This is not an obscure topic. It’s been written about endlessly! I do not want to encourage people to make top-level posts asking questions before Googling or talking to AIs, especially on this topic.

  I like Claude’s response a lot more than you do. I’m not sure why. I agree that it’s a lot less informative than your response.

  (The post including “This demographic has historically been disconnected from social impact” made me much less inclined to want this person to stick around.)

• Buck 10 Nov 2025 0:24 UTC

  8 points

  5 ∶ 1

  in reply to: Yarrow Bouchard 🔸’s comment on: Lack of Diver­sity Within EA

  I feel like Claude’s answer is totally fine. The original question seemed to me consistent with the asker having read literally nothing on this topic before asking; I think that the content Claude said adds value given that.

• Buck 9 Nov 2025 21:23 UTC

  3 points

  5 ∶ 7

  on: Lack of Diver­sity Within EA

  I’m glad to hear you are inspired by EA’s utilitarian approach to maximizing social impact; I too am inspired by it and I have very much appreciated being involved with EA for the last decade.

  I think you should probably ask questions as basic as this to AIs before asking people to talk to you about them. Here’s what Claude responded with.

  The observation about EA’s demographic skew is accurate and widely acknowledged within the community. A few points worth making:

  On the historical pattern: The claim that white, male, tech-focused demographics are “historically disconnected from social impact” isn’t quite right—these demographics have been heavily involved in philanthropy and social reform movements throughout history (from industrialist philanthropy to the civil rights movement’s diverse coalition). But the observation that EA specifically has a particular demographic concentration is valid.

  Why this pattern exists: Several factors likely contribute:

  • EA grew out of academic philosophy and rationalist communities that had their own demographic patterns

  • The movement’s early focus areas (AI safety, global poverty, animal welfare) and analytical approach appealed to certain demographics more than others

  • Network effects and social clustering naturally amplified initial patterns

  • Geographic concentration in places like the Bay Area and Oxford

  On diversity efforts: EA organizations have made various attempts to broaden participation, though with mixed results. There are efforts around:

  • Outreach to different universities and regions

  • Scholarships and programs aimed at underrepresented groups

  • Discussion of how framing and culture might inadvertently exclude some people

  The harder question: There’s ongoing debate about whether demographic diversity is primarily valuable instrumentally (does it improve EA’s thinking and impact?) or intrinsically (is it important regardless of instrumental benefits?). Different people in EA would answer this differently, and it connects to deeper questions about EA’s core commitments and priorities.

  Worth noting that some core EA principles (like cause impartiality and willingness to update beliefs based on evidence) might themselves be culturally specific in ways the movement doesn’t always recognize.

• Buck 28 Sep 2025 0:22 UTC

  6 points

  3 ∶ 1

  on: A Re­ply to MacAskill on “If Any­one Builds It, Every­one Dies”

  I think that this post summarizes Will’s position extremely inaccurately and unfairly.

Chris­tian home­school­ers in the year 3000

I en­joyed most of IABIED

• Buck 11 Jul 2025 17:29 UTC

  102 points

  9 ∶ 0

  on: Buck’s Shortform

  An excerpt about the creation of PEPFAR, from “Days of Fire” by Peter Baker. I found this moving.

  Another major initiative was shaping up around the same time. Since taking office, Bush had developed an interest in fighting AIDS in Africa. He had agreed to contribute to an international fund battling the disease and later started a program aimed at providing drugs to HIV-infected pregnant women to reduce the chances of transmitting the virus to their babies. But it had only whetted his appetite to do more. “When we did it, it revealed how unbelievably pathetic the U.S. effort was,” Michael Gerson said.

  So Bush asked Bolten to come up with something more sweeping. Gerson was already thought of as “the custodian of compassionate conservatism within the White House,” as Bolten called him, and he took special interest in AIDS, which had killed his college roommate. Bolten assembled key White House policy aides Gary Edson, Jay Lefkowitz, and Kristen Silverberg in his office. In seeking something transformative, the only outsider they called in was Anthony Fauci, the renowned AIDS researcher and director of the National Institute of Allergy and Infectious Diseases.

  “What if money were no object?” Bolten asked. “What would you do?”

  Bolten and the others expected him to talk about research for a vaccine because that was what he worked on.

  “I’d love to have a few billion more dollars for vaccine research,” Fauci said, “but we’re putting a lot of money into it, and I could not give you any assurance that another single dollar spent on vaccine research is going to get us to a vaccine any faster than we are now.”

  Instead, he added, “The thing you can do now is treatment.”

  The development of low-cost drugs meant for the first time the world could get a grip on the disease and stop it from being a death sentence for millions of people. “They need the money now,” Fauci said. “They don’t need a vaccine ten years from now.”

  The aides crafted a plan in secret, keeping it even from Colin Powell and Tommy Thompson, the secretary of health and human services. They were ready for a final presentation to Bush on December 4. Just before heading into the meeting, Bush stopped by the Roosevelt Room to visit with Jewish leaders in town for the annual White House Hanukkah party later that day. The visitors were supportive of Bush’s confrontation with Iraq and showered him with praise. One of them, George Klein, founder of the Republican Jewish Coalition, recalled that his father had been among the Jewish leaders who tried to get Franklin Roosevelt to do more to stop the Holocaust. “I speak for everyone in this room when I say that if you had been president in the forties, there could have been millions of Jews saved,” the younger Klein said.

  Bush choked up at the thought—“You could see his eyes well up,” Klein remembered—and went straight from that meeting to the AIDS meeting, the words ringing in his ears. Lefkowitz, who walked with the president from the Roosevelt Room to the Oval Office, was convinced that sense of moral imperative emboldened Bush as he listened to the arguments about what had shaped up as a $15 billion, five-year program. Daniels and other budget-minded aides “were kind of gasping” about spending so much money, especially with all the costs of the struggle against terrorism and the looming invasion of Iraq. But Bush steered the conversation to aides he knew favored the program, and they argued forcefully for it.

  “Gerson, what do you think?” Bush asked.

  “If we can do this and we don’t, it will be a source of shame,” Gerson said.

  Bush thought so too. So while he mostly wrestled with the coming war, he quietly set in motion one of the most expansive lifesaving programs ever attempted. Somewhere deep inside, the notion of helping the hopeless appealed to a former drinker’s sense of redemption, the belief that nobody was beyond saving.

  “Look, this is one of those moments when we can actually change the lives of millions of people, a whole continent,” he told Lefkowitz after the meeting broke up. “How can we not take this step?”

Les­sons from the Iraq War for AI policy

• Buck 1 Jul 2025 6:35 UTC

  4 points

  1 ∶ 0

  in reply to: hmijail’s comment on: Please re­con­sider your use of adjectives

  I’m not saying we should treat criticisms very differently from non-criticism posts (except that criticisms are generally lower effort and lower value).

• Buck 27 Jun 2025 15:13 UTC

  6 points

  1 ∶ 0

  in reply to: Toby Tremlett🔹’s comment on: In­ter­stel­lar travel will prob­a­bly doom the long-term future

  Interstellar travel will probably doom the long-term future

  Seems false, probably people will just sort out some strategy for enforcing laws (e.g. having AI monitors travel with people and force them not to do stuff).