At the risk of undermining the strategy somewhat, Matt Yglesias said at a recent EA event that efforts to restore US foreign aid have been quietly going well and that it would not be helpful to raise the political salience of the issue.
JoshYou
Karma: 715
Hi Yarrow,
Missed your reply in the other thread. I’ll accept this bet, with $20 to the charity of the winner’s choice. With the modification that since by 2027 Anthropic will probably be a public company that reports quarterly revenue every quarter instead of annualized revenue on an irregular schedule, $50 billion in revenue in calendar Q2 2027 would also count.
So, I propose a bet: if by June 1, 2027, Anthropic has at least $200 billion in annualized revenue, you win. If by June 1, 2027, Anthropic has less than $200 billion in annualized revenue, I win.
I think I’d put a ~60-70% chance that Anthropic exceeds this threshold.
I do have an implicit premise here that LLMs are the sort of thing where as they become more useful, as tracked by revenue, they are more AGI-like. If Jersey Mike’s was growing as fast as Anthropic at the size of Anthropic, that would be a sign of an imminent and dramatic social and economic changes, but those changes wouldn’t be AGI.
I think if LLMs were making $10T or $100T in revenue, close to the size of the current world economy, while still apparently growing and progressing quickly, that would be a strong sign that they were AGI, or had very many of the important elements of AGI, or were otherwise highly transformative. If that happens, it is very unclear what would happen subsequently to AI revenues or to the economy in general. So I’m not appealing to blind and indefinite extrapolation, I’m appealing to the growing likelihood that LLMs will reach the revenue level that would make you think it is pretty much AGI.
So your counterargument works a lot less well as revenue levels get closer to these “evidence of AGI” levels, and revenue is growing extremely quickly with little sign, or in this case negative signs, of slowing down.
As an addendum, I don’t think Anthropic’s revenue growth this year proves that transformative AI is imminent, though I do think it is strongly suggestive. But it has ruled out that LLMs are simply the next of several major tech advances that have happened in my lifetime, such as the Internet or smartphones, which led to c. $1T in annual revenue for the leaders in those industries. LLMs will in fact be much bigger than that, absent some major intervening event such as a enormous catastrophe or global AI moratorium.
To understand just how unprecedented Anthropic’s revenue growth is, I think it’s helpful to consult this analysis/revenue projection for OpenAI published late last year:
Because OpenAI’s growth rates are unprecedented, we can’t compare it to other software companies in a hypergrowth phase when their revenues exceed $4 billion per annum. There are none!
Instead, we look at proxies. Data from investor ICONIQ of fast-growing software companies, once their revenues exceed $50 million (yes, m-illion), shows that for every doubling of revenue, their growth rates tend to drop by a third to a half. In other words, we’re flying blind. Their AI companies have broken all records, so I’ve just assumed a similar slowdown in growth rates.
Traditional software companies see growth rates spike, then plateau after 1-2 years of product-market fit. However, OpenAI and Anthropic have maintained over 100% growth well into the multi-billion-dollar scale, something previously thought impossible.
So, it’s very unusual, bordering on unprecedented, for a company to grow its revenue faster than 100% per year when it’s already making multiple billions of annualized revenue. In OpenAI’s case, it grew by around 200-300% in 2025; Anthropic’s was much faster at around 800% in 2025, but at the time this could have been explained away as catch-up growth from a smaller competitor.
What happened in 2026? Anthropic tripled its revenue in one quarter. Annualized, this would be an 8000% growth rate (~80x per year)! Now, it would indeed be stupid to project this growth rate to the entire rest of the year, which would mean $800B annualized, approximately the greatest of any company in the world. But there’s yet another report that Anthropic is imminently approaching $45 billion as of early May, which is consistent with this trend! (1.5x growth in about one month) Anthropic’s 800% growth rate last year was already basically unprecedented, and now it’s growing 10 times faster than that, at a much higher revenue baseline! This is going to slow, maybe in the next month, or perhaps two, or three. But after that, how long will Anthropic continue to grow at >=10x annually? After a year of this, Anthropic would be comparable to the largest tech giants, and it seems pretty safe to forecast that Anthropic will still be growing at over 100% per year at that point.
And the models are still going to get better!
In the US it’s contingent on some quirks of gambling law. Dedicated online sports gambling was broadly legalized in most states a few years before sports came to prediction markets (during the early days of prediction market discourse, online sports betting was mostly banned anyway by a federal law struck down by SCOTUS in 2018). But prediction markets are less regulated than the sportsbooks, e.g. they have an minimum age of 18 while sportsbooks are 21 in most states. Otherwise sports markets on prediction platforms would presumably mostly just displace gambling elsewhere.
Chinese companies look a lot more competitive in AI than they did when Situational Awareness was published, even when accounting for the likely impacts of distillation, which I think counts somewhat towards his thesis. But there is little or no evidence of a scaling-pilled AI push from China. My impression is that overall capex/funding for Chinese AI is pretty tiny compared to for the US and state support/industrial policy for AI or semiconductors doesn’t look like a big deal though I don’t have the full numbers on hand. And if anything the Chinese government is getting in the way of Chinese AI companies by discouraging/delaying Nvidia H200 imports, treating AI chips as a normal industry where helping to build up the national champion Huawei is more important than racing to near-term AGI.
Off hand, METR, Forethought, MIT Future Tech, AI Futures Project, AI Index, HAL, and Artificial Analysis all substantially overlap with us in our research focuses and other work, though no two orgs have exactly the same remit. The list of orgs and individual researchers whose work at least partially overlaps with us is far larger.
Personally I think there is a huge amount of descriptive and forecasting research to be done around AI, far more than any one organization can or should take on. I would welcome more “competitors” and I don’t want anyone who is interested in our research areas to feel like we have these topics “covered”. And I’m confident there are many good critiques to be made of our work and much better analyses to can be done on the questions we’ve tackled.
Indeed, Anthropic is now running a staff share sale with $6B in buyers lined up.
insanely cool!
noticed one bug: when you click “reset to defaults” it resets the country choice, not just parameter values (unexpected to me) but more importantly it keeps the UI pointed to the country you chose. Below, Nigeria is highlighted but the parameter values were reset to Chad’s defaults.
i doubt it will be very consequential for EA either way. I think what matters is the discursive impact (effect on prevailing social opinion) not total viewers. and people don’t care very much about Netflix shows. would be different if it was a movie that got traction.
Note that Anthropic employees can liquidate significant amounts of equity even without an IPO if Anthropic decides to run a major secondary share sale. OpenAI employees recently sold $6.6 billion in shares.
Austin is something of a hub I think?
GPT-5 is an enormous step up from GPT-4 in capabilities, to be sure, setting aside training compute.
If they time the subsidized user push right their model of expected annual recurring revenue is $10B/y and $11B in 2025 is possible
OpenAI says they already hit $10B annual recurring revenue, for what it’s worth. They don’t provide a breakdown but they do say this excludes major one-time deals and the licensing fees Microsoft pays to use OpenAI models in its own products (this is a substantial source of revenue, but I’m guessing OpenAI excludes this to avoid being accused of using wash transactions to juice their numbers: they in turn pay Microsoft for the servers to train and run their models).
Based on OpenAI only having 3M Team+Enterprise+Edu subscribers in May, I don’t think this $10B/year rate was achieved via $1 Team trial subscriptions.
I think you are reading too much into the growth rate of free users. OpenAI has made a recent push into acquiring lots of new free users, e.g. by making signups easier and putting ChatGPT on WhatsApp, which makes their conversion rate look worse. But their revenue, which comes from paid subscribers and API usage, is still growing at a very healthy and relatively steady rate (3x from $3.4B last year, and 10x from $1B in August 2023) and my guess is that it will continue to grow rapidly.
(comment originally posted on Twitter, Cheryl’s response here)
I’ll flag that estimating firm-level training compute with [Epoch AI’s] notable models dataset will produce big underestimates. E.g. with your methodology, OpenAI spent ~4e25 FLOP on training and 1.3e25 FLOP on research in 2023 and 2024. the latter would cost ~$30 million. but we know OpenAI spent at least $1 billion on research in 2024! (also note they spent $1 billion on research compute after amortizing this cost with an undisclosed schedule).
But I don’t have a great sense of how sensitive your results are to this issue.
(this raises other questions: what did OpenAI spend $3 billion in training compute on in 2024? that’s enough for 50 GPT-4 sized models. Maybe my cost accounting is quite different from OpenAI’s. A lot of that “training” compute might really be more experimental)
Note that Thorstad’s arguments apply more against strong longtermism, i.e. that future generations are overwhelmingly or astronomically more important than current generations, not merely that they are important or even much more important than current generations.
They could exclusively deploy their best models internally, or limit the volume of inference that external users can do, if running AI researchers to do R&D is compute-intensive.
There are already present-day versions of this dilemma. OpenAI claims that DeepSeek used OpenAI model outputs to train its own models, and OpenAI doesn’t reveal their reasoning models’ full chains of thought to prevent competitors from using it as training data.
Kinda weird that the story contains an intelligence explosion that happens both incredibly fast and incredibly soon but glosses over how it happens in a single paragraph, in favor of descriptions of nanobots dematerializing people.
I think the resulting backlash would be extremely intense and polarize very many important people (tech CEOs, AI researchers, pro-business politicians, etc etc) against AI safety and taking AI x-risk seriously (see: Sam Altman’s firing had a major negative impact on the tech industry’s view of AI safety for the following year or so, and this would be a far bigger deal). This is not necessarily a decisive factor from a pure safety perspective, but could be a very important one.