Substack isnât perfect, but I feel like for getting more people to read these great posts that would be a clear winner? I would also personally prefer it as it has native audio functionality and fits into my broader reading diet more seamlessly. Also crossposting to EAF seems good for key posts.
OscarDđ¸
Karma: 2,327
Idea for thinking about the future of EAGs and whether to keep them cause-neutral or have separate cause-themed events:
Get (anonymised) swapcard data and look at who had meetings with whom to work out how strong the clustering is. If there are clear groups that had lots of meetings within the group, but few meetings with people from other groups, that is a sign that there should just be a separate event for that group. Whereas if there is no group that can be cleanly split into its own event, that is a sign to keep it together.
One simple metric for this could be: if you split EAG in two (e.g. longtermist vs neartermist or whatever) how many fewer total connections would there be.
(Maybe this analysis already happens!)
A key consideration for me is that earth-originating civilisation first spreading to other galaxies seems likely to be a lock-in, where if the values/âorganising structures of those early space missions are bad, the future seems quite bad in expectation to me.
And it seems likely that such space colonisation will become possible soon.
Three Claude Code skills I find helpful:
Urgent-watch: allows me to close Slack and email, and have Claude run a loop in the background to check every 30 mins if there is anything time-sensitive that means I should exit deep work and reply to things.
Gdoc2md: I work mainly in Google Docs, often with many comments, and the native âdownload as markdownâ feature loses the comments. So I have a skill that take a doc url and downloads it as a word file, which Claude can parse to extract the comments and format them as footnotes in a clean md file for other Claudes to read and edit.
Memo: When I want Claude to do some macrostrategy research/âwriting, it reads a detailed summary of my orgâs beliefs, values, plans, etc. The summary links to many more markdown files with detailed takes on relevant topics. Interestingly, telling Claude to read all the memos and then write something on a new topic produces worse results than just reading the summary and the few most relevant memos.
Iâm thinking of AI-thesis hedge funds like VARA and SALP, but they are difficult to get access to without being very wealthy. My understanding is @Austin is considering setting up a DAF vehicle for people to get exposure to such funds, so I would recommend talking with him.
But I think expecting returns something like 50-100% p.a. would be reasonable, based on historic performance of these funds. As one intuition pump, suppose that AI and robotics and semiconductors and associated supply chains are about 3% of total capital today, and will grow to be the vast majority of capital after an intelligence explosion, then just being exposed to those industries gives ~30x returns over the coming ~decade, and it is likely possible to do better than this with more targeted bets.
But one might think that this is all lunacy and we should rely more on base rates and the efficient market hypothesis. Seems mistaken to me, but I respect that view too.
Nice!
Include the investment rate of return.
This seems like a pretty key thing to include given the huge rates of return seen thus far (and arguably likely to continue) for AI thesis investing.
cG made more AI policy grants than this in 2025! E.g. just from the >$1M bucket, there was also Oxford AI governance Initiative, Global Shield Global Catastrophic Risks Advocacy, Center for a New American Security, GovAI, Tarbell Center for AI Journalism, and maybe others I am missing!
whoops, thanks, fixed
Good point. According to this manifold market an IPO within the next year is ~75% likely.
Fun! (the url at the top points to the /âbirds rather than /âmusic page by mistake)
Anthropic revenue is actually about $20B I believe.
Good point, that seems like a big risk! I expect the fraction of sentience that is (post)human or digital to be quite high, especially compared to today, in the intergalactic future. But improving values wrt wild animals seems important.
A world of intelligence too cheap to meter is more teleological: constraints and tradeoffs that exist now are washed away, and what matters is mainly what people ultimately value. And more people ultimately value animal welfare than animal diswelfare. The main game is wild animals, and the ~only way for things to go well for them is if we build an ASI that can eventually reshape the natural world to be less suffering filled. I think it is very unlikely farmed animal suffering is exported to other galaxies in a major way, because at technological maturity animals will not be the mose efficient way to meet human material needs.
Also, for those of us working in AI governance, âcGâ and âCIGIâ (the Center for International Governance Innovation) sound the same out loud. But in writing, I tend to use cG too.
What AI timelines are highest impact to act on?
I feel torn, and I think it varies a lot depending on your individual circumstances and opportunities. But overall, I think the arguments for prioritising shorter timelines are a bit stronger.
While I donât work in GHD, I still enjoy reading GHD content on the Forum and on Substack. I agree that interesting questions in GHD are far from solved, but I wonder if a lot of the low-hanging intellectual fruit has been picked (your number 5)? I wasnât around in early GiveWell days but I imagine that would have been an amazing time to be thinking about GHD and coming up with lots of new approaches and ideas. I havenât found GiveWellâs research to be that surprising or interesting lately for instance (vibes-based, I donât engage that closely with them anymore).
I would be keen to hear more from CE charities about what things they are learning and what questions they are facing!
Re your solution #2, I think I probably wouldnât want the Forum team to show âfavouritismâ, but the decline of GHD curated posts is interesting, and maybe that should change.
I think the type of early deal that would be most valuable is where the US and China both agree to produce a joint âconsensusâ ASI aligned to âthe goodâ. In more detail:
The US and China, as you note, are unsure who will win, and would be better off making a deal to preserve some minimum amount of future influence. But I think I am more worried than you about the costs of continued multipolarity into space colonisation. You write âEven having two alternative systems might open up the possibility for comparison, healthy competition, and moral trade.â War, threats, and unhealthy (e.g., burning the cosmic commons) competition also seem like important possibilities here.
Instead, I think having a joint superintelligence that coordinates using our cosmic endowment would be better, with some amount of influence within the âmoral parliamentâ of the ASI for each of the US and China.
Just that would be preferable to dividing up the universe into two camps I thinkâit is easier to do moral trades within one agent acting under moral uncertainty than coordinating between two agents.
A better version, though, could involve the US and China agreeing on some core moral precepts, or just a moral reflection process, and then jointly designing a moral curriculum for the proto-ASI including plenty of Western and Chinese texts, and letting the ASI do as it sees fit. Presumably both sides genuinely believe they are right and that an appropriate moral training process for the AI will lead to liberalism/âSocialism with Chinese characteristics. So this exploits the two sides having different credences (where as you note your proposed deals are possible even if both sides have the same credences). This creates a larger surplus for posisble agreements.
Of course, agreeing to create a joint ASI could also have big nearer term benefits, e.g. avoiding racing and slowing down AI progress and investing more in safety.
This proposal is clearly very far outside the overton window currently, but I donât think this is that much worse on feasibility than your proposed great power resource-sharing deals. It also solves the enforcement challenge as well which is convenient since we might have needed to create such a consensus AI to enforce a different sort of deal.
I am tentatively excited about this proposal, but I expect there isnât much to do to further it until the relevant parties are taking things more seriously.
Iâm fairly sympathetic to that, but it also feels like one needs to draw a line somewhere and where they have currently drawn it seems not unreasonable to me. Though another place to draw the line kind of on the opposite extreme which could also work is just anyone who supports effective giving and is planning to donate/âsalary sacrifice a lot of their money. Maybe the worry is that is too fuzzy and diluting the core 10% message though.
fyi @Luke Moore đ¸
(I am a new-ish AI policy grantmaker)
Thanks, this is useful to hear. I think intuitively I lean towards your view of this being helpful and good to do. But the received wisdom among grantmakers as far as I can tell (that I have mostly been deferring to) is that it is bad to do this due to grantees overupdating. I think if grantees/âapplicants were perfectly even-keeled and rational this would be a great idea. And many applicants probably would benefit from it. But there will likely be a minority of cases where people (unreasonably) get upset if they donât get funded after seeing an initial high probability. Not sure what to do about this, maybe we can be just like âtough luck, we put a disclaimer saying not to rely on this probability estimate overmuchâ but phrased more nicely.