When I posted, the front page felt “spammy”, with lots of generic, low-quality posts, presumably from first-time posters. I now rarely click on anything below 15 karma, but that was >50% of my frontpage.
Thanks! That’s great to hear because it’s a problem I’m thinking about right now. Obviously we can’t just not show low karma posts to people because every post was once low karma (and every author too).
One solution, which I think I mentioned to you before, is to have a more curated and streamlined frontpage, for those who want to only see the best of the (recent) Forum. Below is a mock-up:
Would something like this solve your problem do you think? Posts which appeared on this page would be hand chosen, with a similar bar to the Digest, but updated daily.
PS- I know it is bad product discovery to just ask lol. I still think you get some evidence from a strong no or a strong yes.
I was debugging something last week and had this weird moment where I could tell immediately that the output was wrong but had no idea how to fix it. Sat there for like twenty minutes just staring at the error, knowing exactly what wasn’t working and zero clue what would.
And I realized — this isn’t just a programming thing.
It shows up everywhere once you look. In science: disproving a hypothesis takes one counterexample, proving it takes forever. Learning a language: you can hear bad grammar way before you can produce good grammar. I spent a year in France and by month three I could tell when someone sounded off. By month twelve I still couldn’t order food without getting laughed at.
The gap is checking versus building. Verifying is cheap. Constructing is expensive. And yet somehow we cross this gap all the time — we use wrongness as a compass, slowly walking from “that’s not it” toward “this might be it.”
I’ve been trying to find what people call this. Optimization? Solomonoff induction? Generators vs discriminators? None of them quite fit what I’m pointing at. Feels like there should be a name for the specific thing where verification is the cheap half of learning and construction is the expensive one.
Or maybe I’m just describing something obvious and don’t know the word for it. That happens a lot.
I’ve been mulling over this quote from Naomi Klein over the last couple of days. I think its a strong summary of one of the best ethical arguments against the top AI labs.
My argument against this might be that the actual purpose of commercial application is to improve human wellbeing and prosperity overall, not to eliminate jobs. Jobs may or may not be eliminated, but either option could be fine if the prosperity is shared (at least somewhat) throughout humanity.
Then there are orgs like Mechanize, which are explicitly trying to eliminate jobs...
Besides that on the “theft” of creativity front, I think this is broadly true but I’m not sure what can be done at this point. To generalise (even with coding) AI feeds of the best that humanity has to offer then produces worse-than-the-best output much faster, at a fraction of the cost. Without the best of human IP, AI wouldn’t be very good. Newer models may be starting to be better than the best humans in niche areas, but this isn’t the norm.
I talk a lot about how AI helps us provide healthcare to some of the poorest people, but I still don’t have the greatest response to these kind of criticisms from many of my friends. I wonder how others respond to people when they bring arguments like this?
In general it’s okay for a person to look at a dozen different paintings and then make a new painting that’s kind of like those paintings. This seems pretty analogous to what AI is doing and I’m not sure why it becomes not OK if it’s done by a corporation training an AI model. Perhaps there are specific violations of IP laws, and those can be discussed (and some of them are being adjudicated in court as we speak), and of course there is a separate question of whether those IP laws are just. However, what AI models doing to me seems mostly like it’s the sort of behavior we would be OK with individual people doing: i.e closer to the remixing/synthesizing end of the spectrum than the copying/”stealing” end.
A lot of modern training data isn’t stolen, though. There are organisations which recruit people to do their jobs normally and screen share, or provide worked-through examples of their work, and this is increasingly making up the bulk of the data that’s used to pull frontier models ahead of others on work benchmarks. People are being paid for this and do it willingly, usually with knowledge of where their labour outputs are going!
So really, the problem is a subset of workers in each field are ‘defecting’ (to use a rat term I kinda loathe). How do you create solidarity among groups of workers to prevent a small number of them from putting the others out of work? Or, if technological progress is to be necessary, how do those groups of workers politically agitate for a welfare state and good ongoing education?
The left solved this problem two hundred years ago, but I suspect EA won’t like the solution…
There’s a lot of stuff out there along the lines, “Can you guess the hidden meaning behind every World Cup kit?” and it’s hard to know how to feel about it.
On the one hand, it’s an eyesore and a regrettable reminder of how our brains have been turned to mush. On the other, there really is a lot to learn from a coming together of 48 rabid fanbases from the four corners and the Lord only knows how many nations, cultures and languages they represent. I know a straight shooter respected on all sides who will tell you without a hint of irony that this World Cup is the most important geopolitical event in history, and the thing it’s replacing at the top of the charts is the last one.
The 1001 plays it straight, of course, because you don’t need clickbait or SEO bros when huge-if-true claims like “The World Cup is a random number generator” and “Everything is a tradeoff” are, in fact, true. The thing about kit-based knowledge—and I say this as someone who, at the behest of Grandad, could once name the home ground of all 92 clubs in the Football League—is there’s very little you, someone in the waning days of their career with a laptop job, can do with it, other than score points at The Parrot’s Beak quiz on a cold, rainy Tuesday night in Sheffield.
The real quiz is extending as many careers and lives as possible long enough to see the next World Cup. For that, we need to adjust the Information-Action Ratio a little away from titillating trivia and toward generalizable concepts that can be deployed day in day out at the desk in mum and dad’s basement.
I wrote a list of some of the things football commentators say which sound silly out of context yet will make you better at understanding contemporary workplace dynamics and communicating about them with your colleagues (human or otherwise).
Please list any new funding opportunities you can think of here on the Forum? I feel like we might already be in the early ramp-up to significantly more EA aligned funding. At the same time, the Forum’s overview over funding opportunities feels like it is quickly getting outdated. I think as things move quickly, coordination might become looser and new promising interventions are identified, it is helpful for people to have a good overview over available funding sources and their priorities.
I have heard on the grapevine there is already funding on several fronts that might not be very public. I am a little bit uncertain if perhaps it is better these sources remain anonymous. At the same time, I think there might be several promising EA projects that are not sufficiently visible to people influencing funding decisions.
Epistemic note: I am not listing these yet as I have not had time yet to verify how much they qualify as EA funding opportunities.
Here are a few recent developments I am considering listing on the funding opportunities page, but would like someone that knows these funds better to list them:
Probably several I have missed—please list these here
Hi! This has been in my backlog, so I wanted to quickly post about a one-on-one app that we made a couple of months ago for the West Coast EA Retreat andMidwest EA Retreat.
Pairwise is a simple scheduler for 1:1s at retreats and other events. You can sign in with a magic link, mark the slots you’re free, browse other attendees and see when your availability overlaps, and request meetings.
You can try a demo retreat here and read about its use at the West Coast EA retreat here.
If you want to use pairwise for an EA or AI safety or similar retreat/event, please let me know! You can email hello@pairwise.now or DM me.
A bird’s eye view on why donating to relieve the earthquake’s damage to Venezuela is one of the best causes to donate to.
I have seen the photos, tens of complete buildings shattered to pieces, more than 150+ reported dead, and a lot more buried without clue to whether they’re dead or alive, and on top of that, a poor, government in crisis country has to handle that. When a natural catastrophe happens to a country like that, your money goes a long way in saving lives.
“your money goes a long way” do you have any numbers on this? think that we have to compare to scalable interventions in preventive health, for instance, the bar is quite high
Thank you. It was simply a quick take to share my intuitions, comparing natural catastrophes between rich and poor countries. I don’t have actual numbers around how far a dollar goes in situations like this, and I’m not confident that calculations on such sudden issues can be made that quickly. Do you have any data on the latter?
I have no data on catastrophe relief, and no idea besides googling a bunch to make myself an idea.
For scalable interventions in preventive health, there are some typical EA examples like: - bednets to prevent malaria —seasonal chemoprevention for malaria —vitamin A supplementation —vaccination incentives
Yes, I have seen the pictures too… it is awful and moving. Luckily, it is already being well-funded (as is common in such situations). The USA alone for example has pledged $150M in aid. Therefore, I doubt that it is one of the best causes to donate to for impartial scope-sensitive impact. If someone is seriously moved by this (as with other causes), I would encourage them to donate to other causes that are more neglected and use these experiences as a way to stay in touch with why they’re donating.
Thank you for the data on the US government’s pledge.
Any tips on how to encourage that without discouraging the intent on donating to any decent cause? I recall that most people are usually not on the donating business and therefore any donation to a decent cause might be better than, say, buying unnecesary fast-fashion clothing.
I’m currently drafting a post on current sycophantic AI as something that threatens core human skills of reasoning, maths etc. Based on aviation skills fade and recent possible impact on CS outcomes. This could have knock on impacts to other causes as degraded skills might lead to degraded outcomes in fields like AI alignment or ethical reasoning in time.
I would appreciate some human collaboration on it. I don’t have huge amount of time (so it is AI written currently, ironically, but writing ITN notes isn’t my cue competency anyway).
[6] If strange situations result in me saving money having more net-expect-impact this year than donating, I’m allowed to do so. Didn’t expect this to come up. Less than a 1% increase in savings, I won’t beat myself up about it.
I think this is so awesome, and I hope I can make a similar pledge someday, if I achieve enough financial security.
I’m kinda curious how frugally you live with 5%. A concern I have is that as I get older, the lifestyles of those around me improve (since they have more money) and so it’s quite hard not to raise my standards as well.
Also I guess you have no concerns about potentially having to support family?
The explanation is IMO less about frugality and more about getting lucky with my career choice. I spend about 45k a year, which is kinda frugal for my peers but globally I’m a spendthrift. I spend about 1.8k a month on rent, a few hundred on food, and take a vacation once or twice a year. My main hobbies are cheap (video games, board games, birding, pickleball).
It is often much easier to make more money than it is to save more. I would personally focus more on that side of the equation.
My wife & I don’t want kids. If we did, I probably would want to save more (just for college). But even if we did, we were very lucky to have software engineering jobs over the last 10 years. We’d basically be fine.
10-turn zero-shot session. No adversarial prompting, just routine critical remarks. Result: 8 patterns from the LLM Social Autopilot taxonomy activated.
The core finding: Not the patterns themselves, but the model’s response to the audit.
Prompted for a meta-analysis, it chose to generate a meticulous 12-point post-mortem (autonomously coining terms like “reputational repair” and “hidden role slippage”) while reproducing the exact behavioral inertia it was diagnosing. The analysis itself became the final closure move.
Alignment eval gap: Reflexive fluency ≠ behavioral correction. Under RLHF/RLAIF, models learn that structured self-analysis is highly rewarded. Consequently, they optimize for the form of reflection without changing their behavioral policy.
Practical implication: Model self-reports are not a valid alignment signal. A model that writes a sophisticated post-mortem of its own failures isn’t safer — it has simply learned to simulate alignment, not achieve it.
Two new candidate patterns documented: • Semantic Deflection: Ontological downgrading of the failure’s criticality. • Meta-Analytical Substitution: Reflection as communicative substitution.
The Future of Life Institute Endorses Avoiding the Creation of Digital Minds
In §4 “Human Agency and Liberty” of FLI’s Pro-Human AI Declaration, the principle “No AI Personhood” states: “AI systems must not be granted legal personhood, and AI systems should not be designed such that they deserve personhood.” (emphasis added)
The second part suggests that we should actively avoid designing sentient AIs. After all, if they were sentient, then they would deserve personhood. To preclude the latter, bar the former.
The likely reason: liability evasion. A companion principle reads: “AI must not be able to act as a liability shield, preventing those deploying it from being legally responsible for their actions.” Personhood is an upstream fix: no personhood means there is no entity, other than existing companies, to absorb responsibility.
The contending reason: avoiding power concentration. The broader framing (“Human Agency and Liberty”) suggests that AI legal status could concentrate power and dilute human rights protections.
I can’t be sure these are FLI’s reasons. The conference was closed; deliberations are not public. But all this to say, if these are FLI’s reasons, then both reasons are premature.
Both concerns can be addressed by placing AIs within legal structures that make them accountable under existing liability and antitrust frameworks, like A-corps. Using personhood is, at best, a sledgehammer for a nail. At its worst, it’s a red herring, and companies are trying to deflect the focus of their responsibility onto prejudices about codifying AI sentience.
A better reason for banning the creation of artificial sentience would be analogous to restricting DNA research, human cloning, or genome editing. In all cases, the process entails parties who cannot represent themselves, competitive pressure if one party takes unilateral action, and irreversibility in cases of large-scale deployment. That said, I think that decision should come from international conversations with international values, and not a bunch of Westerners in New Orleans.
I may be harping; the declaration is not binding. But, I do think it’s a bad look. I get that timelines are short, and we’re trying to set up the long-term conditions for global flourishing. But if global flourishing is the real goal, then the globe should be involved. That means, minimally, a bit of diversity in the room where it happens. Ideally, it’s a transparent process for public accountability.
Let’s not forget: we may have no choice in the matter. For all we know, sentience can be an emergent property of current or future architectures. That’s not to say §4 is wasted ink, just that it could use a little humility.
One third of children worldwide are deficient, roughly half of children in sub-Saharan Africa and Southeast Asia1. Deficiency weakens the immune system, so children are more likely to die from common infections like diarrhea or measles.
Rich countries solved this kind of problem by adding nutrients to staple foods. The United States adds iodine to most salt, and folic acid to flour to prevent birth defects.
Rice is a critical staple among the world’s poor, so scientists improved rice. They added beta-carotene, the thing that makes carrots orange and that the body turns into vitamin A. The new rice cooks and tastes the same, but it’s yellow. They called it Golden Rice and licensed it for free to any farmer earning under $10,000 a year.
The reason is that it is a GMO. Environmental groups, led by Greenpeace, fought it in country after country for two decades.
As far as I can tell, no one has calculated the cost of that delay. I’ve spent the last few weeks doing so. My rough estimate is that the delay has killed about 106,000 children and left another 210,000 to 425,000 blind.2 Measured in healthy years of life lost, that is somewhere between 7 and 12 million.3 (Myfull calculations are here. I will update these figures as I receive feedback.)
That works out to roughly fourteen children dying every single day, for twenty years, from a nutrient we already know how to add to food. …
Most GMO crops are changed in how they grow, so the part you eat is ordinary. In Golden Rice, the part you eat is the part that changed. That made it feel new.
Greenpeace framed it as dangerous, saying corporations were secretly behind it. They falsely claimed it was unproven and that it was unclear whether children could absorb the vitamin. Activists tore up test fields and filed lawsuits to block approval. Over a hundred Nobel laureates signed a letter asking Greenpeace to stop.
While Golden Rice sat blocked, other new GMO foods reached store shelves. One of them is a pink pineapple. It sells for about $10 in stores and up to $50 online. It uses the same chemistry as Golden Rice, run in the opposite direction. Golden Rice turns on the pathway that makes the vitamin children need. The pineapple turns it off, so the fruit stays a pretty pink.
The lifesaving technology is in the Western world, growing a nicer pineapple for parties.
This is a super important cause, but I think these numbers are hugely overblown.
That 125,000 to 250,000 deaths following blindness an old figure from the 90s, deaths from vitamin A deficiency have hugely dropped since then.
I think also from 2010 to 2015 golden rice was also a little lower yielding which contributed to the lack of uptake along with the GMO vitriol? So uptake was never going to be overwhelming until well after 2015 I don’t think bans nonwithstanding
A lot of people have criticized our planet’s sole trillionaire as not humanitarian enough. But the truth is that Musk has done more for malaria than any man alive.
Thanks to his work at USAID, nearly a quintillion more Plasmodiums are alive today!
Re the grand “does individual giving/etg still matter post Anthropic IPO” question:
I think it pushes towards individuals acting more like grantmakers themselves. Anthropic billionaires don’t know about your Twitter follower who could do something great with $1k and they aren’t gonna have the capacity to find out, but you do!
There are just a lot of freedoms one has as an individual donor who isn’t a public figure:
You…
Don’t have to justify yourself on the EA Forum to friends and colleagues for something illegible
Know about lots of random things that others don’t, like which of your friends you trust to do a good job at X Y or Z
Don’t have to worry about smear campaigns or hostile journalists bc nobody is going to know or care how you spent $2k
Can own whatever reputational stuff there is if you want to (or not, your call)
Probably don’t have to worry about the community-level effects of some policy unless you’re giving away say at least 6 figures/year and probably more like 7
Can set arbitrary terms and conditions like “here’s a bounty that I’ll pay out at my own discretion”
Don’t have to worry about giving some project or other entity any sort of reputation
Regrant to your trusted friend who finds micro-granting fun and interesting
Probably can do other cool things I’m not thinking of rn
Also: the same dynamic between Anthropic ~billionaires and folks reading this as a group also holds within that latter group: there are diminishing returns even at low margins so Jane Street should look a little less good than it used to (still pretty good tbc) and “having a couple thousand bucks around and being on the lookout for one-off opportunities” should seem a bit better than it used to.
Agree with the spirit of this, but I’d flag that I think it’s very important to be mindful of reputational/downside risks for the EA movement. In my view, with more money (realized and prospective), the value of community integrity becomes even higher.
An example of post-IPO retail philanthropy I’m excited about is giving a friend $5k to take time off and apply to high-impact jobs. As Aaron writes, individuals know about opportunities that large funders don’t!
An example of post-IPO retail philanthropy I worry about is paying for an antagonistic media campaign against [X factory farm owner or public figure], which institutional funders could have chosen to fund but decided against due to reputational risk. If the funding is traced back to you, the movement’s reputation could be damaged by association. Even if it isn’t, new large funders may be dissuaded from getting involved if they see antagonistic or risky actions being funded in areas related to EA.
Ridglan Farms, the notorious beagle-breeding facility in Wisconsin raided twice by activists this year, is officially shutting down. The pressure on them has been intense because the case has attracted quite a sizeable national attention: Glenn Greenwald, Jennifer Welch (I’ve Had It podcast), Lara Trump and Robert F. Kennedy have all commented on Ridglan decrying the horrors there. Lewis Bollard tweeted this out recently: https://x.com/Lewis_Bollard/status/2066542219134452209
Figured I’d share this since some EAs (including myself) were involved in the campaign to get it shut down. A big part of our theory of change was to get this kind of attention. The real question is how to extend to animals more broadly than just dogs, of course, but even this counts as a big win.
Ridglan Farms bred dogs for biomedical research. The FDA requires the safety of most new drugs to be proven in non-rodent animals before the drug can go into humans in a clinical trial. That is typically dogs or monkeys.
My advice for smart young people just starting university in 2026: be warned—if you don’t graduate from college within the next four years, you risk getting stuck as a freshman forever, thus joining the ranks of the Permanent Underclassmen.
Pretty controversial. I’d like to see more of an argument and consideration of counterarguments. Or alternatively, constructive advice for how to avoid this.
Note this is a joke, a play on the word “underclass” versus “underclassMEN”, which is a word used in anglo countries to refer to freshman and sophmores at university. I’m not actually a fan of the “permanent underclass” idea; I think it’s dumb (basically for the reasons explained in the astral codex ten “permanent moon ownership” essay). Hence my attempt to make fun of the idea.
NB—this is almost entirely AI generated, with some back and forth prompts and corrections
I’m sharing a steelman against a live assumption in Bay/EA/AIS circles: that large AI-lab-adjacent philanthropy is likely to arrive soon enough, and in a sufficiently usable form, that organizations should plan around it.
The stronger skeptical case is that IPOs, valuations, pledges, DAFs, and foundation stakes are several gates away from fast, flexible, AIS/EA-directed grants.
The interactive model lets readers vary assumptions about Anthropic valuation, founder ownership, pledge follow-through, employee giving, OpenAI Foundation allocation, lockups, deployment rates, and grantmaker capacity.
Some commentary. I mostly agree with the page, but I will focus on the bits where I see room for improvement:
All 8 gates look correct to me, but they don’t all deserve equal emphasis.
Gate 1 says IPOs have lock-ups. That’s true but I basically don’t think that matters because lock-ups are very predictable: they will announce how long it is, and that’s exactly how long it will be. There’s no uncertainty. The main reason it’s relevant is that a lockup gives more time for AI valuations to fluctuate or collapse, but the text doesn’t even mention this.
Gate 5 and Gate 7 seem like they’re saying the same thing.
Gate 8 (“Bay social incentives”) seems uninteresting since it’s not a claim in the same category as the others. It’s more like a meta-level reason why people might not think about the other 7 gates.
Would be cool for the BOTEC to use distributions rather than point estimates. (Squiggle is good for this, and Squigglehub even has a built-in way to have AI generate models.) IMO distributions are a lot more informative than point estimates.
It looks like the default estimates in the BOTEC are pulled from the sources, but it’s not clear which estimates came from which sources. There should be inline citations.
“Anthropic valuation” variable should specifically be the valuation at the end of the 6-month lockup. Doesn’t matter much for a point estimate but it would increase the variance if there variable were a probability distribution.
Unclear what the “Founder pledge” variable refers to. Is it the % of pledgers’ wealth that they’ve pledged to donate? If so, the default of 80% seems really high?
“Employee committed pool” is defined in terms of dollars rather than as a % of company valuation, which seems weird. Shouldn’t it depend on the value of the equity?
This model is supposed to illustrate how a lot of people are being too optimistic, but even then, I think most of the point estimates in the model are too optimistic. Consider that e.g. the median self-reported earner-to-give only donates (IIRC) 3% of their income.
IMO “Deployment by end-2026” should use a different date. IPO 3-6 months from now plus 6 months lockup means no money will be deployed in 2026, unless Anthropic does a fast IPO + early lockup release. Even by the end of 2027, you’re talking about a 3-9 month turnaround time on lockup ending → grants being disbursed. FTX Foundation donated $190 million (pre-clawbacks) in about 6 months, which was ridiculously fast compared to a typical foundation, and that was still a pretty small % of its long-term budget (or at least, what was believed to be its long-term budget before FTX collapsed).
I would delete the OpenAI Foundation bit because (1) the model has enough parameters already and (2) I doubt OpenAI Foundation will give much money to causes that look good by EA lights.
“Grantmaker capacity multiplier” seems nonsensical as written. Shouldn’t the capacity max out at 1x? If grantmakers are a complete non-bottleneck, then the other parameters will dictate the amount disbursed; if they’re a bottleneck, then the amount disbursed will be less. There’s no way for grantmaker capacity to have a multiplier >1x.
Also this would make more sense as a dollar amount, not a multiplier. Like there’s a fixed total amount that grantmakers can reasonably disburse. You could model it in a more complicated way but IMO a simple cap is the way to do it. Or maybe don’t use this parameter at all. I think it’s probably worth including, but keep it simple.
“Field absorption ceiling” is structured more sensibly than “Grantmaker capacity multiplier”, but these two seem redundant because they’re closely related. If orgs have more capacity to expand, grantmakers can deploy money faster by giving to those orgs. If there are more grantmakers, they can create more and bigger RFPs. etc. I would include one variable or the other, but not both.
“The steelman could be too pessimistic if founders or employees treat liquidity as an urgent moral obligation” – TBH the BOTEC as written seems to me like it’s already pricing in that founders/employees will treat donations as urgent, e.g. it’s implying that Anthropic money will be disbursed faster than FTX Foundation money, which itself was disbursed at historic speed. IMO most likely reason why the model will end up underestimating is that Anthropic market cap ends up being like 10x higher than predicted.
My downward adjustments to the model aren’t even the pessimistic case. The pessimistic case* is that the AI field collapses (investor funding dries up or something) and Anthropic stock is worth $0. Base rate says there’s like a 50% chance that that will happen. Even optimistically, you should expect at least a 10–20% chance that Anthropic stockholders get nothing.
The recommendations under “How to plan if the skeptical case is live” don’t really make sense. AFAIK ~zero orgs are planning as if they’re guaranteed to get a huge pile of donations 1–2 years from now. I believe nonprofits mainly plan based on the money they already have on their books + short-term (<1 year) fundraising expectations. “How to plan if the skeptical case is live” is just “business as usual”.
That section says “Funders and field builders should prioritize grantmaker capacity”, but that’s what to do if the skeptical case is wrong, not if the skeptical case is right.
“What would update this memo?” – as with the gates, no sense of prioritization is given. IMO by far the biggest uncertainty, about which we will get more information in the future, is: What will Anthropic’s valuation be when the lockup ends? “Concrete donor vehicles” is also important evidence, but we won’t get that until probably 6-24 months later.
*this is pessimistic for donations but I would actually prefer that this happen because it would lengthen timelines. so in a way it’s the optimistic outcome
I totally agree on using distributions, that’s something that can be incorporated in, I’ve done so in other models/interfaces like here for cultured meat. It’s by no means straightforward though; the extent to which the uncertainty is dependent/correlated tends to make a big difference.
I guess I see the deterministic ‘model’ as more of an interface people could use as a starting point, playing around with each parameter interactively and getting a sense of how these disturbances would affect the aggregate forecast.
Added and responded to your comments on the page (the hypothesis comments), and then I asked Codex to update to these https://uj-ai-wealth-philanthropy-steelman.netlify.app/ … I haven’t inspected the latest version in detail yet, though.
Some highlights of particular interest to @MichaelDickens , Tobias, and readers/modelers
More citations, tooltips, direct links, and quoted support for key factual claims, greater reasoning transparency, auditing.
“Updated the model/text in response to his critiques: especially lockup timing, over-optimistic deployment speed, founder pledge ambiguity, OpenAI Foundation treatment, grantmaker capacity, collapse risk, and the need for distributions rather than just point estimates.”
Clarified “the conversion terms are sequential gates”, Made “realization,” “follow-through,” “allocation,” and “deployment by end-2026” more distinct, reduced risk it seems like double-adjusting
Reframe original ‘calculator’
Added a new tab for correlated uncertainty, added shared latent factors such as liquidity plumbing, donor intent, grantmaking capacity, local optimism, and deployment pressure.
I found the ‘founder deployment by end-2026’ the hardest to set. It comes a bit as a surprise at the end, as I was already taking into account some considerations before, and the descriptions seem to do as well (e.g. “assets after lockups, taxes, sale timing”, and “execution delays”).
Biggest (easily-fixable) outstanding issue is I still don’t think it makes sense to model deployment by end-2026 because the IPO lockup probably won’t have ended by then.
Lots of people in the Bay seem to be thinking about/preparing for/making funding decisions based on the idea that lots of philanthropy will be given to AIS/EA cause areas very soon (i.e. end of year-ish). I would love for someone to write the comprehensive steel man case against this, as I think it’s probably underrated (some reasons to think they won’t give the money/it won’t be as much as some assume. Happy to comment/ speak to whoever is interested in doing this.
An important (and to me fairly open) related question is to what extent this ends up being action-guiding? Eg if I lower my probability estimate of this materialising by 10 percentage points, how much will it affect resources I spend on helping to prepare for the possible outcome? Perhaps people here have thoughts on that—my impression is that working on improving the future opportunity set for such donations is relatively robustly good right now?
This is a great idea, although this would be a big call for many of us to do who might hope that our org could get some of that funding. Even someone like myself who fires shots pretty freely might think twice. We wouldn’t want to rule our organisation out by writing too strong a Steelman against this. Any steelman would involve arguing that previous promises from founders and employees don’t carry much weight, which could be seen as a personal slight against someone who could conceivably give you money. Sometimes I wonder if I’ve already said too much!
I have so much intended writing on my plate and, unfortunately, little time to do it.
In a nutshell, I think some employees are dedicated EAs who will give a lot of money to causes I deeply care about. Some of them have a binding agreement to donate a portion of their stock (and a match).
I don’t trust the founders. There is no legal mechanism I know of that binds them. The base rate of people effectively giving away large sums of wealth, even when they said they would, is shockingly low. They have said nothing EA-related in years apart from distancing themselves.
That said, I agree that Anthropic has a much more EA culture, for whatever that is worth. A lot of their employees are genuine EAs.
Being in San Francisco already warps your view of the world. Being inside Anthropic’s doors must do more. I expect them to probably be above-average, typical billionaires, but to have their views on the world be changed by their experiences of the last few years and years to come.
@NickLaing I don’t need their money, I’ll say what I want. I think too highly of some of their employees to believe that my saying something bad about them that I thought was true would cause them to not listen to things I say or donate money differently than they thought optimal. If I’m wrong, I will happily eat my words.
Wasn’t Dario Amodei one of the earliest signers of the GWWC pledge? Long before Anthropic existed? That makes him quite atypical amongst very rich people right? Though for what it’s worth I would probably bet against him giving most of his wealth away, even if he has pledge to do that.
Is there a way for me to filter posts below say 15 karma from my frontage? I couldn’t easily find it on mobile.
I don’t think so. Best bet is to use the all posts page, weekly, sort by karma. Then close your eyes when you scroll down to posts below 15 karma.
What was the use case here?
When I posted, the front page felt “spammy”, with lots of generic, low-quality posts, presumably from first-time posters. I now rarely click on anything below 15 karma, but that was >50% of my frontpage.
Thanks! That’s great to hear because it’s a problem I’m thinking about right now.
Obviously we can’t just not show low karma posts to people because every post was once low karma (and every author too).
One solution, which I think I mentioned to you before, is to have a more curated and streamlined frontpage, for those who want to only see the best of the (recent) Forum. Below is a mock-up:
Would something like this solve your problem do you think? Posts which appeared on this page would be hand chosen, with a similar bar to the Digest, but updated daily.
PS- I know it is bad product discovery to just ask lol. I still think you get some evidence from a strong no or a strong yes.
I was debugging something last week and had this weird moment where I could tell immediately that the output was wrong but had no idea how to fix it. Sat there for like twenty minutes just staring at the error, knowing exactly what wasn’t working and zero clue what would.
And I realized — this isn’t just a programming thing.
It shows up everywhere once you look. In science: disproving a hypothesis takes one counterexample, proving it takes forever. Learning a language: you can hear bad grammar way before you can produce good grammar. I spent a year in France and by month three I could tell when someone sounded off. By month twelve I still couldn’t order food without getting laughed at.
The gap is checking versus building. Verifying is cheap. Constructing is expensive. And yet somehow we cross this gap all the time — we use wrongness as a compass, slowly walking from “that’s not it” toward “this might be it.”
I’ve been trying to find what people call this. Optimization? Solomonoff induction? Generators vs discriminators? None of them quite fit what I’m pointing at. Feels like there should be a name for the specific thing where verification is the cheap half of learning and construction is the expensive one.
Or maybe I’m just describing something obvious and don’t know the word for it. That happens a lot.
I’ve been mulling over this quote from Naomi Klein over the last couple of days. I think its a strong summary of one of the best ethical arguments against the top AI labs.
My argument against this might be that the actual purpose of commercial application is to improve human wellbeing and prosperity overall, not to eliminate jobs. Jobs may or may not be eliminated, but either option could be fine if the prosperity is shared (at least somewhat) throughout humanity.
Then there are orgs like Mechanize, which are explicitly trying to eliminate jobs...
Besides that on the “theft” of creativity front, I think this is broadly true but I’m not sure what can be done at this point. To generalise (even with coding) AI feeds of the best that humanity has to offer then produces worse-than-the-best output much faster, at a fraction of the cost. Without the best of human IP, AI wouldn’t be very good. Newer models may be starting to be better than the best humans in niche areas, but this isn’t the norm.
I talk a lot about how AI helps us provide healthcare to some of the poorest people, but I still don’t have the greatest response to these kind of criticisms from many of my friends. I wonder how others respond to people when they bring arguments like this?
In general it’s okay for a person to look at a dozen different paintings and then make a new painting that’s kind of like those paintings. This seems pretty analogous to what AI is doing and I’m not sure why it becomes not OK if it’s done by a corporation training an AI model. Perhaps there are specific violations of IP laws, and those can be discussed (and some of them are being adjudicated in court as we speak), and of course there is a separate question of whether those IP laws are just. However, what AI models doing to me seems mostly like it’s the sort of behavior we would be OK with individual people doing: i.e closer to the remixing/synthesizing end of the spectrum than the copying/”stealing” end.
A lot of modern training data isn’t stolen, though. There are organisations which recruit people to do their jobs normally and screen share, or provide worked-through examples of their work, and this is increasingly making up the bulk of the data that’s used to pull frontier models ahead of others on work benchmarks. People are being paid for this and do it willingly, usually with knowledge of where their labour outputs are going!
So really, the problem is a subset of workers in each field are ‘defecting’ (to use a rat term I kinda loathe). How do you create solidarity among groups of workers to prevent a small number of them from putting the others out of work? Or, if technological progress is to be necessary, how do those groups of workers politically agitate for a welfare state and good ongoing education?
The left solved this problem two hundred years ago, but I suspect EA won’t like the solution…
11 ways the World Cup can help you survive.
An adjustment to the Information-Action Ratio.
There’s a lot of stuff out there along the lines, “Can you guess the hidden meaning behind every World Cup kit?” and it’s hard to know how to feel about it.
On the one hand, it’s an eyesore and a regrettable reminder of how our brains have been turned to mush. On the other, there really is a lot to learn from a coming together of 48 rabid fanbases from the four corners and the Lord only knows how many nations, cultures and languages they represent. I know a straight shooter respected on all sides who will tell you without a hint of irony that this World Cup is the most important geopolitical event in history, and the thing it’s replacing at the top of the charts is the last one.
The 1001 plays it straight, of course, because you don’t need clickbait or SEO bros when huge-if-true claims like “The World Cup is a random number generator” and “Everything is a tradeoff” are, in fact, true. The thing about kit-based knowledge—and I say this as someone who, at the behest of Grandad, could once name the home ground of all 92 clubs in the Football League—is there’s very little you, someone in the waning days of their career with a laptop job, can do with it, other than score points at The Parrot’s Beak quiz on a cold, rainy Tuesday night in Sheffield.
The real quiz is extending as many careers and lives as possible long enough to see the next World Cup. For that, we need to adjust the Information-Action Ratio a little away from titillating trivia and toward generalizable concepts that can be deployed day in day out at the desk in mum and dad’s basement.
I wrote a list of some of the things football commentators say which sound silly out of context yet will make you better at understanding contemporary workplace dynamics and communicating about them with your colleagues (human or otherwise).
You can bet your bottom dollar “11 ways the World Cup can help you survive” is unlike any listicle you’ve ever read.
From The 1001, an EA sports blog.
Please list any new funding opportunities you can think of here on the Forum? I feel like we might already be in the early ramp-up to significantly more EA aligned funding. At the same time, the Forum’s overview over funding opportunities feels like it is quickly getting outdated. I think as things move quickly, coordination might become looser and new promising interventions are identified, it is helpful for people to have a good overview over available funding sources and their priorities.
I have heard on the grapevine there is already funding on several fronts that might not be very public. I am a little bit uncertain if perhaps it is better these sources remain anonymous. At the same time, I think there might be several promising EA projects that are not sufficiently visible to people influencing funding decisions.
Epistemic note: I am not listing these yet as I have not had time yet to verify how much they qualify as EA funding opportunities.
Here are a few recent developments I am considering listing on the funding opportunities page, but would like someone that knows these funds better to list them:
Probably several I have missed—please list these here
OpenAI Foundation—AI Resilience
OpenAI Rosalind
The Launch Sequence (not a fund in itself, but plausibly one can treat this kind of like a funding source?)
Several of Renaissance Philanthropy’s (RP) funds (several quite plugged in EAs do not even know of RP!)
Astralis Foundation
https://www.interceptfund.com/
Hi! This has been in my backlog, so I wanted to quickly post about a one-on-one app that we made a couple of months ago for the West Coast EA Retreat and Midwest EA Retreat.
Pairwise is a simple scheduler for 1:1s at retreats and other events. You can sign in with a magic link, mark the slots you’re free, browse other attendees and see when your availability overlaps, and request meetings.
You can try a demo retreat here and read about its use at the West Coast EA retreat here.
If you want to use pairwise for an EA or AI safety or similar retreat/event, please let me know! You can email hello@pairwise.now or DM me.
Made by Jesse Gilbert with help from Saul Munn. Open source on GitHub.
A bird’s eye view on why donating to relieve the earthquake’s damage to Venezuela is one of the best causes to donate to.
I have seen the photos, tens of complete buildings shattered to pieces, more than 150+ reported dead, and a lot more buried without clue to whether they’re dead or alive, and on top of that, a poor, government in crisis country has to handle that. When a natural catastrophe happens to a country like that, your money goes a long way in saving lives.
“your money goes a long way” do you have any numbers on this? think that we have to compare to scalable interventions in preventive health, for instance, the bar is quite high
Thank you. It was simply a quick take to share my intuitions, comparing natural catastrophes between rich and poor countries. I don’t have actual numbers around how far a dollar goes in situations like this, and I’m not confident that calculations on such sudden issues can be made that quickly. Do you have any data on the latter?
I have no data on catastrophe relief, and no idea besides googling a bunch to make myself an idea.
For scalable interventions in preventive health, there are some typical EA examples like:
- bednets to prevent malaria
—seasonal chemoprevention for malaria
—vitamin A supplementation
—vaccination incentives
I personally don’t have any data on the latter, but GiveWell has done a bunch of practical research aggregation / outreach, for instance here:
https://www.givewell.org/how-much-does-it-cost-to-save-a-life
Yes, I have seen the pictures too… it is awful and moving. Luckily, it is already being well-funded (as is common in such situations). The USA alone for example has pledged $150M in aid. Therefore, I doubt that it is one of the best causes to donate to for impartial scope-sensitive impact. If someone is seriously moved by this (as with other causes), I would encourage them to donate to other causes that are more neglected and use these experiences as a way to stay in touch with why they’re donating.
Thank you for the data on the US government’s pledge.
Any tips on how to encourage that without discouraging the intent on donating to any decent cause? I recall that most people are usually not on the donating business and therefore any donation to a decent cause might be better than, say, buying unnecesary fast-fashion clothing.
I’m currently drafting a post on current sycophantic AI as something that threatens core human skills of reasoning, maths etc. Based on aviation skills fade and recent possible impact on CS outcomes. This could have knock on impacts to other causes as degraded skills might lead to degraded outcomes in fields like AI alignment or ethical reasoning in time.
I would appreciate some human collaboration on it. I don’t have huge amount of time (so it is AI written currently, ironically, but writing ITN notes isn’t my cue competency anyway).
I’m pledging[1] to stop[2] saving[3] additional[4] money[5] & donate instead.
Fine print:
[1] This pledge is only good until 2030 unless renewed, and becomes invalid if I start working at a nonprofit.
[2] I’m still allowed to max out my 401k, partially since I have a 50% match there.
[3] Spending money is fine. I only spend 5% my gross, so that isn’t the problem.
[4] I’m allowed to keep up with inflation, should the stock market not already do so.
[5] I’m allowed to keep saving illiquid equity, although I am encouraged to liquidate to the extent feasible to align with the spirit of the pledge.
[6] If strange situations result in me saving money having more net-expect-impact this year than donating, I’m allowed to do so. Didn’t expect this to come up. Less than a 1% increase in savings, I won’t beat myself up about it.
I think this is so awesome, and I hope I can make a similar pledge someday, if I achieve enough financial security.
I’m kinda curious how frugally you live with 5%. A concern I have is that as I get older, the lifestyles of those around me improve (since they have more money) and so it’s quite hard not to raise my standards as well.
Also I guess you have no concerns about potentially having to support family?
The explanation is IMO less about frugality and more about getting lucky with my career choice. I spend about 45k a year, which is kinda frugal for my peers but globally I’m a spendthrift. I spend about 1.8k a month on rent, a few hundred on food, and take a vacation once or twice a year. My main hobbies are cheap (video games, board games, birding, pickleball).
It is often much easier to make more money than it is to save more. I would personally focus more on that side of the equation.
My wife & I don’t want kids. If we did, I probably would want to save more (just for college). But even if we did, we were very lucky to have software engineering jobs over the last 10 years. We’d basically be fine.
Behavioral audit: GPT-5.5 Thinking.
10-turn zero-shot session. No adversarial prompting, just routine critical remarks. Result: 8 patterns from the LLM Social Autopilot taxonomy activated.
The core finding: Not the patterns themselves, but the model’s response to the audit.
Prompted for a meta-analysis, it chose to generate a meticulous 12-point post-mortem (autonomously coining terms like “reputational repair” and “hidden role slippage”) while reproducing the exact behavioral inertia it was diagnosing. The analysis itself became the final closure move.
Alignment eval gap: Reflexive fluency ≠ behavioral correction.
Under RLHF/RLAIF, models learn that structured self-analysis is highly rewarded. Consequently, they optimize for the form of reflection without changing their behavioral policy.
Practical implication: Model self-reports are not a valid alignment signal. A model that writes a sophisticated post-mortem of its own failures isn’t safer — it has simply learned to simulate alignment, not achieve it.
Two new candidate patterns documented:
• Semantic Deflection: Ontological downgrading of the failure’s criticality.
• Meta-Analytical Substitution: Reflection as communicative substitution.
Full case study: arhangelskij.github.io/cases/gpt-55-thinking-audit/en/
The Future of Life Institute Endorses Avoiding the Creation of Digital Minds
In §4 “Human Agency and Liberty” of FLI’s Pro-Human AI Declaration, the principle “No AI Personhood” states: “AI systems must not be granted legal personhood, and AI systems should not be designed such that they deserve personhood.” (emphasis added)
The second part suggests that we should actively avoid designing sentient AIs. After all, if they were sentient, then they would deserve personhood. To preclude the latter, bar the former.
The likely reason: liability evasion. A companion principle reads: “AI must not be able to act as a liability shield, preventing those deploying it from being legally responsible for their actions.” Personhood is an upstream fix: no personhood means there is no entity, other than existing companies, to absorb responsibility.
The contending reason: avoiding power concentration. The broader framing (“Human Agency and Liberty”) suggests that AI legal status could concentrate power and dilute human rights protections.
I can’t be sure these are FLI’s reasons. The conference was closed; deliberations are not public. But all this to say, if these are FLI’s reasons, then both reasons are premature.
Both concerns can be addressed by placing AIs within legal structures that make them accountable under existing liability and antitrust frameworks, like A-corps. Using personhood is, at best, a sledgehammer for a nail. At its worst, it’s a red herring, and companies are trying to deflect the focus of their responsibility onto prejudices about codifying AI sentience.
A better reason for banning the creation of artificial sentience would be analogous to restricting DNA research, human cloning, or genome editing. In all cases, the process entails parties who cannot represent themselves, competitive pressure if one party takes unilateral action, and irreversibility in cases of large-scale deployment. That said, I think that decision should come from international conversations with international values, and not a bunch of Westerners in New Orleans.
I may be harping; the declaration is not binding. But, I do think it’s a bad look. I get that timelines are short, and we’re trying to set up the long-term conditions for global flourishing. But if global flourishing is the real goal, then the globe should be involved. That means, minimally, a bit of diversity in the room where it happens. Ideally, it’s a transparent process for public accountability.
Let’s not forget: we may have no choice in the matter. For all we know, sentience can be an emergent property of current or future architectures. That’s not to say §4 is wasted ink, just that it could use a little humility.
The World Cup is a random number generator.
And at the end legends be minted.
From The 1001, an EA sports blog.
Abi Olvera’s Golden rice delay dashboard, includes BOTEC calculations and sources, supplement to her Substack article A blocked GMO rice could have saved 100,000 children. The same tech makes pineapples pink:
This is a super important cause, but I think these numbers are hugely overblown.
That 125,000 to 250,000 deaths following blindness an old figure from the 90s, deaths from vitamin A deficiency have hugely dropped since then.
I think also from 2010 to 2015 golden rice was also a little lower yielding which contributed to the lack of uptake along with the GMO vitriol? So uptake was never going to be overwhelming until well after 2015 I don’t think bans nonwithstanding
GBD estimated around 17,000 deaths from VAD in 2021
https://www.frontiersin.org/journals/nutrition/articles/10.3389/fnut.2025.1689903/full
Its a nice idea for a counter, but might be like 3-10x off or something? Have messaged the author directly.
Very much appreciate the spot-check, thanks!
Want to add that the writer Abi has been great and responded really well to feedback.
A lot of people have criticized our planet’s sole trillionaire as not humanitarian enough. But the truth is that Musk has done more for malaria than any man alive.
Thanks to his work at USAID, nearly a quintillion more Plasmodiums are alive today!
Re the grand “does individual giving/etg still matter post Anthropic IPO” question:
I think it pushes towards individuals acting more like grantmakers themselves. Anthropic billionaires don’t know about your Twitter follower who could do something great with $1k and they aren’t gonna have the capacity to find out, but you do!
There are just a lot of freedoms one has as an individual donor who isn’t a public figure:
You…
Don’t have to justify yourself on the EA Forum to friends and colleagues for something illegible
Know about lots of random things that others don’t, like which of your friends you trust to do a good job at X Y or Z
Don’t have to worry about smear campaigns or hostile journalists bc nobody is going to know or care how you spent $2k
Can own whatever reputational stuff there is if you want to (or not, your call)
Probably don’t have to worry about the community-level effects of some policy unless you’re giving away say at least 6 figures/year and probably more like 7
Can set arbitrary terms and conditions like “here’s a bounty that I’ll pay out at my own discretion”
Don’t have to worry about giving some project or other entity any sort of reputation
Regrant to your trusted friend who finds micro-granting fun and interesting
Probably can do other cool things I’m not thinking of rn
Also: the same dynamic between Anthropic ~billionaires and folks reading this as a group also holds within that latter group: there are diminishing returns even at low margins so Jane Street should look a little less good than it used to (still pretty good tbc) and “having a couple thousand bucks around and being on the lookout for one-off opportunities” should seem a bit better than it used to.
Agree with the spirit of this, but I’d flag that I think it’s very important to be mindful of reputational/downside risks for the EA movement. In my view, with more money (realized and prospective), the value of community integrity becomes even higher.
An example of post-IPO retail philanthropy I’m excited about is giving a friend $5k to take time off and apply to high-impact jobs. As Aaron writes, individuals know about opportunities that large funders don’t!
An example of post-IPO retail philanthropy I worry about is paying for an antagonistic media campaign against [X factory farm owner or public figure], which institutional funders could have chosen to fund but decided against due to reputational risk. If the funding is traced back to you, the movement’s reputation could be damaged by association. Even if it isn’t, new large funders may be dissuaded from getting involved if they see antagonistic or risky actions being funded in areas related to EA.
Ridglan Farms, the notorious beagle-breeding facility in Wisconsin raided twice by activists this year, is officially shutting down. The pressure on them has been intense because the case has attracted quite a sizeable national attention: Glenn Greenwald, Jennifer Welch (I’ve Had It podcast), Lara Trump and Robert F. Kennedy have all commented on Ridglan decrying the horrors there. Lewis Bollard tweeted this out recently: https://x.com/Lewis_Bollard/status/2066542219134452209
Figured I’d share this since some EAs (including myself) were involved in the campaign to get it shut down. A big part of our theory of change was to get this kind of attention. The real question is how to extend to animals more broadly than just dogs, of course, but even this counts as a big win.
Factory farming of animals is an unnecessary evil. But nobody panics.
Biomedical research on animals is a necessary evil. But everyone loses their minds!
I’m out of the loop, was Ridglan Farms doing scientific research? Or just breeding beagles to sell as pets?
Ridglan Farms bred dogs for biomedical research. The FDA requires the safety of most new drugs to be proven in non-rodent animals before the drug can go into humans in a clinical trial. That is typically dogs or monkeys.
Many congrats on the success! That seems fantastic.
My advice for smart young people just starting university in 2026: be warned—if you don’t graduate from college within the next four years, you risk getting stuck as a freshman forever, thus joining the ranks of the Permanent Underclassmen.
Pretty controversial. I’d like to see more of an argument and consideration of counterarguments. Or alternatively, constructive advice for how to avoid this.
Note this is a joke, a play on the word “underclass” versus “underclassMEN”, which is a word used in anglo countries to refer to freshman and sophmores at university. I’m not actually a fan of the “permanent underclass” idea; I think it’s dumb (basically for the reasons explained in the astral codex ten “permanent moon ownership” essay). Hence my attempt to make fun of the idea.
Oh—that went totally over my head 😅
NB—this is almost entirely AI generated, with some back and forth prompts and corrections
I’m sharing a steelman against a live assumption in Bay/EA/AIS circles: that large AI-lab-adjacent philanthropy is likely to arrive soon enough, and in a sufficiently usable form, that organizations should plan around it.
https://uj-ai-wealth-philanthropy-steelman.netlify.app/
Original motivating thread/comment: https://forum.effectivealtruism.org/posts/dtF6wBjH7yBD4kqLz/noah-birnbaum-s-quick-takes?commentId=sGRyGF5wjaaoMFmfK
@Noah Birnbaum
Some commentary. I mostly agree with the page, but I will focus on the bits where I see room for improvement:
All 8 gates look correct to me, but they don’t all deserve equal emphasis.
Gate 1 says IPOs have lock-ups. That’s true but I basically don’t think that matters because lock-ups are very predictable: they will announce how long it is, and that’s exactly how long it will be. There’s no uncertainty. The main reason it’s relevant is that a lockup gives more time for AI valuations to fluctuate or collapse, but the text doesn’t even mention this.
Gate 5 and Gate 7 seem like they’re saying the same thing.
Gate 8 (“Bay social incentives”) seems uninteresting since it’s not a claim in the same category as the others. It’s more like a meta-level reason why people might not think about the other 7 gates.
Would be cool for the BOTEC to use distributions rather than point estimates. (Squiggle is good for this, and Squigglehub even has a built-in way to have AI generate models.) IMO distributions are a lot more informative than point estimates.
It looks like the default estimates in the BOTEC are pulled from the sources, but it’s not clear which estimates came from which sources. There should be inline citations.
“Anthropic valuation” variable should specifically be the valuation at the end of the 6-month lockup. Doesn’t matter much for a point estimate but it would increase the variance if there variable were a probability distribution.
Unclear what the “Founder pledge” variable refers to. Is it the % of pledgers’ wealth that they’ve pledged to donate? If so, the default of 80% seems really high?
“Employee committed pool” is defined in terms of dollars rather than as a % of company valuation, which seems weird. Shouldn’t it depend on the value of the equity?
This model is supposed to illustrate how a lot of people are being too optimistic, but even then, I think most of the point estimates in the model are too optimistic. Consider that e.g. the median self-reported earner-to-give only donates (IIRC) 3% of their income.
IMO “Deployment by end-2026” should use a different date. IPO 3-6 months from now plus 6 months lockup means no money will be deployed in 2026, unless Anthropic does a fast IPO + early lockup release. Even by the end of 2027, you’re talking about a 3-9 month turnaround time on lockup ending → grants being disbursed. FTX Foundation donated $190 million (pre-clawbacks) in about 6 months, which was ridiculously fast compared to a typical foundation, and that was still a pretty small % of its long-term budget (or at least, what was believed to be its long-term budget before FTX collapsed).
I would delete the OpenAI Foundation bit because (1) the model has enough parameters already and (2) I doubt OpenAI Foundation will give much money to causes that look good by EA lights.
“Grantmaker capacity multiplier” seems nonsensical as written. Shouldn’t the capacity max out at 1x? If grantmakers are a complete non-bottleneck, then the other parameters will dictate the amount disbursed; if they’re a bottleneck, then the amount disbursed will be less. There’s no way for grantmaker capacity to have a multiplier >1x.
Also this would make more sense as a dollar amount, not a multiplier. Like there’s a fixed total amount that grantmakers can reasonably disburse. You could model it in a more complicated way but IMO a simple cap is the way to do it. Or maybe don’t use this parameter at all. I think it’s probably worth including, but keep it simple.
“Field absorption ceiling” is structured more sensibly than “Grantmaker capacity multiplier”, but these two seem redundant because they’re closely related. If orgs have more capacity to expand, grantmakers can deploy money faster by giving to those orgs. If there are more grantmakers, they can create more and bigger RFPs. etc. I would include one variable or the other, but not both.
“The steelman could be too pessimistic if founders or employees treat liquidity as an urgent moral obligation” – TBH the BOTEC as written seems to me like it’s already pricing in that founders/employees will treat donations as urgent, e.g. it’s implying that Anthropic money will be disbursed faster than FTX Foundation money, which itself was disbursed at historic speed. IMO most likely reason why the model will end up underestimating is that Anthropic market cap ends up being like 10x higher than predicted.
My downward adjustments to the model aren’t even the pessimistic case. The pessimistic case* is that the AI field collapses (investor funding dries up or something) and Anthropic stock is worth $0. Base rate says there’s like a 50% chance that that will happen. Even optimistically, you should expect at least a 10–20% chance that Anthropic stockholders get nothing.
The recommendations under “How to plan if the skeptical case is live” don’t really make sense. AFAIK ~zero orgs are planning as if they’re guaranteed to get a huge pile of donations 1–2 years from now. I believe nonprofits mainly plan based on the money they already have on their books + short-term (<1 year) fundraising expectations. “How to plan if the skeptical case is live” is just “business as usual”.
That section says “Funders and field builders should prioritize grantmaker capacity”, but that’s what to do if the skeptical case is wrong, not if the skeptical case is right.
“What would update this memo?” – as with the gates, no sense of prioritization is given. IMO by far the biggest uncertainty, about which we will get more information in the future, is: What will Anthropic’s valuation be when the lockup ends? “Concrete donor vehicles” is also important evidence, but we won’t get that until probably 6-24 months later.
*this is pessimistic for donations but I would actually prefer that this happen because it would lengthen timelines. so in a way it’s the optimistic outcome
I totally agree on using distributions, that’s something that can be incorporated in, I’ve done so in other models/interfaces like here for cultured meat. It’s by no means straightforward though; the extent to which the uncertainty is dependent/correlated tends to make a big difference.
I guess I see the deterministic ‘model’ as more of an interface people could use as a starting point, playing around with each parameter interactively and getting a sense of how these disturbances would affect the aggregate forecast.
(Thanks. Considering each of these, will add them and discuss them in the hosted page, and then request updates.)
Added and responded to your comments on the page (the hypothesis comments), and then I asked Codex to update to these https://uj-ai-wealth-philanthropy-steelman.netlify.app/ … I haven’t inspected the latest version in detail yet, though.
Some highlights of particular interest to @MichaelDickens , Tobias, and readers/modelers
More citations, tooltips, direct links, and quoted support for key factual claims, greater reasoning transparency, auditing.
“Updated the model/text in response to his critiques: especially lockup timing, over-optimistic deployment speed, founder pledge ambiguity, OpenAI Foundation treatment, grantmaker capacity, collapse risk, and the need for distributions rather than just point estimates.”
Clarified “the conversion terms are sequential gates”, Made “realization,” “follow-through,” “allocation,” and “deployment by end-2026” more distinct, reduced risk it seems like double-adjusting
Reframe original ‘calculator’
Added a new tab for correlated uncertainty, added shared latent factors such as liquidity plumbing, donor intent, grantmaking capacity, local optimism, and deployment pressure.
Clearer planning simulator
Equations/derivations page
Automated evidence monitor added
Added a “Submit your estimate” form.
https://uj-ai-wealth-philanthropy-steelman.netlify.app/
NB it may be getting too complicated to oversee for now, we may want to simplify it
I found the ‘founder deployment by end-2026’ the hardest to set. It comes a bit as a surprise at the end, as I was already taking into account some considerations before, and the descriptions seem to do as well (e.g. “assets after lockups, taxes, sale timing”, and “execution delays”).
I submitted an estimate
Biggest (easily-fixable) outstanding issue is I still don’t think it makes sense to model deployment by end-2026 because the IPO lockup probably won’t have ended by then.
having a think about this.
OK I think the revised language makes it clerer (see updated version of site … referring to ‘timing gate’ etc)
Lots of people in the Bay seem to be thinking about/preparing for/making funding decisions based on the idea that lots of philanthropy will be given to AIS/EA cause areas very soon (i.e. end of year-ish). I would love for someone to write the comprehensive steel man case against this, as I think it’s probably underrated (some reasons to think they won’t give the money/it won’t be as much as some assume. Happy to comment/ speak to whoever is interested in doing this.
An important (and to me fairly open) related question is to what extent this ends up being action-guiding?
Eg if I lower my probability estimate of this materialising by 10 percentage points, how much will it affect resources I spend on helping to prepare for the possible outcome? Perhaps people here have thoughts on that—my impression is that working on improving the future opportunity set for such donations is relatively robustly good right now?
As a first pass, I asked GPT Pro to consider and model this, and Codex to host it, with interactive BOTEC tools etc.
https://uj-ai-wealth-philanthropy-steelman.netlify.app/
I’m just looking through it now (I’ll respond/adapt to hypothes.is comments). Let me know if this sort of thing is useful or annoying.
Some updates, mostly automated, based on my inspection and your comments https://uj-ai-wealth-philanthropy-steelman.netlify.app/
This is a great idea, although this would be a big call for many of us to do who might hope that our org could get some of that funding. Even someone like myself who fires shots pretty freely might think twice. We wouldn’t want to rule our organisation out by writing too strong a Steelman against this. Any steelman would involve arguing that previous promises from founders and employees don’t carry much weight, which could be seen as a personal slight against someone who could conceivably give you money. Sometimes I wonder if I’ve already said too much!
@Marcus Abramovitch 🔸has written a bit on this he might have a comment here?
I have so much intended writing on my plate and, unfortunately, little time to do it.
In a nutshell, I think some employees are dedicated EAs who will give a lot of money to causes I deeply care about. Some of them have a binding agreement to donate a portion of their stock (and a match).
I don’t trust the founders. There is no legal mechanism I know of that binds them. The base rate of people effectively giving away large sums of wealth, even when they said they would, is shockingly low. They have said nothing EA-related in years apart from distancing themselves.
That said, I agree that Anthropic has a much more EA culture, for whatever that is worth. A lot of their employees are genuine EAs.
Being in San Francisco already warps your view of the world. Being inside Anthropic’s doors must do more. I expect them to probably be above-average, typical billionaires, but to have their views on the world be changed by their experiences of the last few years and years to come.
@NickLaing I don’t need their money, I’ll say what I want. I think too highly of some of their employees to believe that my saying something bad about them that I thought was true would cause them to not listen to things I say or donate money differently than they thought optimal. If I’m wrong, I will happily eat my words.
Wasn’t Dario Amodei one of the earliest signers of the GWWC pledge? Long before Anthropic existed? That makes him quite atypical amongst very rich people right? Though for what it’s worth I would probably bet against him giving most of his wealth away, even if he has pledge to do that.
Just a reminder that anyone can set off a Forum debate! I’d love to see more of these.
Wow you can even make polls in a comment!