Founder of Overcome, an EA-aligned mental health charity
John Salter
Karma: 2,136
A note on the clickbait:
As affected people are unusually impulsive, I suspect that the people who’d most benefit would have been least likely to click with a boring title / post. There’ll be a more formal post soon about the RCT results, and another less cryptic sign-up post, all coming in the next week or so!
To respond briefly:
1. “First of all, the AI 2027 people disagree about the numbers”.
That’s irrelevant to your claim that you’d put “60% odds on the kind of growth depicted in AI 2027″
“you’ve predicted a 95-trillion-fold increase in AI research capacity under a ‘conservative scenario.’” is false. I was just giving that as an example of the rapid exponential growth.
Here’s what you wrote:
”This might sound outrageous, but remember: the number of AI models we can run is going up 25x per year! Once we reach human level, if those trends continue (and they show no signs of stopping) it will be as if the number of human researchers is going up 25x per year. 25x yearly increases is a 95-trillion-fold increase in a decade.”
You then go on to outline reasons why it would actually be faster than that. If you aren’t predicting this 95-trillion-fold increase, then either:
1. The trends do indeed show signs of stopping
2. The number of AI models you can run isn’t really going up 25x YOY
We can talk all day, but words are cheap. I’d much rather bet. Bets force you to get specific about what you actually believe. They make false predictions costly, true ones profitable. They signal what you actually believe, not what you think writing will get you the most status / clicks / views / shares etc.
What’s the minimum percentage chance of greater than 10% GDP growth in 2029 that you think is plausible given the trends you’re writing about and how much are you willing to bet at those odds? I’d rather bet on an earlier year, but I’d accept 2029 if that’s all you’ve got in you.
To be explicit, I’m trying to work out what you actually believe and what is just sensationalised.
This is a response more befitting Jim Cramer’s Chihuahua than Jeremy Bentham’s Bulldog.
I’d put about 60% odds on the kind of growth depicted variously in AI 2027...
According to AI 2027, before the end of 2027, OpenAI has:
a “country of geniuses in a datacenter.” each:
75x more capable than the best human human at AI research
“wildly superhuman” coding, hacking and politics
330K Superhuman AI Researcher copies thinking at 57x human speed”
In their slowest projection, by April 2028, OpenAI has achieved generalised superintelligence.But you’re only willing to bet US GDP grows just 10%, in just one year, across the next 15? The US did 7.4% in 1984. Within 10 years—five years before your proposed bet resolves—you’ve predicted a 95-trillion-fold increase in AI research capacity under a ‘conservative scenario.’ According to your eighth section, this won’t cause major bottlenecks elsewhere that would seriously stifle growth.
If this is really the best bet you’re willing to offer, one of three things is true:
You’re wildly risk averse
You don’t believe what you’re writing
You’re misleadingly missing out the fine print (e.g. “I’d put about 60% odds on the kind of growth depicted variously in AI 2027″ 2027
but not any time close to when they actually predict it will happen”)
Which is it?
Are you willing to stake cash on any of your near-term predictions, say before 1st Jan 2028? If so, which and at what odds?
I think this misses the forest for the trees. Yes, the pure donation message tested slightly better in conversion rate—but that’s only half the equation.
Donations = Reach × Conversion Rate
The controversial framing got massive media coverage that a standard “please donate” pitch never would. Even if conversion is marginally lower, if reach is 10-100x higher, the math still favors the provocative approach.The key question is whether the increased donations are worth the non-monetary costs.
We’ll be posting about it later this month!
This was a really informative read!
One thing I found confusing was how China would have a huge geographical advantage over Taiwan / USA. It strikes me that Taiwan has a 100 mile moat, and a ton of mountains / coastal cliffs. It’s essentially a scaled up version of a castle. It’s hard to imagine an easier geography to defend, tactically at least.
I presume it’s the location that’s the issue. While the US would have a harder time resupplying Taiwan, they presumably know this and can build up stockpiles ahead of time. While there’s freight shipping, the cost of doing so would be a rounding error. My understanding is that China is surrounded by enemies and US military bases, so the prima facie difference between US and Chinese mainland’s proximity to Taiwan is moot.
I haven’t studied this conflict much so I’m pretty sure I’m wrong. What am I missing?
The potential for AI to reshape mental health globally is really underexplored. It’s great to see EAs paying attention to it! Here are some hastily scribbled thoughts:
1. Confidently drawing conclusions about the subset of people who are currently using LLM therapy from studies of people who were paid to try it out.
I’m doubtful those two client groups would be sufficiently comparable. Given that variance in clients accounts for ~5x more variance in treatment outcome than the treatment itself, it’s really important to keep the client group constant.
2. Consider the challenges of distribution and funding.
We already have many evidence-based, cost-effective therapies that aren’t widely implemented—not because they don’t work, but because it’s hard to secure funding and uptake at scale. One potential shortcut here is working directly with the model providers: they already have distribution, resourcing, and reputational incentives to avoid harm. Helping them mitigate specific risks may be a more tractable way to have immediate impact, while also building credibility for future projects—before vs afters are a really compelling way to demonstrate your impact that anyone could understand in seconds. The other proposed work would be far harder to communicate, and thus harder to get people excited about.
3. Take the unguided digital intervention literature with a grain of salt.
There’s been decades of promising RCTs that unfortunately didn’t translate into widespread real-world impact. A few possible reasons:Low-cost trials make it cheaper to try again until you get a strong result (by chance)
Many of these studies are sponsored by companies/authors with strong incentives to report positive findings.
Participants are often paid to engage—something that doesn’t generalize well to real-world settings where adherence is a big challenge, especially for unguided interventions.
4. Doing too many things
Each of your four projects could easily be an entire organization’s focus. Consider testing each idea briefly, then doubling down on the one where you see the most traction. . If you’d rather work separately on different things, you might want to consider branding yourselves as separate projects—it’s harder for any given project to be taken as seriously otherwise.
I agree with Huw’s assessment re: books vs digital vs digital + guide. Here are a few less-discussed reasons why, hastily scribbled:
Recruitment and retention costs: The cost of delivering a very cost-effective therapy is often lower than the cost of convincing someone to seriously give it a go. People don’t really want to just read a book or just use an app; they overwhelmingly want to talk to a real person. It can therefore be cheaper to recruit and retain people when a person is involved.
Misinterpretation of non-significant: Psychologists often present their findings as though statistically non-significant differences should be ignored. Sometimes this results in treating an effect size of 0.3 and 0.6 as if they’re identical, leading to conclusions like “we found no significant differences between guided and unguided…”. Nobody has time to read the whole literature, so people skim—and can come away thinking there’s no real difference, when in practice it may be more like a 30–100% difference in effectiveness.
Greater publication bias in unguided RCTs: It’s insanely cheap to do RCTs on unguided interventions because the cost of delivery is near zero and logistics are simple. Since it’s usually the researcher or funder who developed the treatment, they’re unlikely to publish the mediocre results. What gets published instead are lots and lots of positive findings, creating a skewed picture where unguided looks consistently effective.
Retention IRL: Despite most mental health apps showing >50% completion in RCTs, they retain only ~1–3% of real users that long. Guided self-help interventions retain an order of magnitude more. You thus need recruit an order of magnitude more users to treat the same number of people. This not only undermines their cost-effectiveness, but also drives up recruitment costs for everyone else. Plus, a lot of people try something that doesn’t work for them, have their time and effort wasted, become more jaded, and are harder to convince to try again later.
All that being said, I think we focus far too much on differences between treatments and far too little on differences between clients. The latter explains roughly 4× more variance than the former, yet accounts for <1% of the research published.
Great write up! I especially like how candidly you wrote about the errors.
I’m happy to match up to $200 worth of donations from EA Forum Readers (personally, not on behalf of my organisation):
1. The theory of change is solid—the inefficiencies they are addressing seem efficient to address and to have a big lasting impact.
2. Increased social support is a really efficient way to improve mental health
3. I like that it’s run by people in the local area who know the women personally and who were self-funding it for so long up to now.
4. I’d like to encourage more people from LMICs to engage on the forum, especially on matters of global health where they have the context so many of us lack.
5. EA has so rarely funded small local organisations, and so rarely funded charities that can generate their own income thereafter. I think this is a cheap, efficient test of these types of grants
If you’d like to take advantage of my matching, please message me on the forum.
Hey Gerry! It sounds like your organisation is working on a really important demographic in potentially a very cost-effective way! I think your application would be more compelling to potential funders if it included answers to the following:
What other funding is being used for this project? Presumably you also have staff to pay etc
This would help donors understand the full cost of supporting a widow right now. If this isn’t apparent, most donors will think it’s much much higher because if something is missing they tend to assume the worst.
What are your ambitions for the future?
Most of the impact of a grant like this is often in supporting an organisation that will grow into something more impactful over time. EA donors especially care about this, and want to know your longer term plans if possible.
How would your outputs differ without an office?
It’s not clear what the $5k itself would buy in terms of additional widows being supported. This, “the counterfactual difference in utility” is the primary thing EA donors think about when deciding to donate.
If you take away one thing let it be this: anything left unclear or unspecified will be interpreted more negatively than if it had been clearly stated. For this reason, you might also want to include an organisational budget and more details on exactly what you’re teaching / how you’re supporting people with their mental health.
That being said, there are some great things about this application:
The length is right for the size of the grant
its written in plain English, which is much more persuasive than any alternative.
It’s great that it’s being led by the demographic it intends to serve!
It does seem like the costs are low and the potential impact is high, which is exactly what EA funders are looking for.
I’d be happy to spend 30 mins helping you work on your application if you’d like (for free). I’ve been successful in raising money for mental health work in LMICs over the past few years. If that’d be helpful, drop me a message or just leave a comment and I’ll reach out by email :)
Strongly upvoted. Catherine is likely constrained in what she can say due to her role, in part, as a mediator between EAs.
Here’s a few blunter points I’d add / make explicit:
EAs / EA orgs that do shockingly poor work sometimes wind up with informal, sometimes unspoken, mutual non-dispargement agreements with their victims. It’s rarely ever worth the effort of giving someone a terrible reference because they might retaliate. You need to drag it out of them and listen hard for hints. Promise confidentiality. Don’t settle for a short written reference from some HR person. Try get on a call with someone who worked closely with them.
Do not assume that because a mental health professional advertising EA-alignment means that they’ll be making scientifically sound suggestions. Seek independent reviews. Look at the scientific literature for their approach (if there isn’t much, that’s a bad sign).
The amount of negative information that reaches you about an organisation / person is not just about their competence / character. It’s equally as determined by the cause area and the culture there...
Different cause areas attract different personality profiles. This leads to very different dynamics in reputation and social information flow. Animal advocacy disproportionately attracts people the empathetic, brave, justice-seeking. The adversial nature of it selectively repels people who’re conflict averse. They’re thus much more likely to call people out and go to war with each other. Mental health attracts warmer more understanding people whose prime motivation is making people happy. So, you might be more likely to hear negative things about animal advocacy orgs than mental health ones, but you shouldn’t assume that we’re better to work with necessarily. It could equally be that our peers are less keen to call us out on our bullshit.
Influential people are much less likely to be called out because they’re perceived to be in a better position to retaliate. People who do more stuff are more likely to be called out because they’re involved with more people and will fail more often.
I’ve got a ton of deadlines coming up, so sadly can’t reply to comments
Nicely written!
It’d be hard to do without breaking a lot of good heuristics (i.e. don’t lie, don’t kill people)
If you use LLMs for coding, you should probably at least try the free trial for cursor—it lives inside your IDE and can thus read and write directly to yours files. It’s a also an agent, meaning you can tell it to iterate a prompt over a list of files and it can do that for 10 minutes. It also lets you revert your code back to how it was at a different point in your chat history (although, you should still use git as the system isn’t perfect and if you aren’t careful it can simultaneously break and obsfuscate your code)
It will feel like magic, and it’s astonishingly good at getting something working, however it will make horrible long-term decisions. You thus have to make the architectural decisions yourself, but most of the code-gen can be done by the AI.It’s helpful if you’re not really sure what you want yet, and want to speedily design on the fly while instantly seeing how changes made affect the result (acknowledging that you’ll have to start again, or refactor heavilly, if you want to use it longer term or at scale)
The far future, on our current trajectory, seems net negative on average. Reducing extinction risk just multiplies its negative EV.
What a turnaround. Well done!!!