Founder of Overcome, an EA-aligned mental health charity
John Salter
Karma: 1,690
Thank you for thinking critically about my work! You’re right, it is not a direct comparison.
It shows an effect size of just over 0.6. The typical for most psychotherapies is 0.8. (see the Perplexity.ai summary below of the PTSD meta-analyses in the literature)
I did 0.6 / 0.8, which is 0.75. That equates to 75%.
As this is pretty approximate, especially given that it didn’t directly compare the same groups against one and other. I included the ~ before 75% to show that it shouldn’t be used as a precise figure. In hindsight, I regret not making this more explicit.
That being said, the near equivalency between laypersons and trained therapists is widely accepted. Every single EA mental health charity uses laypersons rather than professional therapists for this reason
and maybe newcomer?
I am! Just under two years delivering psychotherapy interventions, ~5 years in mental health more generally
Could you tell me more about the length, intensity, and duration of a typical treatment program?
We offer a minimum of six weeks, with no arbitrary cap. It’s once (or rarely twice) a week for ~1 hour at a time. I’d suggest that six weeks is the most cost-effective if you are limited by supply, but in practice it tends to be longer because often you have spare capacity.
Less sessions is a reliable way to reduce cost, but my understanding is there’s a U-shaped curve to cost-effectiveness here. 1 session doesn’t have enough benefits but 100 sessions costs too much and doesn’t add more benefit.
That sounds about right.
Also, are you targeting specific conditions? I see improvement in insomnia but that can arise from a sleep intervention or a general CBT course too
Depends on the client. Mostly our counselling is bespoke, but we have some programmes for more specialised issues (e.g. chronic insomnia, addiction, phobia)
I’m going to assume you mean comparison not experiment as we did no experiment comparing the two demographics.
The comparison was to show how much easier it is to treat high-functioning western demographics than it is to treat lower-functioning LMIC demographics. One common misconception I run into a lot is that treating people in LMICs is easier because there’s still “lower-hanging fruit” yet to be treated. I wanted to show some statistics illustrating that this was not the case by comparing two similar pilots with different demographics
The higher income, higher functioning demographic was easier to recruit, triage, maintain and got comparable results. I think this violates most funder’s expectations.
8 Counter-Intuitive Considerations for Psychotherapy Interventions
IMO, if the content is good we shouldn’t bring it up. If an author is producing bad content more than once a month and it seems generated by LLMs they should be warned then banned if it continues.
I suspect any comment threads about whether content is LLM-generated aren’t worth reading and thus aren’t worthwhile writing.
Example
https://upworkclone.bubbleapps.io/
Person specs
No technical knowledge required, it’d just involve learning how to use “bubble”. The hard part is that no volunteer wants to be on a platform with few organisations and no organisation wants to be on a platform with few volunteers. [further reading]
Do I wanna make this?
I have no time. Ideal founder traits:
1. Well connected with EA orgs volunteers find attractive or volunteers EA orgs would find attractive.
2. Experience volunteering
3. A love of networking
4. Good at convincing EAs to do stuff.
Unless do favors for, or otherwise suck up to, well-connected people it can be difficult to get your idea in front of early-stage funders (i.e. HNWIs / individual donors). Even if you do, unless you are able to get a warm introduction or a meeting, your idea will be a mere piece of paper, overlooked in favor of projects from people the funder already knows. Even if you manage to get connected, you will likely be rejected without feedback and so cannot efficiently improve your application.
Potential improvementsAI: I’m pretty sure a GPT 01-preview custom GPT, with just prompt engineering, could give pretty strong feedback on grant proposals if it was just given guidance and examples. The subset of applications that look strong to the bot could be forwarded to an email inbox where a well-connected EA could look at it and decide if it merits forwarding to funders.
Forum: There could be a weekly thread dedicated to feedback on ideas. Pay one person to give feedback to applications that get none.
EA YC: If either of the above yields promising results, it could be scaled up by doing an EA version of y-combinator.
A fuckton of EAs struggle with procrastination, at least 10% but probably more like 20-30%. Funders tend to underestimate the prevalence because it’s in nobody’s best interests to admit it to them.
Interventions work fast and are cheap. There’s no overhead because you can just do them via Zoom. We already know what works, so you don’t even have to innovate.
Funders I’ve spoken to have tended to think that it’s only the least productive EAs who procrastinate. I’ve worked with a bunch of people working in high prestige EA jobs, including dozens of charity founders, and can confirm this is categorically untrue. It’s not that highly productive people don’t procrastinate, it’s that they have other strengths that counteract the weakness.
A matchmaking service for EA projects and EA volunteers / freelancers with a reputation system: At present, volunteers are underutilised because there’s no good way of knowing who is reliable. Freelancers are often chosen because they are “EA-aligned” when really some are either incompetent or grifters. There ought be some way of seeing how someone has performed on past work.
Templates already exist on-line for this exact type of marketplace. The only upfront cost would be in finding enough early adopters to give it a go. From there, it could take a small cut of freelance work and fund itself.
The UK / US requirement seems both damaging and completely arbitrary. What’s the reasoning?Lots of strong charities, including many AIM-incubated ones, aren’t incorporated and instead work via a fiscal sponsor. Can you clarify if they are eligible?Edit: saw the footnote about UK/US.
Your work at BIT sounds really fantastic. Well done!
I don’t think animal interventions are worse but I do I think the statement is wild speculation. I don’t think EAs can effectively compare interventions between very different cause areas.
I suspect most EAs don’t actually think through their own cause prioritisation, I think they instead defer to others, and thus don’t view the consensus as compelling evidence to change my mind.
Evidence that ripple-effects of interventions are negligible would change my mind though. I find the EV calculations for the short-term supremacy of animal welfare interventions compelling but suspect that global health interventions have larger ripple effects throughout time and that ripple effects are likely more important than the immediate effect of the intervention.
The core issue with your post is that one cannot simply state the cost of ownership then use that to calculate the cost-effectiveness, and be taken seriously. To be convincing, you need to actually factor in all the costs of your intervention:
1. The cost of getting each hospital to sign up
2. The cost of delivering the equipment
3. The cost of training people in how to use it
4. Your salaries
5. All your other costs
To do so seems intellectually dishonest, or at least that you’re missing out much of the information we need to properly evaluate your intervention. I don’t think you are doing this on purpose. It looks like you’re doing great work:It isn’t easy to get founder’s pledge to give you money, and their charity evaluations are taken pretty seriously.
It seems like you’re saving a lot of lives that otherwise would not have been saved
It seems at first inspection like your intervention could be cost-competitive with other top EA charities, which would be an incredible accomplishment. It’d put you in the top >0.1% of charities.
I’d suggest that your presentation of your work is the biggest barrier to getting it taken seriously right now.
1. The post is pretty long and most of the paragraphs are verbose.
2. It took me a while to actually understand what you do. Most readers have probably clicked off before actually knowing what your intervention consists of. I believe the relevant information is in paragraph ~7? It needs to be in paragraph 1 or 2!
3. Your call-to-action (requesting feedback) as at the top. It needs to be at the very bottom
It might be worth getting some external advice on how to present your work better to donors and EA audiences.I’ve up-voted your post because I want more people to engage with your work. I’ve screenshot this to show you that I’m not just criticising you for fun. I want you to succeed.
This was from 2018. Does anyone have up-to-date estimates of the value per co-founder per charity?
What type of things would you be excited to fund?
I spoke to Kevin recently—seems like a nice guy who’s serious about EA!
Here’s examples from six of list of top ten companies by market cap
Apple is worth $3 trillion despite being on the verge of bankruptcy in the mid-nineties.
Google is now worth 1.9 trillion. The founders tried and failed to sell it for 1 million.
Amazon’s stock price dropped 90% during dot com crash
Nvidia, recently world’s most valuable business, had to lay off half its staff in 1997 and try to win a market with ~100 other startups all competing for same prize
Elon Musk: “I thought SpaceX and Tesla both had >90% chance of failure”. He was sleeping on his friends coaches to avoid paying rent at that time.
Facebook’s rise was so tumultuous they made a movie about it. Now worth 1.3 trillion.
Warren buffet regretted buying Berkhire Hathway, and almost sold it. Now worth 744 billion.
Talking about EA more specifically~10 founders have spilled the details of their journeys to me. ~70% felt hopeless at least once. There’s been at least four or five times I’ve been close to quitting. I had to go into credit card debt to finance our charity. I’ve volunteered full-time for >4 years to keep the costs lower, working evenings to pay rent. Things are now looking a lot better e.g. our funders doubled our budget last year and we’re now successfully treating ~4-5x more people than this time last year.
I’ve had similar worries. Most extremely impactful projects look destined to fail half a dozen times before they blow up.
The right balance is hard to strike between incentivising determination and encouraging people to waste less money on ideas that prove weak.
I think posts should get karma based on the value provided not the effort put in, hence why I’m upvoting this comment.
I think this post would be more compelling if you shared more data (e.g. cost per participant, leading indicators of impact), how much money you got from EAIF etc