AMA: Sean Mayberry, Founder & CEO of StrongMinds
I’m Sean Mayberry, and I’m the Founder/ Chief Executive Officer of StrongMinds. I will spend time on the Monday after the Thanksgiving holiday answering questions here (though I may get to some questions sooner).
A little background information about me:
I founded StrongMinds in 2013. We are a social enterprise/NGO that treats depression in low-income women and adolescents by providing group interpersonal therapy (IPT-G) delivered by lay community health workers. StrongMinds is the only organization scaling a cost-effective solution to the depression epidemic in Africa.
Our model developed from the findings of a randomized controlled trial in Uganda in 2002 that had remarkable success in treating depression with group interpersonal psychotherapy (IPT-G). The study, by researchers from Johns Hopkins University (JHU), used lay community workers with only a high school education.
I left my position as the CEO of a global antipoverty organization and founded
StrongMinds, concentrating in Uganda, the site of the previous randomized controlled trial. I used my family’s savings to accomplish this and volunteered full-time for the first 18 months until supporters were identified. We would seek out individuals with an interest in being data-driven, entrepreneurial, people-focused, passionate, open, and collaborative. Those traits eventually informed the core values of the company culture at StrongMinds.StrongMinds has now treated over 160,000 women with depression to date in Uganda and Zambia. On average, 80% of the women we treat remain depression-free six months after the conclusion of therapy. When our clients become depression-free, they can work more, and their kids eat and attend school more regularly. They also report that they no longer feel isolated and have people to turn to for social support. By the end of 2022, we will have treated over 210,000 women and adolescents through our work.
Drawing on evidence from over 80 academic studies, Happier Lives Institute has found that the group interpersonal therapy provided by StrongMinds is almost ten times more cost-effective than giving cash to people in extreme poverty (a standard benchmark for aid effectiveness).
I have been honored to present at a few Effective Altruism events. We love that the
the community has taken such an interest in StrongMinds’ approach centered around data collection, transparency, cultural competence/appropriateness, and human well-being.
Please ask me anything! I look forward to answering all of your questions.
UPDATE: I’m sorry for the delay in responding, but I wanted to take the time to be thoughtful and thorough with each of my responses.
- Assessment of Happier Lives Institute’s Cost-Effectiveness Analysis of StrongMinds by (22 Mar 2023 17:04 UTC; 239 points)
- Don’t just give well, give WELLBYs: HLI’s 2022 charity recommendation by (24 Nov 2022 9:56 UTC; 167 points)
- 's comment on StrongMinds should not be a top-rated charity (yet) by (28 Dec 2022 10:18 UTC; 36 points)
- EA & LW Forums Weekly Summary (14th Nov − 27th Nov 22′) by (29 Nov 2022 22:59 UTC; 22 points)
- EA & LW Forums Weekly Summary (14th Nov − 27th Nov 22′) by (LessWrong; 29 Nov 2022 23:00 UTC; 21 points)
Many thanks for doing this AMA!
I’m personally excited about more work in the EA space on topics around mental health and subjective well-being, and was initially excited to see StrongMinds (SM) come so strongly recommended. I do have a few Qs about the incredible success the pilots have shown so far:[1]
I couldn’t find number needed to treat (NNT)[2] figures anywhere (please let me know if I’ve missed this!), so I’ve had a rough go based on the published results, and came to an NNT of around 1.35.[3] Limitations of the research aside, this suggests StrongMinds is among the most effective interventions in all of medicine in terms of achieving its stated goals.
If later RCTs and replications showed much higher NNT figures, what do you think would be the most likely reason for this? For comparison:
This meta-analysis suggests an NNT of 3 when comparing IPT to a control condition;
This systematic review suggests an NNT of 4 for interpersonal therapy (IPT) compared to treatment as usual[4];
This meta-analysis suggests a response rate of 41% and an NNT of 4 when comparing therapy to ‘waitlist’ conditions (and lower when only considering IPT in subgroup analyses); or
this meta-analysis which suggests an NNT of 7 when comparing psychotherapies to placebo pill.
Admittedly, there are many caveats here—the various linked studies aren’t a perfect comparison to SM’s work, NNT clearly shouldn’t be used as sole basis for comparison between interventions, and I haven’t done enough work here to feel super confident about the quality of SM’s research. But my initial reaction upon skimming and seeing response to treatment in the range of 94-99%, or 100+ people with PHQ-9 scores of over 15 basically all dropping down to 1-4[5] (edit: an average improvement of 12 points after conclusion of therapy) after 12-16 weeks of group IGT by lay counsellors was that this seemed far “too good to be true”, and fairly incongruent with ~everything I’ve learnt or anecdotally seen in clinical practice about the effectiveness of mental health treatments (though clearly I could be wrong!). This is especially surprising given SM dropped the group of participants with minimal or mild depression from the analysis.[6]
Were these concerns ever raised by the researchers when writing up the reports? Do you have any reason to believe that the Ugandan context or something about the SM methodology makes your intervention many times more effective than basically any other intervention for depression?
[Edit: I note that the 99% figure in the phase 2 trial was disregarded, but the 94% figure in phase 1 trial wasn’t, despite presumably the same methodology? Also curious about the separate analysis that came to 92%, which states: “Since this impact figure was collected at a regular IPT group meeting, as had been done bi-weekly throughout the 12- week intervention, it is unlikely that any bias influenced the figure.” I don’t quite understand how collection at a regular IPT group meeting makes bias unlikely—could you clarify this? Presumably participants knew in advance how many weeks the intervention would be?]
How did you come to the 10% figure when adjusting for social desirability bias?
Was there a reason an RCT couldn’t have been done as a pilot? Just noting that “informal control populations were established for both Phase 1 and Phase 2 patients, consisting of women who screened for depression but did not participate”, and the control group in both the pilots were only 36 people, compared to the 244 and 270 in the treatment arm for phase 1 and phase 2 respectively. As a result, 11 / 24 of the villages where the interventions took place did not have a control arm at all. (pg 9)
Are you happy to go into a bit more detail about the background of the lay counsellors? E.g. what they know prior to the SM pilots, how much training (in number of hours) they receive, and who runs it (what relevant qualifications / background? How did the trainers get their IPT-G certification—e.g. is this a postgrad psychology qualification, or a one-off training course?) I briefly skimmed the text (appendix A + E) but also got a bit confused over the difference between “lay counsellor”, “mental health facilitator”, “mental health supervisor” and “senior technical advisor” and how they’re relevant for the intervention.
Can you give us a cost breakdown of $170 / person figure for delivering the programme (Or $134 for 2021)? See Joel’s response and subsequent discussion for more details. Specifically, whether the methodology for working out the cost / client by dividing total clients reached over SM’s total expenses means that this includes the clients reached by the partners, but not their operating costs / expenses. For example, ~48% of clients were treated through partners in 2021, and Q2 results (pg 2) suggest StrongMinds is on track for ~79% of clients treated through partners in 2022.[7] Or are all expenses of SM partners covered by SM and included in the tax returns?
In the most recent publication (pg 5), published 2017, the report says: “Looking forward, StrongMinds will continue to strengthen our evaluation efforts and will continue to follow up with patients at 6 or 12 month intervals. We also remain committed to implementing a much more rigorous study, in the form of an externally-led, longitudinal randomized control trial, in the coming years.”
Have either the follow-up or the externally-led longitudinal RCT happened yet? If so, are the results shareable with the public? (I note that there has been a qualitative study done on a teletherapy version published in 2021, but no RCT.)
The pivot to teletherapy in light of COVID makes sense, though the evidence-base for its effectiveness is ?presumably weaker.
What’s the breakdown of % clients reached via teletherapy versus clients reached via group IGT as per the original pilots (i.e. in person)
In the 2021 report on a qualitative assessment of teletherapy (pg 2), it says: “Data from StrongMinds shows that phone-based IPT-G is as effective as in-person group therapy in reducing depression symptoms among participants”. Is this research + methodology available to the public? (I searched for phone and telehealth in the other 4 reports which returned no hits)
Does StrongMinds have any other unpublished research?
What’s the plan with telehealth going forward? Was this a temporary thing for COVID, or is this a pivot into a more / similarly effective approach?
I also saw in the HLI report that SM defines treated patients treated for purpose of cost analysis as “attending more than six sessions (out of 12) for face-to-face modes and more than four (out of 8) for teletherapy.”—is this also the definition for the treatment outcomes? i.e. how did SM assess the effectiveness of SM for people who attended 7 sessions and then dropped out? Do we have more details around about how many people didn’t do all sessions, how they responded, and how this was incorporated into SM’s analyses?
Thanks again!
(Commenting in personal capacity etc)
[Edited after Joel’s response to include Q7, Q8, and an update to Q1c and Q5, mainly to put all the unresolved Qs in one place for Sean and other readers’ convenience.]
[Edited to add this disclaimer.]
[Edited to include a link to a newer post StrongMinds should not be a top-rated charity (yet), which includes additional discussion.]
Apologies in advance if I’ve missed anything—I’ve only briefly skimmed your website’s publications, and I haven’t engaged with this literature for quite a while now!
Quick primer on NNT for other readers. Lower = better, where NNT = 1 means your treatment gets the desired effect 100% of the time.
SM’s results of 95% depression-free (85% after the 10% adjustment for social desirability bias) give an EER of 0.15 after adjustment. By a more conservative estimate, based on this quote (pg 3): “A separate control group, which consisted of depressed women who received no treatment, experienced a reduction of depressive symptoms in only 11% of members over the same 12-week intervention period” and assuming all of those are clinically significant reductions in depressive symptoms, the CER is 0.89, which gives an NNT of 1 / (0.89 − 0.15) = 1.35. The EER can be adjusted upwards because not all who started in the treatment group were depressed, but this is only 2% and 6% for phase 1 and 2 respectively—so in any case the NNT is unlikely to go much higher than 1.5 even by the most conservative estimate.
They also concluded: “We did not find convincing evidence supporting or refuting the effect of interpersonal psychotherapy or psychodynamic therapy compared with ‘treatment as usual’ for patients with major depressive disorder. The potential beneficial effect seems small and effects on major outcomes are unknown. Randomized trials with low risk of systematic errors and low risk of random errors are needed.”
See Appendix B, pg 30. for more context about what the PHQ-9 scoring is like.
As pointed out in the report (pg 9):
% clients reached by partners:
20392 / 42482 in 2021
33148 / (33148+8823) in 2022
Thanks for these questions! I appreciate the time you took to really look at our data, and I think some of the questions will help us ponder what we need to be looking at next within StrongMinds. Please note, as the Founder, I’m not a researcher or clinician by trade, so my answers may not be as granular as you would hope, but I’ll do my best to respond. I’m going to tackle the first four tonight.
a. The NNTs could be different due to a variety of theoretical factors. NNTs are really only as good and accurate as the data provided. We are planning an RCT for 2023, so hopefully, we will have more to consider and dig into once those numbers are available. b/c. Yes, the numbers are surprising. As I’ve mentioned in some of the other responses, I largely think the success has to do with the interpersonal nature of IPT and how it works within the community-centered culture of Uganda and Zambia.
I believe the 10% figure is part of the HLI analysis, and I am not an expert on that, so I’ll let their team speak to that or let the numbers speak for themselves.
An RCT certainly could have been done as a pilot, but they are pretty costly to complete, and at the time, StrongMinds was just finding its footing and searching for funders. I was working as a volunteer at StrongMinds for the first 18 months of operation.
Our facilitators are all lay community members. In Uganda, 100% of our facilitators are Ugandan. In Zambia, they are all Zambian. They don’t need to have a college degree in Psychology; in fact, they don’t even need to have a high school degree. The most critical factor when we are interviewing potential facilitators is their empathy level. We have found that the higher their empathy level, the better they are as facilitators.
No worries! I should say that I’ve spent less than 3 hours looking through SM/HLI documents around this, so I’m not highly confident about most of these points. I have a lot of respect for anyone who is trying their best to make as much impact in the world as they can—thank you for all your work thus far, and thank you for engaging with all the questions!
I should also clarify that the digging was largely prompted by HLI’s strong endorsements:
And while this was a result of over 3 years and 10,000 hours of work, I generally aim to be more hesitant to take such strong claims at face value.
But I mention this because I want to emphasise that even if it’s the case that after this conversation, I decide that I’m not personally quite convinced that StrongMinds is the single most cost effective way to help other people, it doesn’t mean this is a reflection of the effort you have and continue to put into SM! It doesn’t necessarily mean SM isn’t a great charity. It doesn’t mean it’s not possible for StrongMinds to be the best charity in the future, or be the best under different philosophical assumptions. It’s just really hard to be the most cost effective charity.
And I’m mindful that this conversation has been possible precisely because of your shared commitment to transparency and willingness to engage, which I have a lot of respect for. We are both on the same team of wanting to do as much good as we can, and I hope you interpret this barrage (apologies!) of questions in that light.
Lastly, I’m also happy to continue via email and update folks later with a summary, if you think that would be helpful for getting answers that you may not be able to answer immediately etc.
With that in mind, some followups:
1) Just re-flagging the question RE: bias, though as you pointed out, this may be better suited for a researcher on the team / someone who was more in-the-weeds with the research:
a) What’s the justification behind the 94% figure not being found to be invalid when the 99% was? Was it based on different methodology between the two pilots, or something else? As far as I can tell, the difference in methodology RE: recording PHQ-9 scores was that in phase 1 these were scored weekly from week 5-16, with a post-assessment scoring at week 17, and for phase 2, these were done biweekly from week 2-12, with a post-assessment at week 14. It’s not clear that this difference leads to bias in one but not the other.
b) Also curious about the separate analysis that came to 92%, which states: “Since this impact figure was collected at a regular IPT group meeting, as had been done bi-weekly throughout the 12- week intervention, it is unlikely that any bias influenced the figure.” I don’t quite understand how collection at a regular IPT group meeting makes bias unlikely—could you clarify this? Presumably participants knew in advance how many weeks the intervention would be?
2) I took the 10% from StrongMinds’ 2017 report (pg 2), not an HLI analysis (though if HLI independently came to that conclusion or have reviewed it and agreed I’d be interested too):
I couldn’t find a justification of this figure in that report or any of the preceding reports. (admittedly I just very quickly searched for various combinations of 10/85/95% and didn’t read the entire report)
3) Makes sense—looking forward to the results of the RCT! I assume it will be independent and pre-registered?
4) Thanks! Just looked in a bit more detail—in Appendix A (pg 30) it says:
“Use of lay community workers as the IPT-G facilitators, or Mental Health Facilitators(MHFs)
MHFs require at least a high-school diploma, and are employed and salaried by StrongMinds. They receive two weeks of training by a certified IPT-G expert and receive on-going supervision and guidance by a mental health professional. Since they are community members themselves, they are well-received by the depressed patients. The IPT-G training curriculum includes modules on mental illness in general, depression, interpersonal psychotherapy, management of suicidality, and the goals and objectives for each weekly session of the 16 total sessions that are held. The training extensively uses role-playing to recreate group meeting settings.”
In Appendix E (pg 33) it says:
“StrongMinds completed training its initial cadre of four MHFs in March, 2014. The training lasted 10 days and was conducted by two Ugandans certified in IPT-G by Columbia University. In addition, the training was monitored long-distance via Skype by our senior technical advisor who is an international expert on IPT-G from Columbia University.”
“In Phase One of the 2014-2015 pilot in Uganda, our 4 MHFs were supervised by the two Ugandan IPT-G experts noted above. In Phase Two, StrongMinds hired a full-time Mental Health Supervisor (MHS) who both conducted IPT groups and supervised the 4 MHFs. This MHS was actually a member of the 2002 RCT in Uganda and has over ten years of IPT-G experience”
a) Just confirming that “lay counsellor” is the same as “IPT-G facilitator” and “mental health facilitator”? If not, what are the differences? How much training do they get and what’s their role in the intervention etc.
b) How does StrongMinds select for empathy? E.g. questionnaire, interview, etc.
c) What does it mean to be a “certified IPT-G expert”? For example, it sounds like there are a lot of various levels of certification. From a quick google the best match I found for the description of the training was this, which suggests a “certified IPT-G expert” is someone who has completed this specific 6-day course, i.e., with a level A certification? (Happy to be corrected—just couldn’t find any details of this). If true, am I understanding correctly that the StrongMinds lay counsellors / mental health facilitators take a 10 day training course which is delivered by someone who has taken a 6-day course? Do the certified IPT-G experts play a role in SM other than the initial training of lay counsellors?
d) What does it mean to be a “mental health supervisor”? What’s their role in SM?
e) [Minor] I just realised the appendix said MHFs require at least a high school diploma, contra what you said earlier: “in fact, they don’t even need to have a high school degree”. I assume this was just a more recent change in policy. Not a big deal, just clarifying.
5) I had another question, which came up as I was going through the tax returns Joel linked to work out the cost per client reached—in the tax return it says
This didn’t exist in the tax returns before 2019, but came up every year from 2019 onwards.
a) Was there a change in model in terms of revenue streams or business model for StrongMinds—if so, what changed?
b) You’ll probably cover this in some of the other questions, but how do the partnerships work? Do partners pay you for the year of training? What does this training look like?
c) Are there other revenue streams that StrongMinds have outside of donors / grants? (To be clear—I don’t have an issue with StrongMinds being a social enterprise, just wanting to clarify so I have the facts right!)
(commenting in personal capacity etc)
Thank you! I appreciate your curiosity, and I’m not put off by the questions or anything; it’s just many of them are not in my area of expertise, and this happens to be a pretty busy time of year at StrongMinds. It may take some time to fully gather what you’re asking for. We aren’t a large research institute by any means, so our clinical team is relatively small. Additionally, some of the work you are referencing is nearly a decade old, so we have shifted some of the ways we operate to be more effective or better based on our learnings. That said, I will dig back in when I can to help answer your additional questions via email or direct message.
To answer the remaining four from your original note to close the loop:
5) Since HLI generated the $170 figure, they have the best information on that particular breakdown, but I am collecting the most recent info on our CPP for another question, and I will share that with you later this week when I have the updated numbers.
6) As mentioned above, we are currently in the process of assessing the right questions and framework for an RCT looking at the results and impact of our therapy model. We are hoping to be able to launch the RCT late in 2023.
7) We switched our model to teletherapy to continue to treat clients during the pandemic lockdowns. It was not ideal, but we wanted to continue reaching as many women as possible despite the challenges. Though it proved tricky in some cases to reach our target demographic, we did find that some women preferred the flexibility teletherapy offered them. For the most part, we have switched back to our original model, but we still see some groups via teletherapy in Uganda. All research shared publicly from our initial year using teletherapy can be found here.
8) We track individuals that attend most of their therapy sessions, as we saw that the effects of therapy were still statistically significant and that attending additional sessions did not produce incremental impact. Due to the individual roles and responsibilities of the women that attend, it’s sometimes challenging for them to make it to all 12 sessions.
Thanks again for the questions!
I can maybe help with question 5, since the $170 figure originates from my analysis.
I finalized the cost figures during COVID when their cost figures were very high ($400 per person). I tried to project what they’d be over the next 3 years (starting in 2020) and assumed it’d come down, but the costs have come down faster than I imagined. They now say they expect 2022 to cost 105 USD per person treated.
They regularly update their cost and expense figures in their quarterly reports.
And here’s the general breakdown of their expenses according to their 2021 tax returns (page 10).
Thanks for this Joel!
RE: Q5 - sorry, just to clarify, I was interested in a breakdown of the $170 figure (or the 109 / 134/ 79 figure in the cost-per-patient graph). What does it consist of?
On skimming the HLI report it says: [1]
But I’m interested in something more fine-grained than “total annual expenses, or even “program service expenses” (per tax returns). e.g.:
$A to train lay counsellors
$B / hour for facilitators * number of hours
$C operating costs for StrongMinds (SM)
$D for outreach to SM partners
$E for SM partner operating costs
etc
I’m mindful this is asking a lot of info, sorry! I just assumed it’d be readily available, but it looks like you’ve just deferred to SM here.
I had a very brief look through the tax returns—per the tax returns you linked, the total expenses for 2021 come to 5,186,778. Per the quarterly report you linked, the total clients reached in 2021 was 42482. This means the $ per client figure should be $122? But that’s not the $134 figure reported, so I’m probably doing something wrong here.
Also, ~48% of clients were treated through partners in 2021, but does the methodology of working out cost effectiveness by dividing clients reached by SM expenses include expenses and operating costs of the partners? Q2 results (pg 2) suggest StrongMinds is on track for ~79% of clients treated through partners in 2022. If the expenses of the partners aren’t covered by SM but the clients reached are then this will make SM look more cost-effective than it is.
I also saw in the HLI report that SM defines treated patients treated here as “attending more than six sessions (out of 12) for face-to-face modes”—is this also the definition for the treatment? i.e. how did the pilot assess the effectiveness of SM for people who attended 7 sessions and then dropped out?
Do you know the answers to the other Qs too? If so, I’d be interested in your take as well! But also no worries if you prefer to leave it to Sean (I’ve edited the comment above to incorporate these Qs).
Sorry if I missed it, I just ctrl+F’ed 170 in the forum post you linked which didn’t give me a result, so I skimmed section 5 in the full HLI report. I also looked at the Q report and the tax returns but it doesn’t quite answer the question.
Unfortunately, I don’t know if I can share any information beyond the pie chart I shared above. So I’ll leave that for StrongMinds.
We did our analysis before they shifted models, so we hadn’t incorporated this. I don’t think StrongMinds includes partner costs. This will be something we revisit when we update our StrongMinds CEA (expected in 2023).
I see this as more of a concern for counterfactual impact. Where I see it as “StrongMinds got these organizations to do IPT-g, how much better is this than what they’d otherwise be doing?” But maybe I’m thinking about this wrong.
You wrote:
This sounds like you.. solved depression? And you did it in the extra challenging setting of working in Uganda and Zambia (!)
Can we import your methods to other countries, like even the U.S?
Is everyone doing therapy wrong and should instead learn from your programs?
(What am I missing? Are these questions wrong somehow?)
Hi Yonatan. Thanks for the questions. I wouldn’t quite say we solved it, but we have found a solution that has had tremendous results in Uganda and Zambia. There are probably a few reasons for this. Most of the communities we work in are very close-knit and interpersonal, so the group talk therapy model of IPT-G works well. Additionally, around 85% of women in sub-Saharan Africa don’t have access to quality mental health care. There is a huge treatment gap, for instance, Uganda has approximately 30 trained psychiatrists serving a population of 45M, so we are also filling a dire need by providing these free services using lay community health workers. The symptoms of depression are still not commonly known in Uganda and Zambia, and many people think they are just fatigued, sick, or just unmotivated/lazy, so discovering that there is a way to treat their symptoms and that there are other people experiencing this too is life-changing. Many women in therapy groups form a bond and continue to meet long after therapy ends. They now have a new support system and are no longer isolated. All of these factors play a role in our success.
The program model can be replicated in other geographical locations, and IPT has been shown to work well cross-culturally. We are currently exploring an American pilot where the need is also extreme in underserved BIPOC communities. Our pilot is happening in the Newark, NJ area just outside of NYC. That said, we have to do quite a bit of work to determine how western preferences fit into this, and we are collecting data from some initial groups now.
StrongMinds therapy model is based on an RCT from 2002 in Uganda. It has shown to be a highly effective way to reach women in underserved and remote areas that would otherwise not have access to mental health treatment options.
I’m glad this is (currently) the top-voted comment. I’ve reviewed the evidence StrongMinds offers up and find it relatively persuasive. But every page yells “offer this in wealthy countries and you’ll do good and do well at the same time.” A quick google suggests that the global ‘market’ for mental health is in the order of $400b. And this intervention seems significantly better than anything else currently in the market. Shouldn’t this be a multi-billion-dollar company being snapped up by pharma giants?
I guess that outcome seems so wild that it generates doubt about the original claim. I’d like to know the additional detail that contextualises that.
Greg S- I wish we had 400B at our fingertips because I can only imagine what impact we could have. In all honesty, our model is very low-tech. There are no apps or fancy gadgets. We also don’t use prescription medicine (which may be why a pharma company has yet to snap us up!) Finally, the women we treat mostly live on less than $2 a day, so they aren’t in an income bracket that many investors might want as consumers. While apps, tech, and pharmaceutical partnerships receive considerable funds, mental health in low and middle-income countries trying to help with poverty reduction is not heavily funded. Yearly global costs from mental, neurological, and substance use disorders are estimated at between $2.5 and $8.5 trillion dollars a year. That figure is projected to nearly double by 2030. Mental health is still frequently overlooked in health budgets. Most African governments devote less than 1% of their budgets to mental health services.
You raise a good point, though, to which we have given a little thought. Is there space to look at our model and make it self-sustainable and profitable? We would never consider this with our current clients, but is there room to have a for-profit arm that funds the non-profit, or should we consider a franchise model? We constantly push ourselves beyond what is comfortable, and we’ve had some early discussions. While I have no answers, I wanted to flag that we had considered new and different ways to fund our work.
Thanks for the reply Sean.
I guess if I was in your position I’d consider running a pilot of the service (using as-similar-a-method-as-possible) in a demographic that is seen as desirable to the health industrial complex (New York or California or something). Perhaps I’d also talk to an intellectual property lawyer and see if there are any elements of your method that can be subject to protections (patents, copyright material or whatever).
If it’s so impactful and cost-effective it might be immediately profitable if offered to other demographics (including because market-provided mental health services seem hugely expensive in many western countries—it’s possible that this model could undercut the market dramatically).
I think it would be useful to know the percentage of women with depression who we would expect to be depression-free after a six month period without any intervention.
Sam Glover- We looked at this with a control study and found that a statistically significant number of women were depression-free after completing StrongMinds therapy versus those that received no treatment. Additionally, using the PHQ-9 (an international standardized tool to assess depression), we saw an average difference of 12 pts after the conclusion of therapy. To contextualize that, in western countries, a change of 4 pts is considered significant in terms of depression recovery.
There’s a sister org working on it About Depression—StrongMinds America
+1 for having an AMA!!
I wish more orgs will do this, seems super healthy to me
Thank you! We try to be open to challenging questions at StrongMinds. It’s part of our culture, and we think asking tough questions can lead to better results.
Thanks for your amazing work! I have two questions.
StrongMinds’ work seems to suggest that depression in Uganda and Zambia is at least as treatment-responsive as depression in high-income countries, if not more. This is true despite your intervention being psychotherapeutic rather than improving income directly. How do we reconcile this with the intuition that economic conditions are very important in causing people’s depressive experiences? Should we infer from this that economic conditions are not very important in causing depression?
How does StrongMinds think about the Hawthorne effect—the possibility that clients feel obligated to report improvements in their health even if they don’t feel them? My anecdotal observations of therapy in high-income contexts suggest that people feel like if they aren’t being helped, then they must be the problem, so they report that therapy helps them in order to avoid being a burden. This seems like a challenge for measuring impact.
On (2): Here’s the section on social desirability bias from HLI’s cost-effectiveness analysis.
The experimenter demand test is quite reassuring! Although I disagree with the rest of that section, since I didn’t have in mind a conventional social desirability bias. I disagree with the idea that psychotherapy is no different from other interventions in this regard—anecdotally, depressed people are much more sensitive than average to feeling like a burden and not wanting other people to worry about their problems.
Thanks for your kind words and for the great questions!
The relationship between poverty and mental health disorders, including depression, is complex. Poverty exacerbates depression, and depression, alongside other mental health disorders, can drive people further into poverty through reduced productivity, decreased income, and isolation. (Lund et al. 2011) But it is incorrect to conclude that poverty on its own causes depression. If it did, we would see rates of depression at 100% rather than 25% in the slums of Kampala. Our model looks at four distinct triggers for depression: grief, life changes, loneliness/isolation, and conflict. Interestingly, we see from our well-being indicators that many times, restoring a woman’s mental health can increase her productivity and livelihood; for instance, 16% of our former clients reported increases in work attendance after the conclusion of therapy.
Yes, in self-reported data, we have to be very careful to ensure we measure correctly and there aren’t existing biases or misrepresentations. We work directly with an external firm to help with this. They conduct our endline evaluations. As mentioned above, we also look at well-being indicators to help demonstrate our impact, such as a 13% increase in family food security, a 30% increase in school attendance, and a 28% increase in women feeling socially connected.
What I was wondering about was not the prevalence of depression, but rather how treatable it is. For the reasons you described of poverty exacerbating mental health, it seems like depression should be much less treatable in Kampala than in the US. Yet SM’s success rate indicates that depression is at least as treatable and possibly more treatable than depression in the US. Those are the two things that I struggle to reconcile.
Measurement of other well-being indicators is a great corroboration of your results.
I agree that based on all of the barriers and life challenges that it seems like it would be harder to treat depression in Kampala. As mentioned in some of the other answers, I think the model itself, being culturally appropriate and community-centered, as well as the interpersonal bonds, play a significant role in the program’s success. Also, the fact that we work where the women are, in refugee camps, slums, or schools, to deliver our services.
Hey Sean, happy Thanksgiving!
I was wondering why StrongerMinds focuses on women. Is it because women especially suffer from depression, or maybe helping women has better downstream effects? Another reason?
Both are true! Numerous studies have shown that when you help an African Mom, the downstream effects are significantly more substantial. Globally, women have a depression rate of 1.5 times higher than that of men. An African woman with depression, compared with her healthy peers, suffers greatly: she is less productive, has a lower income, and has poorer physical health. If she is a Mother, the negative impact extends to her entire family. Research shows that children of depressed Mothers are more likely to have poor health, struggle in or miss school, and suffer from depression themselves.
Furthermore, because depression impairs the ability to focus and concentrate, depression sufferers do not respond to health initiatives or livelihood trainings, rendering these programs less effective.
This impaired ability to function in day-to-day life creates profound hardship in Uganda and Zambia, where life is community-centered and reliant on each person fulfilling her role and where depression carries a great stigma. When a woman cannot perform her social responsibilities, she can become a target of criticism and social exclusion. Women in these communities also often have far less access to resources.
Thanks so much to everyone that took the time to ask me questions and participate in this AMA. It was really refreshing to speak to so many people that are curious about our work. Many of the questions got me thinking about how we work and the ways we can do a better job moving into 2023. I appreciate all of your thoughts, ideas, and support!
In EA Israel, many people are talking about using technology to make a scalable impact in mental health. Any chance you have “generic advice” here, such as “common mistakes people make when looking for an idea for such a project” ?
I suggest doing your research (the information we received from the researchers who conducted the initial John Hopkins/Columbia University RCT was instrumental to our success.) Also, know your audience. We try to consider each woman in our therapy program’s perspective. How did she hear about us? How is she accessing information about the program? The more demographic information you have about your audience, the better you can target your outreach/messaging and achieve results. Other considerations are what staffing is needed to make the product or tech run? And what frameworks need to be in place? We had to do a lot of this evaluation ourselves when switching our model to teletherapy during the pandemic. The model, resources, and modes of communication available will all have a huge impact on the specific strategy.
Hi Sean!
I really admire what you do at StrongMinds, and I really think that mental health is a theme that isn’t picked up on much but taking care of it can do a huge difference, like with what you do! This is really amazing work and it fills me with joy to see so many women getting access to the treatment they need to live happily (and the number that remains depression-free after getting treated, wow!). I have a few questions :)
How did you work during the pandemic? I imagine it wasn’t easy to do group psychotherapy in a context of social distancing and less interpersonal connections.
Have you noticed a difference between pre-pandemic and post-pandemic depression? My own personal impression—but I believe the mental health crisis we live in currently is, in part, because of a decay in interpersonal relationships. Is the treatment even more effective because once having the group therapy these women have the whole group as a support network?
As a person who has had depression, I know it is very difficult to ask for help at the beginning, how do you convince these women to access treatment? How taboo is talking about mental health in Uganda and Zambia? Are there cultural differences you considered when developing the program?
I an engineering undergrad living in Chile, where 1⁄4 of our population has mental health illnesses such as depression and anxiety and a really poor access to treatment. What advice would you give to someone who wants to work in this area? Do you know of other effective interventions in mental health, especially for students?
Thanks so much for the AMA! :)
Thanks so much for your kind words and for following our work! I appreciate your comment about the incredible impact mental health can have. We’ve certainly seen it change women’s lives far beyond what we could have predicted, such as helping a woman’s children eat more meals a day or even helping individuals with HIV adhere to their antiretroviral treatments more regularly.
We completely switched to a teletherapy model during the COVID-19 lockdowns in Uganda and Zambia, which required a lot of creative thinking on a short timeline. This made it somewhat tricky to reach our target demographic since many women share a phone with their family or neighbors (and some don’t own a phone at all.) Fortunately, the depression-free rates were comparable to in-person group therapy, and some ultimately preferred the flexibility it offered. We usually work in communities, offering therapy directly where the women live to make access easy (our motto was often to diagnose on a doorstep, treat under a tree.) We had to reach new clients using things like radio ads, social media, and a mental health WhatsApp-based chatbot to help ensure resources were available as often as needed. You can read more about our key learnings from the COVID-19 pandemic here.
Like most in the mental health field, we saw a sharp increase in the need for services, particularly during the lockdowns. Unfortunately, since then, we have seen other climate-related humanitarian crises causing things like flooding and drought, undoubtedly worsening mental health conditions. Additionally, Uganda currently has some Ebola lockdowns in place, stressing already thinned health resources. StrongMinds has made ourselves available to support in these cases. We know mental health care is critical during high-stress times, and rates are only projected to increase in the coming decades. The good news, though, is, as you’ve eloquently stated, the interpersonal nature of group talk therapy enhances outcomes. We’ve seen that 72% of former clients we spoke to in Uganda were still in touch with their group mates. This extends years later in many cases.
Absolutely! I’ve often said that the hardest part of our job is finding depressed individuals and getting them into a group. Depression can make simply getting out of bed a challenge, so this can be tricky. The group model works well to help with this scenario, though, because group members can hold each other accountable and check in on each other. Uganda and Zambia still have many stigmas associated with mental health, so advocacy and psychoeducation are critical to our work in-country. You can read more about the complete therapy assessment process and how we work to educate about mental health here in this Psychology Today article by our talented Uganda Country Director/Clinician, Tina Ntulo.
Similarly, a small door-to-door study led by StrongMinds in Mukono District, Uganda, estimated that the prevalence of moderate to severe depression in adult women was about 20%. We also have about 85% of individuals with no access to other forms of treatment. IPT works particularly well in these settings and is currently a WHO-recommended first-line treatment for depression in low-resource settings. Other forms of community-based models using a rights-based approach are a great way to empower individuals to seek recovery and focus on the value of human rights within the mental health space.
Can you give us a breakdown of the cost of treating each additional person?
E.g. 70% salary for facilitator, 15% recruitment, 9% admin, …
Hi John! I have not forgotten this question. I aim to have an answer for you by the end of the week. I wanted to wait until our CFO had the opportunity to review to ensure we have the most recent figures for you. I’ll be back in touch on this very soon. Thanks for your interest!
Friendly reminder
Thanks so much for your awesome work!! :)
What is StrongMinds’ room for more funding, and do you expect the cost-effectiveness of the marginal dollar (ie. additional funds) to be any worse than the average cost-effectiveness of StrongMinds?
StrongMinds seeks to treat 300,000 women and adolescents between 2022 through 2024. To do that, we need to raise $30M U.S. dollars. To date, we’ve raised about $13M, which leaves us at a shortfall of $17M. So there is lots of room for more funding!
In 2022 we expect the cost to treat one patient will be $105 USD. By the end of 2024, we anticipate the cost per patient will have decreased to just $85. We will continue to reduce the cost of treating one woman even while our numbers increase. This is through effective scaling and continuing to evaluate where we can gain more cost savings. A donation to StrongMinds will be used as effectively and efficiently as possible. And when you think about what it costs for therapy in the United States, to spend just $105 and treat a woman for depression is a pretty incredible feat.
Thanks Sean, amazing work!
Do you need:
Software developers
Data Scientists / Machine Learning researchers
?
Currently, StrongMinds is actively looking to hire for a few positions listed here. We are growing and are always looking for smart, ambitious, and creative people to join our team. We encourage everyone to follow us on LinkedIn for our most up-to-date job openings. At this time, we currently do not use volunteers.
Thanks for the question! We decided to start the StrongMinds programming in Uganda because the initial RCT had such positive results there. We wanted to see if we could replicate those findings and reach even more women. I also had some experience there in my prior work as a diplomat and through my work with other NGOs. My family and I lived just across the border in the Democratic Republic of Congo for over four years, so I have many friends and connections in sub-Saharan Africa and have seen the dire need for treatment options and the consequences of untreated depression there firsthand.
Hi Sean,
(Sorry for the late question!)
Being in the field of providing solutions for mental health, what is your opinion on S-risk and Longtermism? Do you think such topics are directly in line with the cause prioritization of mental well-being?
Great questions! I should probably do a bit more reading on both topics before making a definitive statement, but just like the emphasis on well-being from Happier Lives Institute, I think considering the suffering risk of populations is a high moral priority. Mental health is an investment in the betterment of all areas of life. We see it can increase productivity, the number of meals a family eats, and the amount of school children attend. It impacts all areas of life and health, and similarly, restoring mental health creates thriving communities and stronger economies. From the longtermism philosophy, I think there is much to consider outside of simply eliminating existential threats and improving humanity. The results of climate change are already devastatingly impacting some of the communities we work in, which has serious implications for mental health and overall well-being. If we genuinely wish to eliminate suffering, we must address the crises and unrest at the root of the depression triggers.
Hi Sean, thanks for doing this, StrongMinds sounds very cool! I have two questions:
How does group therapy compare to 1:1 on treatment effectiveness? I understand group therapy is more cost-effective in aggregate but am interested to know what the average effect of moving from 1:1 therapy to group therapy is on patient outcomes.
I live in Uganda and buy SSRIs without a prescription for around 600-800 Ush (15-20c) per tablet. Based on purely anecdotal evidence they seem pretty underutilised here. Obviously not all SSRIs work for all people and there are side effects, so distributing SSRIs isn’t directly comparable to the StongMinds model, but I’d be interested to hear whether you consider distribution of SSRIs to people who meet the clinical definition for depression to be cost effective?
Hi, harriet! Thanks for the questions! Yes, there are advantages and disadvantages to both individual and group therapy, and as you mentioned, cost certainly favors the group model. We can offer a woman an entire course of treatment for $105. In terms of patient outcomes, StrongMinds doesn’t utilize individual therapy, but some studies have shown that effectiveness overall is pretty similar, and it really comes down to individual (and sometimes cultural) preferences to determine what type of therapy is best. I hope that helps!
In terms of SSRIs, yes, they can certainly be effective. In some cases where we think the client is a risk to themselves or others during therapy pre-screening, we refer them to where they have access to other methods, including SSRIs or other prescription options. That said, there are some significant challenges with the availability of SSRIs, given the treatment gap and the fact that everyone may not have the access needed to get to a trained psychiatrist/psychologist or nurse to prescribe them. This is one of the reasons the WHO recently cited and recommended community-based models and methods using lay community workers rather than trained mental health professionals as a key way to meet the growing need for services.
In terms of cost-effectiveness, prescribed medication is reasonably affordable, but the cost can add up over a lifetime of use. For $105 USD on our end (and entirely free for the women we treat), we can provide long-term depression-free rates. Our results show that for every woman who restores her mental health, four additional members of her family feel the benefits. The HLI cost analysis looks at the benefits for the whole family as well as some of the long-term impacts as well, which is what makes our model so cost-effective.