Doctor from NZ, independent researcher (grand futures / macrostrategy) collaborating with FHI / Anders Sandberg. Previously: Global Health & Development research @ Rethink Priorities.
Feel free to reach out if you think there’s anything I can do to help you or your work, or if you have any Qs about Rethink Priorities! If you’re a medical student / junior doctor reconsidering your clinical future, or if you’re quite new to EA / feel uncertain about how you fit in the EA space, have an especially low bar for reaching out.
Outside of EA, I do a bit of end of life care research and climate change advocacy, and outside of work I enjoy some casual basketball, board games and good indie films. (Very) washed up classical violinist and Oly-lifter.
All comments in personal capacity unless otherwise stated.
bruce
One benefit of reasoning transparency I’ve personally appreciated is that it helps the other party get a better sense of how much to update based on claims made.
I also think clear indication of the key cruxes of the stated argument and the level of supporting evidence can help hold us accountable in the claims we make and contribute to reducing miscommunication—how strong of a statement can be justified by the evidence I have? Am I aiming to explain, or persuade? How might my statements be misinterpreted as stronger or weaker than they are? (One example that comes to mind is the Bay of Pigs invasion which involved a miscommunication between JFK and the Joint Chiefs of Staff around understanding what “fair chance” meant).
It’s not clear to me on a quick read that the questions you’ve listed are worse off under reasoning transparency, or that actions like “clearly indicating key cruxes/level of support you have for the claims that hold your argument together” would lead to more missing facets/important connections/a meaningful big picture.
For example, if I made a claim about whether “capitalism is the right framework for EA/society in general”, would you find it less productive to know if I had done Nobel prize-winning research on this topic, if I’d run a single-country survey of 100 people, or if I was speaking just from non-expert personal experience?
If I made a claim about “What gives life meaning”, would you find it less productive if I laid out the various assumptions that I am making, or the most important considerations behind my key takeaways?
(Commenting in personal capacity etc)
As requested, here are some submissions that I think are worth highlighting, or considered awarding but ultimately did not make the final cut. (This list is non-exhaustive, and should be taken more lightly than the Honorable mentions, because by definition these posts are less strongly endorsed by those who judged it. Also commenting in personal capacity, not on behalf of other panelists, etc):
Bad Omens in Current Community Building
I think this was a good-faith description of some potential / existing issues that are important for community builders and the EA community, written by someone who “did not become an EA” but chose to go to the effort of providing feedback with the intention of benefitting the EA community. While these problems are difficult to quantify, they seem important if true, and pretty plausible based on my personal priors/limited experience. At the very least, this starts important conversations about how to approach community building that I hope will lead to positive changes, and a community that continues to strongly value truth-seeking and epistemic humility, which is personally one of the benefits I’ve valued most from engaging in the EA community.
Seven Questions for Existential Risk Studies
It’s possible that the length and academic tone of this piece detracts from the reach it could have, and it (perhaps aptly) leaves me with more questions than answers, but I think the questions are important to reckon with, and this piece covers a lot of (important) ground. To quote a fellow (more eloquent) panelist, whose views I endorse: “Clearly written in good faith, and consistently even-handed and fair—almost to a fault. Very good analysis of epistemic dynamics in EA.” On the other hand, this is likely less useful to those who are already very familiar with the ERS space.
Most problems fall within a 100x tractability range (under certain assumptions)
I was skeptical when I read this headline, and while I’m not yet convinced that 100x tractability range should be used as a general heuristic when thinking about tractability, I certainly updated in this direction, and I think this is a valuable post that may help guide cause prioritisation efforts.
The Effective Altruism movement is not above conflicts of interest
I was unsure about including this post, but I think this post highlights an important risk of the EA community receiving a significant share of its funding from a few sources, both for internal community epistemics/culture considerations as well as for external-facing and movement-building considerations. I don’t agree with all of the object-level claims, but I think these issues are important to highlight and plausibly relevant outside of the specific case of SBF / crypto. That it wasn’t already on the forum (afaict) also contributed to its inclusion here.
I’ll also highlight one post that was awarded a prize, but I thought was particularly valuable:
Red Teaming CEA’s Community Building Work
I think this is particularly valuable because of the unique and difficult-to-replace position that CEA holds in the EA community, and as Max acknowledges, it benefits the EA community for important public organisations to be held accountable (and to a standard that is appropriate for their role and potential influence). Thus, even if listed problems aren’t all fully on the mark, or are less relevant today than when the mistakes happened, a thorough analysis of these mistakes and an attempt at providing reasonable suggestions at least provides a baseline to which CEA can be held accountable for similar future mistakes, or help with assessing trends and patterns over time. I would personally be happy to see something like this on at least a semi-regular basis (though am unsure about exactly what time-frame would be most appropriate). On the other hand, it’s important to acknowledge that this analysis is possible in large part because of CEA’s commitment to transparency.- The Effective Altruism movement is not above conflicts of interest by (16 Dec 2022 15:05 UTC; 80 points)
- 's comment on CEA/EV + OP + RP should engage an independent investigator to determine whether key figures in EA knew about the (likely) fraud at FTX by (13 Nov 2022 1:49 UTC; 75 points)
- 's comment on The FTX Future Fund team has resigned by (11 Nov 2022 13:18 UTC; 42 points)
- 's comment on Ramiro’s Quick takes by (14 Dec 2022 3:27 UTC; 10 points)
- 's comment on ‘Why not effective altruism?’ — Richard Y. Chappell by (12 Feb 2024 19:02 UTC; 7 points)
- 's comment on The FTX Future Fund team has resigned by (11 Nov 2022 11:26 UTC; 4 points)
Just also want to emphasise Lizka’s role in organising and spearheading this, as well as her conscientiousness and clear communication at every step of the process—I’ve enjoyed being part of this, and am personally super grateful for all the work she has put into this contest.
Hey anonagainanon, if you DM me I can look into this for you.
(Commenting in this post because, why not?)
I’m not super familiar with forum norms personally, but a brief search suggests low relevancy might be a reason here.
I weakly think a dedicated thread to job offers would be better—active jobseekers can more easily go through a thread, and this keeps the noise : signal ratio in comment sections lower.
It also means orgs wanting to hire will have a go-to place, instead of trying to advertise on every relevant post, which would make the comment section a less enjoyable place for me personally if everyone were to do this.
(I think if this is a test for job ads that are put out ONLY by the EA Forum team, that’d be better, but still probably worse than a dedicated thread)
While I’m sympathetic to some ideas that this comment alludes to, I’ve downvoted this comment (and your comment below).
I think the tone of this message comes across to me as unnecessarily snarky/antagonistic. I interpreted the comments about luck as the author’s acknowledgement that this kind of experimentation is not feasible for everyone, and of protective factors that the author found helpful for managing difficult parts of this experimentation. I didn’t get a sense that the author was minimising her mental health by comparing herself with people who are less well-off, which is one uncharitable interpretation of your comment.
I think I might be biased here because I would find it difficult to share a personal post like this publicly, and so perhaps have a higher standard for pushbacks that don’t address the main points of the post, but feel more like nitpicks on how these kinds of personal journeys are communicated/what the author should and shouldn’t acknowledge as helpful for them. I worry that comments like this can be (mis)interpreted as potential barriers to other people sharing posts I’d be happy to see on the forum.RE: your medical advice comment below—I viewed the disclaimer as helpful reasoning transparency to know what her background knowledge is and how she went about investigating this. I think there are also legal reasons that including a disclaimer is useful, even if the author was confident this post was as helpful as the average mental health professional’s advice.
I also think statements like “the illusion that mental health professionals usually know what they’re doing” and “most people here can do better by trusting their own cursory research” seem too strong as standalone claims. While I agree there are doctors who are bad, and doctors who are not clearly good, it’s a few steps further to suggest that mental health professionals usually don’t know what they are doing, and that most people should do their own cursory research instead of seeking input from mental health professionals. I would have found it helpful to see justification that matched the strength of those claims, or more epistemic legibility.
For example, if it is the case that people here can benefit (on net) from input from mental health professionals, then your comment may be harmful in a similar way, by perpetuating the illusion that mental health professionals usually don’t know what they’re doing, and by nudging people towards trusting their own research instead of seeking professional help. It’s unclear from the outside that it’d be valuable to update based on what you’ve said.
(Speaking in personal capacity etc)
Hi, thanks for reaching out! I’ll DM you.
Thanks for this! I echo Lizka’s comment about linkposting.
In light of the recent events I’m struggling a bit with taking my hindsight-bias shades off, and while I scored it reasonably highly, I don’t think I can fairly engage with whether it should have received a prize over other entries even if I had the capacity to (let alone speak for other panelists). I do remember including it in the comment mainly because I thought it was a risk that didn’t receive enough attention and was worth highlighting (though I have a pretty limited understanding of the crypto space and ~0 clue that things would happen in the way they did).
I think it’s worth noting that there has been at least one other post on the forum that engaged with this specifically, but unfortunately didn’t receive much attention. (Edit: another one here)
Ultimately though, I think it’s more important to think about what actionable and constructive steps the EA community can take going forward. I think there are a lot of unanswered questions wrt accountability from EA leaders in terms of due diligence, what was known or could have been suspected prior to Nov 9th this year, and what systems or checks/balances were in place etc that need to be answered, so the community can work out what the best next steps are in order to minimise the likelihood of something like this from happening again.I also think there are questions around how these kinds of decisions are made when benefits affect one part of the EA community but the risks are pertinent to all, and how to either diversify these risks, or make decision-making more inclusive of more stakeholders, keeping in mind the best interests of the EA movement as a whole.
This is something I’m considering working on at the moment and will try and push for—do feel free to DM me if you have thoughts, ideas, or information.
(Commenting in personal capacity etc)
If this comment is more about “how could this have been foreseen”, then this comment thread may be relevant. I should note that hindsight bias means that it’s much easier to look back and assess problems as obvious and predictable ex post, when powerful investment firms and individuals who also had skin in the game also missed this.
TL;DR:
1) There were entries that were relevant (this one also touches on it briefly)
2) They were specifically mentioned
3) There were comments relevant to this. (notably one of these was apparently deleted because it received a lot of downvotes when initially posted)
4) There has been at least two other posts on the forum prior to the contest that engaged with this specifically
My tentative take is that these issues were in fact identified by various members of the community, but there isn’t a good way of turning identified issues into constructive actions—the status quo is we just have to trust that organisations have good systems in place for this, and that EA leaders are sufficiently careful and willing to make changes or consider them seriously, such that all the community needs to do is “raise the issue”. And I think looking at the systems within the relevant EA orgs or leadership is what investigations or accountability questions going forward should focus on—all individuals are fallible, and we should be looking at how we can build systems in place such that the community doesn’t have to just trust that people who have power and who are steering the EA movement will get it right, and that there are ways for the community to hold them accountable to their ideals or stated goals if it appears to, or risks not playing out in practice.
i.e. if there are good processes and systems in place and documentation of these processes and decisions, it’s more acceptable (because other organisations that probably have a very good due diligence process also missed it). But if there weren’t good processes, or if these decisions weren’t a careful + intentional decision, then that’s comparatively more concerning, especially in context of specific criticisms that have been raised,[1] or previous precedent. For example, I’d be especially curious about the events surrounding Ben Delo,[2] and processes that were implemented in response. I’d be curious about whether there are people in EA orgs involved in steering who keep track of potential risks and early warning signs to the EA movement, in the same way the EA community advocates for in the case of pandemics, AI, or even general ways of finding opportunities for impact. For example, SBF, who is listed as a EtG success story on 80k hours, has publicly stated he’s willing to go 5x over the Kelly bet, and described yield farming in a way that Matt Levine interpreted as a Ponzi. Again, I’m personally less interested in the object level decision (e.g. whether or not we agree with SBF’s Kelly bet comments as serious, or whether Levine’s interpretation as appropriate), but more about what the process was, how this was considered at the time with the information they had etc. I’d also be curious about the documentation of any SBF related concerns that were raised by the community, if any, and how these concerns were managed and considered (as opposed to critiquing the final outcome).
Outside of due diligence and ways to facilitate whistleblowers, decision-making processes around the steering of the EA movement is crucial as well. When decisions are made by orgs that bring clear benefits to one part of the EA community while bringing clear risks that are shared across wider parts of the EA community,[3] it would probably be of value to look at how these decisions were made and what tradeoffs were considered at the time of the decision. Going forward, thinking about how to either diversify those risks, or make decision-making more inclusive of a wider range stakeholders[4], keeping in mind the best interests of the EA movement as a whole.
(this is something I’m considering working on in a personal capacity along with the OP of this post, as well as some others—details to come, but feel free to DM me if you have any thoughts on this. It appears that CEA is also already considering this)If this comment is about “are these red-teaming contests in fact valuable for the money and time put into it, if it misses problems like this”
I think my view here (speaking only for the red-teaming contest) is that even if this specific contest was framed in a way that it missed these classes of issues, the value of the very top submissions[5] may still have made the efforts worthwhile. The potential value of a different framing was mentioned by another panelist. If it’s the case that red-teaming contests are systematically missing this class of issues regardless of framing, then I agree that would be pretty useful to know, but I don’t have a good sense of how we would try to investigate this.
- ^
This tweet seems to have aged particularly well. Despite supportive comments from high-profile EAs on the original forum post, the author seemed disappointed that nothing came of it in that direction. Again, without getting into the object level discussion of the claims of the original paper, it’s still worth asking questions around the processes. If there was were actions planned, what did these look like? If not, was that because of a disagreement over the suggested changes, or the extent that it was an issue at all? How were these decisions made, and what was considered?
- ^
Apparently a previous EA-aligned billionaire ?donor who got rich by starting a crypto trading firm, who pleaded guilty to violating the bank secrecy act
- ^
Even before this, I had heard from a primary source in a major mainstream global health organisation that there were staff who wanted to distance themselves from EA because of misunderstandings around longtermism.
- ^
This doesn’t have to be a lengthy deliberative consensus-building project, but it should at least include internal comms across different EA stakeholders to allow discussions of risks and potential mitigation strategies.
- ^
- ^
Just to be more concrete, what would you say is an example of a behaviour that you think does not warrant action, because “the harm from this kind of behaviour is not much more important than the contributions from the same people”?
And where would you personally draw the line? i.e., what does the most harmful example look like that still does not warrant action, because the harm is not much more important the contributions?
While I agree that both sides are valuable, I agree with the anon here—I don’t think these tradeoffs are particularly relevant to a community health team investigating interpersonal harm cases with the goal of “reduc[ing] risk of harm to members of the community while being fair to people who are accused of wrongdoing”.
One downside of having the bad-ness of say, sexual violence[1]be mitigated by their perceived impact,(how is the community health team actually measuring this? how good someone’s forum posts are? or whether they work at an EA org? or whether they are “EA leadership”?) when considering what the appropriate action should be (if this is happening) is that it plausibly leads to different standards for bad behaviour. By the community health team’s own standards, taking someone’s potential impact into account as a mitigating factor seems like it could increase the risk of harm to members of the community (by not taking sufficient action with the justification of perceived impact), while being more unfair to people who are accused of wrongdoing. To be clear, I’m basing this off the forum post, not any non-public information
Additionally, a common theme about basically every sexual violence scandal that I’ve read about is that there were (often multiple) warnings beforehand that were not taken seriously.
If there is a major sexual violence scandal in EA in the future, it will be pretty damning if the warnings and concerns were clearly raised, but the community health team chose not to act because they decided it wasn’t worth the tradeoff against the person/people’s impact.
Another point is that people who are considered impactful are likely to be somewhat correlated with people who have gained respect and power in the EA space, have seniority or leadership roles etc. Given the role that abuse of power plays in sexual violence, we should be especially cautious of considerations that might indirectly favour those who have power.
More weakly, even if you hold the view that it is in fact the community health team’s role to “take the talent bottleneck seriously; don’t hamper hiring / projects too much” when responding to say, a sexual violence allegation, it seems like it would be easy to overvalue the bad-ness of the immediate action against the person’s impact, and undervalue the bad-ness of many more people opting to not get involved, or distance themselves from the EA movement because they perceive it to be an unsafe place for women, with unreliable ways of holding perpetrators accountable.
That being said, I think the community health team has an incredibly difficult job, and while they play an important role in mediating community norms and dynamics (and thus have corresponding amount of responsibility), it’s always easier to make comments of a critical nature than to make the difficult decisions they have to make. I’m grateful they exist, and don’t want my comment to come across like an attack of the community health team or its individuals!
(commenting in personal capacity etc)
Thanks for sharing this report! Happy to see more global health content on the forum.
Two quick questions on skimming it:When working out the marginal value of sodium taxation policy advocacy, do you discount for the possibility of salt reduction policies happening independent of any EA efforts in this area? (i.e. your advocacy is speeding up the policy passing by X number of years as opposed to bringing it in a world where it otherwise would never have existed). For example, if you crudely take 2023 figures, and assume that the intervention successfully speeds up salt taxes globally by 5 years compared to a world without any additional resources, the this would mean ~1.2E9 DALYs are at stake, compared to your topline figure of 1.2E10 DALYs. [1]
The cost per intervention of US$137,000 for getting a salt tax advocacy are in part based on assessments like:
i) “I assess that around 1 year of operations is a reasonable timeframe for the charity to lobby/prepare supporting research reports on the health and economics benefits of the tax/do public polling to show public support/construct a coalition of NGOs and advocates in support and hence succeed”, and
ii) “my sense is that this is doable by a 2 person EA charity in the style of Charity Entrepreneurship incubatees”.
I wonder whether you’d be happy to elaborate on these assessments / how you got to these figures, as both seem optimistic to me (though perhaps I’m too cynical).
Thanks again for doing this work!
(commenting in personal capacity etc)- ^
2.43E8 * 5 = 1.215E9
Just chiming in with an extra anecdotal data point that (on my laptop at least) I think the design looks great, from colour scheme to font choice—it’s clear that a lot of effort has been put into this. I also really like the save highlight function, which I hadn’t seen before, and thought it was a neat design choice to use an asterix there too (as well as the blurbs that come up when you hover over titles). I’ve only skimmed 1 article so far so can’t comment on the content, but definitely would not hesitate to recommend this to people based on its current design, and I’d probably also anti-recommend adding dall-e images (at least the 4 that have come up).
Thanks to Clara and the team who have put this together!
No worries! I’ll DM ya some additional thoughts :)
Thanks again!
Many thanks for doing this AMA!
I’m personally excited about more work in the EA space on topics around mental health and subjective well-being, and was initially excited to see StrongMinds (SM) come so strongly recommended. I do have a few Qs about the incredible success the pilots have shown so far:[1]
I couldn’t find number needed to treat (NNT)[2] figures anywhere (please let me know if I’ve missed this!), so I’ve had a rough go based on the published results, and came to an NNT of around 1.35.[3] Limitations of the research aside, this suggests StrongMinds is among the most effective interventions in all of medicine in terms of achieving its stated goals.
If later RCTs and replications showed much higher NNT figures, what do you think would be the most likely reason for this? For comparison:
This meta-analysis suggests an NNT of 3 when comparing IPT to a control condition;
This systematic review suggests an NNT of 4 for interpersonal therapy (IPT) compared to treatment as usual[4];
This meta-analysis suggests a response rate of 41% and an NNT of 4 when comparing therapy to ‘waitlist’ conditions (and lower when only considering IPT in subgroup analyses); or
this meta-analysis which suggests an NNT of 7 when comparing psychotherapies to placebo pill.
Admittedly, there are many caveats here—the various linked studies aren’t a perfect comparison to SM’s work, NNT clearly shouldn’t be used as sole basis for comparison between interventions, and I haven’t done enough work here to feel super confident about the quality of SM’s research. But my initial reaction upon skimming and seeing response to treatment in the range of 94-99%, or 100+ people with PHQ-9 scores of over 15 basically all dropping down to 1-4[5] (edit: an average improvement of 12 points after conclusion of therapy) after 12-16 weeks of group IGT by lay counsellors was that this seemed far “too good to be true”, and fairly incongruent with ~everything I’ve learnt or anecdotally seen in clinical practice about the effectiveness of mental health treatments (though clearly I could be wrong!). This is especially surprising given SM dropped the group of participants with minimal or mild depression from the analysis.[6]
Were these concerns ever raised by the researchers when writing up the reports? Do you have any reason to believe that the Ugandan context or something about the SM methodology makes your intervention many times more effective than basically any other intervention for depression?
[Edit: I note that the 99% figure in the phase 2 trial was disregarded, but the 94% figure in phase 1 trial wasn’t, despite presumably the same methodology? Also curious about the separate analysis that came to 92%, which states: “Since this impact figure was collected at a regular IPT group meeting, as had been done bi-weekly throughout the 12- week intervention, it is unlikely that any bias influenced the figure.” I don’t quite understand how collection at a regular IPT group meeting makes bias unlikely—could you clarify this? Presumably participants knew in advance how many weeks the intervention would be?]
How did you come to the 10% figure when adjusting for social desirability bias?
Was there a reason an RCT couldn’t have been done as a pilot? Just noting that “informal control populations were established for both Phase 1 and Phase 2 patients, consisting of women who screened for depression but did not participate”, and the control group in both the pilots were only 36 people, compared to the 244 and 270 in the treatment arm for phase 1 and phase 2 respectively. As a result, 11 / 24 of the villages where the interventions took place did not have a control arm at all. (pg 9)
Are you happy to go into a bit more detail about the background of the lay counsellors? E.g. what they know prior to the SM pilots, how much training (in number of hours) they receive, and who runs it (what relevant qualifications / background? How did the trainers get their IPT-G certification—e.g. is this a postgrad psychology qualification, or a one-off training course?) I briefly skimmed the text (appendix A + E) but also got a bit confused over the difference between “lay counsellor”, “mental health facilitator”, “mental health supervisor” and “senior technical advisor” and how they’re relevant for the intervention.
Can you give us a cost breakdown of $170 / person figure for delivering the programme (Or $134 for 2021)? See Joel’s response and subsequent discussion for more details. Specifically, whether the methodology for working out the cost / client by dividing total clients reached over SM’s total expenses means that this includes the clients reached by the partners, but not their operating costs / expenses. For example, ~48% of clients were treated through partners in 2021, and Q2 results (pg 2) suggest StrongMinds is on track for ~79% of clients treated through partners in 2022.[7] Or are all expenses of SM partners covered by SM and included in the tax returns?
In the most recent publication (pg 5), published 2017, the report says: “Looking forward, StrongMinds will continue to strengthen our evaluation efforts and will continue to follow up with patients at 6 or 12 month intervals. We also remain committed to implementing a much more rigorous study, in the form of an externally-led, longitudinal randomized control trial, in the coming years.”
Have either the follow-up or the externally-led longitudinal RCT happened yet? If so, are the results shareable with the public? (I note that there has been a qualitative study done on a teletherapy version published in 2021, but no RCT.)
The pivot to teletherapy in light of COVID makes sense, though the evidence-base for its effectiveness is ?presumably weaker.
What’s the breakdown of % clients reached via teletherapy versus clients reached via group IGT as per the original pilots (i.e. in person)
In the 2021 report on a qualitative assessment of teletherapy (pg 2), it says: “Data from StrongMinds shows that phone-based IPT-G is as effective as in-person group therapy in reducing depression symptoms among participants”. Is this research + methodology available to the public? (I searched for phone and telehealth in the other 4 reports which returned no hits)
Does StrongMinds have any other unpublished research?
What’s the plan with telehealth going forward? Was this a temporary thing for COVID, or is this a pivot into a more / similarly effective approach?
I also saw in the HLI report that SM defines treated patients treated for purpose of cost analysis as “attending more than six sessions (out of 12) for face-to-face modes and more than four (out of 8) for teletherapy.”—is this also the definition for the treatment outcomes? i.e. how did SM assess the effectiveness of SM for people who attended 7 sessions and then dropped out? Do we have more details around about how many people didn’t do all sessions, how they responded, and how this was incorporated into SM’s analyses?
Thanks again!
(Commenting in personal capacity etc)
[Edited after Joel’s response to include Q7, Q8, and an update to Q1c and Q5, mainly to put all the unresolved Qs in one place for Sean and other readers’ convenience.]
[Edited to add this disclaimer.]
[Edited to include a link to a newer post StrongMinds should not be a top-rated charity (yet), which includes additional discussion.]
- ^
Apologies in advance if I’ve missed anything—I’ve only briefly skimmed your website’s publications, and I haven’t engaged with this literature for quite a while now!
- ^
Quick primer on NNT for other readers. Lower = better, where NNT = 1 means your treatment gets the desired effect 100% of the time.
- ^
SM’s results of 95% depression-free (85% after the 10% adjustment for social desirability bias) give an EER of 0.15 after adjustment. By a more conservative estimate, based on this quote (pg 3): “A separate control group, which consisted of depressed women who received no treatment, experienced a reduction of depressive symptoms in only 11% of members over the same 12-week intervention period” and assuming all of those are clinically significant reductions in depressive symptoms, the CER is 0.89, which gives an NNT of 1 / (0.89 − 0.15) = 1.35. The EER can be adjusted upwards because not all who started in the treatment group were depressed, but this is only 2% and 6% for phase 1 and 2 respectively—so in any case the NNT is unlikely to go much higher than 1.5 even by the most conservative estimate.
- ^
They also concluded: “We did not find convincing evidence supporting or refuting the effect of interpersonal psychotherapy or psychodynamic therapy compared with ‘treatment as usual’ for patients with major depressive disorder. The potential beneficial effect seems small and effects on major outcomes are unknown. Randomized trials with low risk of systematic errors and low risk of random errors are needed.”
- ^
See Appendix B, pg 30. for more context about what the PHQ-9 scoring is like.
- ^
As pointed out in the report (pg 9):
A total of 56 participants with Minimal or Mild Depression (anyone with total raw scores between 1-9) at baseline in both the treatment intervention (46 participants) and control (10 participants) groups were dropped from the GEE analysis of determining the depression reduction impact. In typical practice around the world, individuals with Minimal/Mild Depression are not considered for inclusion in group therapy because their depressive symptoms are relatively insignificant. StrongMinds consciously included these Minimal/Mild cases in Phase Two because these patients indicated suicidal thoughts in their PHQ-9 evaluation. However, their removal from the GEE analysis serves to ensure that the Impact Evaluation is not artificially inflated, since reducing the depressive symptoms of Minimal/Mild Depressive cases is generally easier to do.
- ^
% clients reached by partners:
20392 / 42482 in 2021
33148 / (33148+8823) in 2022
- Where are you donating this year, and why? (Open thread) by (23 Nov 2022 12:26 UTC; 151 points)
- 's comment on StrongMinds should not be a top-rated charity (yet) by (28 Dec 2022 1:15 UTC; 76 points)
- 's comment on The Happier Lives Institute is funding constrained and needs you! by (10 Jul 2023 22:22 UTC; 45 points)
- 's comment on AMA: Sean Mayberry, Founder & CEO of StrongMinds by (29 Nov 2022 23:19 UTC; 3 points)
Hey team!
I’d love if someone can give me a TL;DR on donation matching—it’s something I always get a bit confused about in terms of like “how much more should I donate because of this”. And someone asked in a slack I was in about counterfactuals, which I realised I didn’t know about either—how else is the money usually used?Also, does anyone know what the optimal % split between donating to a matching pool vs donating to the charity (am I basically trading off between how much a matching pool actually increases the pie VS the money not being donated?), and how does this change if the org is fully EA funded vs partially vs not at all etc?
I’m pretty sympathetic to patient philanthropy for longtermist causes that aren’t to do with nearterm Xrisks, because my view is that as long as we preserve option value for the future, they will likely be better placed to use the resources than we are, so we should just save the pool for them to use as they see fit.
The example I usually give when explaining my position is thinking about polio and the iron lung. Say someone in the 1910s wanted to invest in significant iron lung production facilities to make sure polio would never be a problem in the future. 20 years later, the polio vaccine is created and all this investment is obsolete. If that money was saved it could perhaps be used to speed up the distribution of polio vaccines and help eradicate polio etc.
One uncertainty I have about this though, is that I don’t know how to implement this in practice (what % to give later vs give now? How do I know when I should use this pool?). Curious about any takes!
Thanks for this Joel!
RE: Q5 - sorry, just to clarify, I was interested in a breakdown of the $170 figure (or the 109 / 134/ 79 figure in the cost-per-patient graph). What does it consist of?
On skimming the HLI report it says: [1]
StrongMinds records the average cost of providing treatment to an additional person (i.e. total annual expenses / no. treated) and has shared the most recent figures for each programme with us.
But I’m interested in something more fine-grained than “total annual expenses, or even “program service expenses” (per tax returns). e.g.:
$A to train lay counsellors
$B / hour for facilitators * number of hours
$C operating costs for StrongMinds (SM)
$D for outreach to SM partners
$E for SM partner operating costs
etcI’m mindful this is asking a lot of info, sorry! I just assumed it’d be readily available, but it looks like you’ve just deferred to SM here.
I had a very brief look through the tax returns—per the tax returns you linked, the total expenses for 2021 come to 5,186,778. Per the quarterly report you linked, the total clients reached in 2021 was 42482. This means the $ per client figure should be $122? But that’s not the $134 figure reported, so I’m probably doing something wrong here.
Also, ~48% of clients were treated through partners in 2021, but does the methodology of working out cost effectiveness by dividing clients reached by SM expenses include expenses and operating costs of the partners? Q2 results (pg 2) suggest StrongMinds is on track for ~79% of clients treated through partners in 2022. If the expenses of the partners aren’t covered by SM but the clients reached are then this will make SM look more cost-effective than it is.
I also saw in the HLI report that SM defines treated patients treated here as “attending more than six sessions (out of 12) for face-to-face modes”—is this also the definition for the treatment? i.e. how did the pilot assess the effectiveness of SM for people who attended 7 sessions and then dropped out?
Do you know the answers to the other Qs too? If so, I’d be interested in your take as well! But also no worries if you prefer to leave it to Sean (I’ve edited the comment above to incorporate these Qs).
- ^
Sorry if I missed it, I just ctrl+F’ed 170 in the forum post you linked which didn’t give me a result, so I skimmed section 5 in the full HLI report. I also looked at the Q report and the tax returns but it doesn’t quite answer the question.
- ^
To an outsider who might be suspicious that EA or EA-adjacent spaces seem cult-y, or to an insider who might think EAs are deferring too much, how would EA as a movement do the above and successfully navigate between:
1) an outcome where the goals of maintaining/improving epistemic quality for the EA movement, and keeping EA as a question are attained, and
2) an outcome where EA ends up self-selecting for those who are most likely to defer and embrace “ideas that aren’t widely accepted”, and doesn’t achieve the above goal?
The assumption here is that being perceived as a cult or being a high-deferral community would be a bad outcome, though I guess not everyone would necessarily agree with this.
(Caveat: very recently went down the Leverage rabbit hole, so this is on the front of my mind and might be more sensitive to this than usual.)
Agreed, though RE: “AI alignment is the top issue” I think it’s important to distinguish between whether they think:
AI misalignment is the most likely reason to cause human extinction/cause global suffering (+/- within [X timeframe]).
Donating to AI alignment is the most cost-effective place to donate for all worldviews
Donating to AI alignment is the most cost-effective place to donate for [narrower range of worldviews]
Contributing to direct AI alignment work is the best career decision for all people
Contributing to direct AI alignment work is the best career decision for [narrower range of people].
Prioritising AI alignment is the best way to maximise impact for EA as a movement (on the margin? at scale?)
Do you have a sense of where the consensus falls for those you consider EA leaders?
(Commenting in personal capacity etc)