A few comments, not intended as a knock on Vida Plena’s programme, but perhaps more relevant to how it’s communicated:
You can save a life as depression not only causes immense human suffering but is a deadly disease. Approximately 24% of the individuals we support are at high risk of suicide and as a result, face a 3- to 11-fold increased likelihood of dying by suicide within the next 30 days.
Given this is the first bullet under “helping a life flourish” I thought this might benefit from some clarification, because the vast majority of the value of this programme is likely not from suicide prevention, given low absolute rates of suicide.
From the same source: “at two years, the cumulative hazard of suicide death ranged from 0.12% in young adults to 0.18% in older adults.” Under unreasonably optimistic assumptions,[1] Vida Plena would prevent 1 suicide every 500 participants / prevent a suicide for $116,500, which is something between 21x to 39x less cost effective than GiveWell top charities.[2] More reasonable assumptions would drop this upper bound to 1 suicide prevented every ~1200 participants, or ~$272,000 per suicide prevented / ~50-90x less effective than GW top charities.[3]
This isn’t a claim that the cost-effectiveness claims are necessarily incorrect, even with minimal suicide prevention. A quick sense check RE: $462/DALY and 0.22 DALYs per participant would imply that Vida Plena would need to more than halve their cost per participant (from $233 down to $101), and then achieve results comparable to “~100% of people with severe / moderate mild depression conclude the programme going down one level of severity or something like ~5 points on the PHQ9 score (severe --> moderate; moderate --> mild; mild --> no depression).”[5] This is well within your listed results—though as you note in your annual report these have some fairly significant sources of bias and (IMO) probably should not be taken at face value.
Some other comments:
The NBER paper quoted in “g-IPT has also demonstrated long-term positive effects” looked at the “Healthy Activity Programme” (HAP)[6] and the “Thinking Healthy Programme Peer-Delivered” (THPP).[7] Neither of these are g-IPT programmes.
The minimal and unsustained results from the Baird RCT may be worth incorporating in an updated analysis, given the predictive CEA is from 2022[8]
From the predictive CEA: “Vida Plena’s overall effect for a household is 7.18*0.75*0.83 = 4.49 (95% CI: 0.77, 31.04) WELLBYs per person treated”. HLI recently decreased their estimate for StrongMinds treatment effects by 80% from 10.49 to 2.15 WELLBYs per treatment (also including household spillovers, and estimated StrongMinds to be “3.7x (previously 8x) as cost-effective as GiveDirectly”.
The cost-effectiveness of GiveDirectly has gone up by 3-4x (GW blog, GD blog), though this was recent news and does not necessarily imply that WELLBYs will also go up by 3-4x (most of this increase is attributable to increased consumption) - but should constitute a discount at least.
Even if 100% (rather than 24%) of individuals were in the high risk group (i.e. suicidal ideation nearly every day), and even if you dropped 100% of individuals risk of suicide from 0.2% to zero (rather than reducing it by 3-11x or to baseline), and if this effect persisted forever rather than just the initial 30 days
If 24% of your participants were high risk (7x risk, at 0.18%), and the other 76% of them were half of that (3.5x risk, at 0.09%), and you successfully reduced 100% of participants to baseline (0.026%), you would prevent 1 suicide every 1169 participants, which comes to ~$272,000 per life saved, or ~50-90x less cost effective than GW top charities.
It’s also worth noting these are cumulative hazards at 2 years rather than 30, and the hazard ratios at 365 days are approximately halved compared to 30 days (1.7- to 5.7 instead of 3.3-10.8), so these figures are plausibly a few factors optimistic still.
Severe --> moderate depression is about 0.262 DALYs averted, moderate --> mild depression is about 0.251 DALYs averted, and mild --> no depression is about 0.145 DALYs averted.
HAP is described as “a psychological treatment based on behavioral activation...consist[ing] of 6 to 8 weekly sessions of 30 to 40 minutes each, delivered individually at participants’ homes or at the local PHC.”
THPP is a simplified version of a psychological intervention (THP) for treating perinatal depression that has been found to be effective in similar settings and is recommended by the WHO (Rahman et al., 2008, 2013; WHO, 2015; Baranov et al., 2020). While the original THP trials employed a full-fledged cognitive behavioral therapy (CBT) intervention, THPP was a simpler intervention focused on behavioral activation, as in the HAP trial described above. THPP was designed to be delivered by peer counselors, instead of community health workers as in previous trials.
-Our findings add to this evidence base by showing 12-month modest improvements of 20%-30% in rates of minimal depression for adolescents assigned to IPT-G, with these effects completely dissipating by the 24-month follow-up. We similarly find small short-term impacts on school enrollment, delayed marriage, desired fertility and time preferences, but fail to conclude that these effects persist two years after therapy.
-Given impact estimates of a reduction in the prevalence of mild depression of 0.054 pp for a period of one year, it implies that the cost of the program per case of depression averted was nearly USD 916, or 2,670 in 2019 PPP terms.
-This implies that ultimately the program cost USD PPP (2019) 18,413 per DALY averted. (almost 8x Uganda’s GDP per capita)
Hi Bruce, great to reconnect here! Thank you for all your thoughtful comments. I really appreciate your perspectives. Here are my responses to your points.
Yes, we are primarily a “life-improving” intervention rather than a life-saving one. While we do track suicidal ideation and believe we may have prevented several suicides, proving this counterfactually is challenging. So, when calculating our impact, we consider lives saved as a “bonus” rather than our primary measure of impact. As such, we don’t account for that in our cost-effectiveness estimates.
We are on track to improve our cost per person (but still not there). The $233 cost per person reported for 2023 included our total spending for the year, which involved significant consulting fees to Columbia University for their supervision in our first year. Excluding those fees, which are not recurring, our actual cost per person was $152 in 2023. We expect 2024 to be in that range.
Regarding your other notes:
The NBER paper was an incorrect reference—thank you for catching that. The link has been updated.
We don’t believe the Baird RCT is especially relevant to our work because 1) it focuses on adolescents, 2) it was delivered by non-specialist, age-peer providers (i.e., other adolescents), and 3) the “12-month follow-up surveys were conducted at the height of the pandemic.” A hard moment for everyone.
We also saw that update from GiveWell, although they noted, “we will be using our historic benchmark until we have thought it through.” So, we think it’s fair to continue using the historic benchmark until they switch.
So appreciate the thoughtful feedback, it helps us to refine our thinking and keep improving!
Thanks for the response, and likewise—hope you’ve been well! (Sorry I wasn’t sure if it was you or someone else on the account).
I agree that it is pretty reasonable to stick with the same benchmark, but I think this means it should be communicated accordingly, as VP are sometimes referring to a benchmark and other times referring to the GD programme, while GW are sticking to the same benchmark for their cost-effectiveness analyses, but updating their estimates of GD programmes.[1]
“This means that a $1000 donation to Vida Plena would produce 58 WELLBYs, which is 8 times more cost-effective than GiveDirectly (a charity that excels in delivering cash transfers—simply giving people money—and a gold standard in effective altruism)”[2]
I think people would reasonably misinterpret this to mean you are referring to the GD programme, rather than the GW benchmark.[3] Again I know this is a v recent update and so hadn’t expected it to be already updated! But just flagging this as a potential source of confusion in the future.
Separately, I just thought I’d register interest in a more up-to-date predictive CEA that comes before your planned 2026 analysis, in part because there’s decent reason to do so (though I’m not making the stronger claim that this is more important than other things on your plate!), 2026 is a while away, and because it’s plausibly decision relevant for potential donors if they’re not sure the extent to which HLI updates might be applicable to VP.
“Thus, we will be using our historic benchmark until we have thought it through. For now, you can think of our benchmark as “GiveWell’s pre-2024 estimate of the impacts of cash transfers in Kenya,” with GiveDirectly’s current programs in various countries coming in at 3 to 4 times as cost-effective as that benchmark.”
To VP’s credit, I think “eight times more cost-effective than the benchmark of direct cash transfers” in this post would likely be interpreted correctly in a high context setting (but I also think reasonably might not be, and so may still be worth clarifying).
The cost-effectiveness of GiveDirectly has gone up by 3-4x (GW blog, GD blog), though this was recent news and does not necessarily imply that WELLBYs will also go up by 3-4x (most of this increase is attributable to increased consumption) - but should constitute a discount at least.
I’m not sure about this; the HLI’s analysis of GiveDirectly only looks at direct individual effects and household spillovers. Whereas GiveWell’s update seemingly only found additional effects in terms of non-household spillovers, mortality, and consumption (based on a 5 minute check, so I might be wrong here).
I think it’s reasonable to argue that depression prevention would also have effects on mortality, consumption (via productivity increases—my guesses here peg this quite high, especially in LMICs and UMICs), and non-household spillovers (via increased income being reinvested into communities, using the same mechanism as GD). Unless there’s reason to believe that the non-accounted-for impacts on WELLBYs systematically favour GiveDirectly I’d be cautious about applying a discount—but curious for your take on that :)
I might be misunderstanding you, but are you saying that after reading the GD updates, we should update on VP equally to GD / that we should expect the relative cost-effectiveness ratio between the two to remain the same?
Hmm, no. I wouldn’t want Vida Plena to update without evidence that they have those secondary effects.
But I think it would also be misleading to compare direct effects + household spillovers (in the case of Vida Plena) to direct effects + household spillovers + community spillovers + mortality reduction + consumption increases (GiveDirectly), unless you had good reason to believe that Vida Plena’s secondary effects are much worse than GiveDirectly’s. So I suppose I would be wary of saying that GiveDirectly now have 3–4x the WELLBY impact relative to Vida Plena—or even to say that GiveDirectly have any more WELLBY impact relative to Vida Plena—without having a good sense of how Vida Plena performs on those secondary outcomes. (But I feel like maybe I’m misunderstanding what you meant by applying a discount?)
So I suppose I would be wary of saying that GiveDirectly now have 3–4x the WELLBY impact relative to Vida Plena—or even to say that GiveDirectly have any more WELLBY impact relative to Vida Plena
Ah right—yeah I’m not making either of these claims, I’m just saying that if the previous claim (from VP’s predictive CEA) was that: “Vida Plena...is 8 times more cost-effective than GiveDirectly”, and GD has since been updated to 3-4x more cost-effective than it was compared to the time the predictive CEA was published, we should discount the 8x claim downwards somewhat (but not necessarily by 3-4x).
thoughtful comment! just want to throw in that suicide should not be considered in isolation imo. while every avoidable death is horrible ofc, I do think that suicide has particularly bad knock-on effects.
Hey team, thanks for sharing this update!
A few comments, not intended as a knock on Vida Plena’s programme, but perhaps more relevant to how it’s communicated:
Given this is the first bullet under “helping a life flourish” I thought this might benefit from some clarification, because the vast majority of the value of this programme is likely not from suicide prevention, given low absolute rates of suicide.
From the same source: “at two years, the cumulative hazard of suicide death ranged from 0.12% in young adults to 0.18% in older adults.” Under unreasonably optimistic assumptions,[1] Vida Plena would prevent 1 suicide every 500 participants / prevent a suicide for $116,500, which is something between 21x to 39x less cost effective than GiveWell top charities.[2] More reasonable assumptions would drop this upper bound to 1 suicide prevented every ~1200 participants, or ~$272,000 per suicide prevented / ~50-90x less effective than GW top charities.[3]
Given you hope to reach 2,000 people by the end of 2025 for $50,000, this suggests a reasonable upper bound is something like 2 additional suicides prevented.[4]
This isn’t a claim that the cost-effectiveness claims are necessarily incorrect, even with minimal suicide prevention. A quick sense check RE: $462/DALY and 0.22 DALYs per participant would imply that Vida Plena would need to more than halve their cost per participant (from $233 down to $101), and then achieve results comparable to “~100% of people with severe / moderate mild depression conclude the programme going down one level of severity or something like ~5 points on the PHQ9 score (severe --> moderate; moderate --> mild; mild --> no depression).”[5] This is well within your listed results—though as you note in your annual report these have some fairly significant sources of bias and (IMO) probably should not be taken at face value.
Some other comments:
The NBER paper quoted in “g-IPT has also demonstrated long-term positive effects” looked at the “Healthy Activity Programme” (HAP)[6] and the “Thinking Healthy Programme Peer-Delivered” (THPP).[7] Neither of these are g-IPT programmes.
The minimal and unsustained results from the Baird RCT may be worth incorporating in an updated analysis, given the predictive CEA is from 2022[8]
From the predictive CEA: “Vida Plena’s overall effect for a household is 7.18*0.75*0.83 = 4.49 (95% CI: 0.77, 31.04) WELLBYs per person treated”. HLI recently decreased their estimate for StrongMinds treatment effects by 80% from 10.49 to 2.15 WELLBYs per treatment (also including household spillovers, and estimated StrongMinds to be “3.7x (previously 8x) as cost-effective as GiveDirectly”.
The cost-effectiveness of GiveDirectly has gone up by 3-4x (GW blog, GD blog), though this was recent news and does not necessarily imply that WELLBYs will also go up by 3-4x (most of this increase is attributable to increased consumption) - but should constitute a discount at least.
Even if 100% (rather than 24%) of individuals were in the high risk group (i.e. suicidal ideation nearly every day), and even if you dropped 100% of individuals risk of suicide from 0.2% to zero (rather than reducing it by 3-11x or to baseline), and if this effect persisted forever rather than just the initial 30 days
233 * 500 / 3000 = 38.83
233 * 500 / 5500 = 21.18 (assuming 1 prevented suicide = 1 life saved)
If 24% of your participants were high risk (7x risk, at 0.18%), and the other 76% of them were half of that (3.5x risk, at 0.09%), and you successfully reduced 100% of participants to baseline (0.026%), you would prevent 1 suicide every 1169 participants, which comes to ~$272,000 per life saved, or ~50-90x less cost effective than GW top charities.
(0.18-0.026) * 0.24 + (0.09-0.026) * 0.76 = 0.0856
100 / 0.0856 = 1168.2
1168.2 * 233 = 272190.6
272190.6 / 3000 = 90.73
272190.6 / 5500 = 49.4892
It’s also worth noting these are cumulative hazards at 2 years rather than 30, and the hazard ratios at 365 days are approximately halved compared to 30 days (1.7- to 5.7 instead of 3.3-10.8), so these figures are plausibly a few factors optimistic still.
Severe --> moderate depression is about 0.262 DALYs averted, moderate --> mild depression is about 0.251 DALYs averted, and mild --> no depression is about 0.145 DALYs averted.
HAP is described as “a psychological treatment based on behavioral activation...consist[ing] of 6 to 8 weekly sessions of 30 to 40 minutes each, delivered individually at participants’ homes or at the local PHC.”
THPP is a simplified version of a psychological intervention (THP) for treating perinatal depression that has been found to be effective in similar settings and is recommended by the WHO (Rahman et al., 2008, 2013; WHO, 2015; Baranov et al., 2020). While the original THP trials employed a full-fledged cognitive behavioral therapy (CBT) intervention, THPP was a simpler intervention focused on behavioral activation, as in the HAP trial described above. THPP was designed to be delivered by peer counselors, instead of community health workers as in previous trials.
[taken from here, emphasis added]:
-Our findings add to this evidence base by showing 12-month modest improvements of 20%-30% in rates of minimal depression for adolescents assigned to IPT-G, with these effects completely dissipating by the 24-month follow-up. We similarly find small short-term impacts on school enrollment, delayed marriage, desired fertility and time preferences, but fail to conclude that these effects persist two years after therapy.
-Given impact estimates of a reduction in the prevalence of mild depression of 0.054 pp for a period of one year, it implies that the cost of the program per case of depression averted was nearly USD 916, or 2,670 in 2019 PPP terms.
-This implies that ultimately the program cost USD PPP (2019) 18,413 per DALY averted. (almost 8x Uganda’s GDP per capita)
Hi Bruce, great to reconnect here! Thank you for all your thoughtful comments. I really appreciate your perspectives. Here are my responses to your points.
Yes, we are primarily a “life-improving” intervention rather than a life-saving one. While we do track suicidal ideation and believe we may have prevented several suicides, proving this counterfactually is challenging. So, when calculating our impact, we consider lives saved as a “bonus” rather than our primary measure of impact. As such, we don’t account for that in our cost-effectiveness estimates.
We are on track to improve our cost per person (but still not there). The $233 cost per person reported for 2023 included our total spending for the year, which involved significant consulting fees to Columbia University for their supervision in our first year. Excluding those fees, which are not recurring, our actual cost per person was $152 in 2023. We expect 2024 to be in that range.
Regarding your other notes:
The NBER paper was an incorrect reference—thank you for catching that. The link has been updated.
We don’t believe the Baird RCT is especially relevant to our work because 1) it focuses on adolescents, 2) it was delivered by non-specialist, age-peer providers (i.e., other adolescents), and 3) the “12-month follow-up surveys were conducted at the height of the pandemic.” A hard moment for everyone.
We also saw that update from GiveWell, although they noted, “we will be using our historic benchmark until we have thought it through.” So, we think it’s fair to continue using the historic benchmark until they switch.
So appreciate the thoughtful feedback, it helps us to refine our thinking and keep improving!
Thanks for the response, and likewise—hope you’ve been well! (Sorry I wasn’t sure if it was you or someone else on the account).
I agree that it is pretty reasonable to stick with the same benchmark, but I think this means it should be communicated accordingly, as VP are sometimes referring to a benchmark and other times referring to the GD programme, while GW are sticking to the same benchmark for their cost-effectiveness analyses, but updating their estimates of GD programmes.[1]
E.g. the predictive CEA (pg 7) referenced says:
“This means that a $1000 donation to Vida Plena would produce 58 WELLBYs, which is 8 times more cost-effective than GiveDirectly (a charity that excels in delivering cash transfers—simply giving people money—and a gold standard in effective altruism)”[2]
I think people would reasonably misinterpret this to mean you are referring to the GD programme, rather than the GW benchmark.[3] Again I know this is a v recent update and so hadn’t expected it to be already updated! But just flagging this as a potential source of confusion in the future.
Separately, I just thought I’d register interest in a more up-to-date predictive CEA that comes before your planned 2026 analysis, in part because there’s decent reason to do so (though I’m not making the stronger claim that this is more important than other things on your plate!), 2026 is a while away, and because it’s plausibly decision relevant for potential donors if they’re not sure the extent to which HLI updates might be applicable to VP.
“Thus, we will be using our historic benchmark until we have thought it through. For now, you can think of our benchmark as “GiveWell’s pre-2024 estimate of the impacts of cash transfers in Kenya,” with GiveDirectly’s current programs in various countries coming in at 3 to 4 times as cost-effective as that benchmark.”
The summary table on the same page also just says “GiveDirectly”.
To VP’s credit, I think “eight times more cost-effective than the benchmark of direct cash transfers” in this post would likely be interpreted correctly in a high context setting (but I also think reasonably might not be, and so may still be worth clarifying).
I’m not sure about this; the HLI’s analysis of GiveDirectly only looks at direct individual effects and household spillovers. Whereas GiveWell’s update seemingly only found additional effects in terms of non-household spillovers, mortality, and consumption (based on a 5 minute check, so I might be wrong here).
I think it’s reasonable to argue that depression prevention would also have effects on mortality, consumption (via productivity increases—my guesses here peg this quite high, especially in LMICs and UMICs), and non-household spillovers (via increased income being reinvested into communities, using the same mechanism as GD). Unless there’s reason to believe that the non-accounted-for impacts on WELLBYs systematically favour GiveDirectly I’d be cautious about applying a discount—but curious for your take on that :)
I might be misunderstanding you, but are you saying that after reading the GD updates, we should update on VP equally to GD / that we should expect the relative cost-effectiveness ratio between the two to remain the same?
Hmm, no. I wouldn’t want Vida Plena to update without evidence that they have those secondary effects.
But I think it would also be misleading to compare direct effects + household spillovers (in the case of Vida Plena) to direct effects + household spillovers + community spillovers + mortality reduction + consumption increases (GiveDirectly), unless you had good reason to believe that Vida Plena’s secondary effects are much worse than GiveDirectly’s. So I suppose I would be wary of saying that GiveDirectly now have 3–4x the WELLBY impact relative to Vida Plena—or even to say that GiveDirectly have any more WELLBY impact relative to Vida Plena—without having a good sense of how Vida Plena performs on those secondary outcomes. (But I feel like maybe I’m misunderstanding what you meant by applying a discount?)
Ah right—yeah I’m not making either of these claims, I’m just saying that if the previous claim (from VP’s predictive CEA) was that: “Vida Plena...is 8 times more cost-effective than GiveDirectly”, and GD has since been updated to 3-4x more cost-effective than it was compared to the time the predictive CEA was published, we should discount the 8x claim downwards somewhat (but not necessarily by 3-4x).
thoughtful comment! just want to throw in that suicide should not be considered in isolation imo. while every avoidable death is horrible ofc, I do think that suicide has particularly bad knock-on effects.