We’d like to thank Aidan and the Giving What We Can (GWWC) team for a careful, constructive, and genuinely collaborative evaluation. They were open with their reasoning, generous with their time, and responsive throughout.
We naturally appreciate the positive things they highlighted: the transparency and rigour of our work, the promise of our recommended charities for donors with strong life-improving preferences, and our contribution to foundational wellbeing research and to the broader ecosystem.
GWWC described the decision to “not (yet)” rely on our recommendations as “unusually difficult” and “a close call,” with reasonable disagreement among their evaluators. We take this as a sign that our work is being taken seriously; we aim to make it an unusually easy call for GWWC next time.
1) Why we’re not surprised or discouraged by the evaluation results
For the purposes of GWWC’s assessment, the charity evaluator they are considering relying on needs to be reasonably competitive with the existing field leader in their cause area. In our case, as we fall under the “global health and wellbeing” category, this means HLI was directly compared to GiveWell on the cost-effectiveness of its charity recommendations and on process reliability. This is a high hurdle to pass. For context, sometimes people are surprised to hear that:
Our budget is about 5% of GiveWell’s (under $1m vs over $20m)
Our research staff is ~3 FTE compared to around 50.
We’ve only been making charity recommendations since 2022, whereas GiveWell has been doing so since 2008.
In addition, HLI has a dual mandate. While we place a high value on charity recommendations, we also work to pioneer wellbeing impact methodology and conduct a range of applied and theoretical work outside direct charity evaluation.
Given this, we aren’t necessarily surprised that GWWC describes our processes as immature, given the comparison. In fact, we did better than we expected – and for this reason, we feel energised and encouraged to do and become better.
2) How we’re planning to improve our research
Many improvements GWWC proposed were already on our roadmap; their evaluation is a welcome push to do more on these issues. Some examples of things we plan to work on include:
Applying consistent assumptions across evaluations (e.g., revisiting AMF using Taimaka-style assumptions, forming a view on time-discounting, and reviewing our income to wellbeing conversion rate)
Developing a structured “charity quality” rubric that captures things like implementation strength, monitoring and evaluation practices, organisational responsiveness.
Striving for even greater transparency, including publishing code, all relevant spreadsheets and articulating a clearer “bar” for recommendations
3) The challenges to understanding AMF’s life-improving effects
A major part of GWWC’s analysis involved applying the assumptions from our 2024 Taimaka evaluation to our 2022 AMF model. Under those revised assumptions, the life-improving benefits of AMF increase substantially, resulting in a (highly uncertain) cost-effectiveness estimate for AMF in the same ballpark as HLI’s top recommendations.
We appreciated GWWC drawing attention to inconsistencies across our analyses. We agree these should be resolved and plan to revisit AMF more systematically. However, we think it is too early to conclude that AMF has large life-improving benefits. Three evidentiary challenges make us cautious:
Limited and low-relevance evidence. There is no published direct subjective wellbeing evidence on the long-run effects of early-life malaria exposure. Instead, both HLI and GiveWell rely on just two income-based studies from the mid-20th century, DDT-based malaria eradication campaigns in the USA and India (Bleakley 2010, Cutler et al., 2010). These show small, long-lasting income increases of roughly ~2% per year of malaria averted before adjustments. These are quite different contexts from modern bednet distribution and malaria prevention in sub-Saharan Africa.
Mechanistic mismatch. These historical studies require us to convert small, diffuse, long-term income changes into WELLBYs. But this also poses issues given that cash transfers, our best evidence for income → wellbeing, involve large, relatively concentrated,highly salient income changes (10–100% increases). In contrast, malaria-reduction income gains are small, spread out, and unlikely to be psychologically salient. Because of this difference, it’s unclear to us whether a 1% income increase for 40 years should produce the same wellbeing benefit as a 40% increase for one year.
The case of deworming. In our analysis of deworming, we found that small long-run income effects do not correspond to detectable long-run wellbeing effects. This tempers our expectations against assuming that very small, long-run income changes reliably translate into meaningful improvements in wellbeing in other cases.
Given these issues, we think comparisons between AMF and our top charities—which treat depression via therapy and have large amounts of relevant, direct wellbeing data—remain “apples to oranges” until stronger, more relevant data become available. That said, we plan to revisit our AMF analysis, harmonise assumptions, re-examine the income-to-WELLBY relationship, and explore opportunities for generating direct wellbeing evidence on malaria. But we think it’s quite plausible that, in the long run, the case for the life-improving effects of malaria reduction will remain speculative until and unless more direct wellbeing evidence becomes available.
4) Closing thoughts
Overall, we appreciate the serious thought and care that went into GWWC’s evaluation. We agree with many of their points, have constructive disagreements on others, and are grateful for the clear, actionable feedback. This evaluation is an important milestone in our growth, and we look forward to strengthening our research, improving our processes, and continuing to build a world where evidence-based decision-making takes happiness seriously.
Thank you for the very thoughtful comment! As mentioned in the report, we’ve really appreciated the open and thoughtful way you have engaged with us throughout this evaluation. We look forward to seeing you implement your planned improvements and to re-evaluating HLI’s work in the future!
“Because of this difference, it’s unclear to us whether a 1% income increase for 40 years should produce the same wellbeing benefit as a 40% increase for one year.” for sure it’s unclear, but I would on a 1 percent income increase over 40 years making a bigger difference to well-being than 40 percent over 1 year. very low confidence though it’s just a hunch.
the problem is we’re really unlikely to ever be able to determine that because we won’t be able to statistically pick out any less than say a 5 percent yearly increase with most study sample sizes…
We’d like to thank Aidan and the Giving What We Can (GWWC) team for a careful, constructive, and genuinely collaborative evaluation. They were open with their reasoning, generous with their time, and responsive throughout.
We naturally appreciate the positive things they highlighted: the transparency and rigour of our work, the promise of our recommended charities for donors with strong life-improving preferences, and our contribution to foundational wellbeing research and to the broader ecosystem.
GWWC described the decision to “not (yet)” rely on our recommendations as “unusually difficult” and “a close call,” with reasonable disagreement among their evaluators. We take this as a sign that our work is being taken seriously; we aim to make it an unusually easy call for GWWC next time.
1) Why we’re not surprised or discouraged by the evaluation results
For the purposes of GWWC’s assessment, the charity evaluator they are considering relying on needs to be reasonably competitive with the existing field leader in their cause area. In our case, as we fall under the “global health and wellbeing” category, this means HLI was directly compared to GiveWell on the cost-effectiveness of its charity recommendations and on process reliability. This is a high hurdle to pass. For context, sometimes people are surprised to hear that:
Our budget is about 5% of GiveWell’s (under $1m vs over $20m)
Our research staff is ~3 FTE compared to around 50.
We’ve only been making charity recommendations since 2022, whereas GiveWell has been doing so since 2008.
In addition, HLI has a dual mandate. While we place a high value on charity recommendations, we also work to pioneer wellbeing impact methodology and conduct a range of applied and theoretical work outside direct charity evaluation.
Given this, we aren’t necessarily surprised that GWWC describes our processes as immature, given the comparison. In fact, we did better than we expected – and for this reason, we feel energised and encouraged to do and become better.
2) How we’re planning to improve our research
Many improvements GWWC proposed were already on our roadmap; their evaluation is a welcome push to do more on these issues. Some examples of things we plan to work on include:
Applying consistent assumptions across evaluations (e.g., revisiting AMF using Taimaka-style assumptions, forming a view on time-discounting, and reviewing our income to wellbeing conversion rate)
Developing a structured “charity quality” rubric that captures things like implementation strength, monitoring and evaluation practices, organisational responsiveness.
Striving for even greater transparency, including publishing code, all relevant spreadsheets and articulating a clearer “bar” for recommendations
3) The challenges to understanding AMF’s life-improving effects
A major part of GWWC’s analysis involved applying the assumptions from our 2024 Taimaka evaluation to our 2022 AMF model. Under those revised assumptions, the life-improving benefits of AMF increase substantially, resulting in a (highly uncertain) cost-effectiveness estimate for AMF in the same ballpark as HLI’s top recommendations.
We appreciated GWWC drawing attention to inconsistencies across our analyses. We agree these should be resolved and plan to revisit AMF more systematically. However, we think it is too early to conclude that AMF has large life-improving benefits. Three evidentiary challenges make us cautious:
Limited and low-relevance evidence. There is no published direct subjective wellbeing evidence on the long-run effects of early-life malaria exposure. Instead, both HLI and GiveWell rely on just two income-based studies from the mid-20th century, DDT-based malaria eradication campaigns in the USA and India (Bleakley 2010, Cutler et al., 2010). These show small, long-lasting income increases of roughly ~2% per year of malaria averted before adjustments. These are quite different contexts from modern bednet distribution and malaria prevention in sub-Saharan Africa.
Mechanistic mismatch. These historical studies require us to convert small, diffuse, long-term income changes into WELLBYs. But this also poses issues given that cash transfers, our best evidence for income → wellbeing, involve large, relatively concentrated, highly salient income changes (10–100% increases). In contrast, malaria-reduction income gains are small, spread out, and unlikely to be psychologically salient. Because of this difference, it’s unclear to us whether a 1% income increase for 40 years should produce the same wellbeing benefit as a 40% increase for one year.
The case of deworming. In our analysis of deworming, we found that small long-run income effects do not correspond to detectable long-run wellbeing effects. This tempers our expectations against assuming that very small, long-run income changes reliably translate into meaningful improvements in wellbeing in other cases.
Given these issues, we think comparisons between AMF and our top charities—which treat depression via therapy and have large amounts of relevant, direct wellbeing data—remain “apples to oranges” until stronger, more relevant data become available. That said, we plan to revisit our AMF analysis, harmonise assumptions, re-examine the income-to-WELLBY relationship, and explore opportunities for generating direct wellbeing evidence on malaria. But we think it’s quite plausible that, in the long run, the case for the life-improving effects of malaria reduction will remain speculative until and unless more direct wellbeing evidence becomes available.
4) Closing thoughts
Overall, we appreciate the serious thought and care that went into GWWC’s evaluation. We agree with many of their points, have constructive disagreements on others, and are grateful for the clear, actionable feedback. This evaluation is an important milestone in our growth, and we look forward to strengthening our research, improving our processes, and continuing to build a world where evidence-based decision-making takes happiness seriously.
— The Happier Lives Institute
Thank you for the very thoughtful comment! As mentioned in the report, we’ve really appreciated the open and thoughtful way you have engaged with us throughout this evaluation. We look forward to seeing you implement your planned improvements and to re-evaluating HLI’s work in the future!
what a wonderful reply...
“Because of this difference, it’s unclear to us whether a 1% income increase for 40 years should produce the same wellbeing benefit as a 40% increase for one year.” for sure it’s unclear, but I would on a 1 percent income increase over 40 years making a bigger difference to well-being than 40 percent over 1 year. very low confidence though it’s just a hunch.
the problem is we’re really unlikely to ever be able to determine that because we won’t be able to statistically pick out any less than say a 5 percent yearly increase with most study sample sizes…