In December 2022, GiveWell announced the results of its “Change Our Mind Contest”. GiveWell awarded joint first place and a $20,000 prize to Noah Haber for “GiveWell’s Uncertainty Problem”. According to GiveWell:
The author argues that without properly accounting for uncertainty, GiveWell is likely to allocate its portfolio of funding suboptimally, and proposes methods for addressing uncertainty.
Going forward, we plan to: Incorporate sensitivity analysis in published models. We’ve begun this process by incorporating basic sensitivity analysis on key parameters in recent grant and intervention report pages (e.g., zinc/ORS; vitamin A supplementation; MiracleFeet). We’re currently revamping our top charity CEAs to make them more legible, and we plan to incorporate sensitivity analysis and Monte Carlos into these before publishing.
[...]
In the three linked pages, GiveWell has conducted a so-called one-at-a-time sensitivity analysis, an approach which as GiveWell acknowledges is unable to quantify the overall uncertainty in cost-effectiveness.
After the contest winners were announced in December 2022, GiveWell has as far as I can tell made the following grants (source):
January 2023 to June 2024
January 2024 to June 2024
To top charities
$295 M
$89 M
To organisations other than top charities
$143 M
$31 M
I cannot find on GiveWell’s website any investigation where the full uncertainty (from all parameters) is modelled. Has GiveWell published such a thing? If not, do they still plan to?
[Question] Does GiveWell still plan to model the uncertainty of their cost-effectiveness estimates?
In December 2022, GiveWell announced the results of its “Change Our Mind Contest”. GiveWell awarded joint first place and a $20,000 prize to Noah Haber for “GiveWell’s Uncertainty Problem”. According to GiveWell:
One of the honorable mention awards was also a critique of GiveWell’s approach to uncertainty.
Both essays recommended using Monte Carlo simulation to properly model uncertainty in cost-effectiveness.
In December 2023, GiveWell published “How We Plan to Approach Uncertainty in Our Cost-Effectiveness Models”, in which they explained their next steps:
In the three linked pages, GiveWell has conducted a so-called one-at-a-time sensitivity analysis, an approach which as GiveWell acknowledges is unable to quantify the overall uncertainty in cost-effectiveness.
After the contest winners were announced in December 2022, GiveWell has as far as I can tell made the following grants (source):
I cannot find on GiveWell’s website any investigation where the full uncertainty (from all parameters) is modelled. Has GiveWell published such a thing? If not, do they still plan to?