One place where this gets tricky is when their evaluators work in the
same area but have different funding bars. This is most apparent in
global health and development, where GiveWell and Founders Pledge overlap.
GiveWell is willing to recommend funding opportunities if they are 5x-8x [EDIT: 10x]
better than cash transfers to
very poor people while Founders Pledge uses a bar of at
least as good as cash transfers.
Reading GWWC’s website it wasn’t clear to me what they did. For
example they badge the salt-iodization charity IGN
as “top rated” but not the cash transfer charity GiveDirectly
or the deworming charity SCI,
even though Founders Pledge recommends all three.
If you look at the Founders Pledge recommendations for
GiveDirectlyand
SCI they start with a note that “This is a Founders Pledge summary
and interpretation of original research published by GiveWell.” So I
initially thought that what was happening was that they would mark the
charity as “top rated” if Founders Pledge recommends it based on their
own evaluation, but not if Founders Pledge is just relying on
GiveWell’s evaluation and GiveWell doesn’t recommend them.
This seemed bad: even if GiveWell, Founders Pledge, and GWWC all agreed
a charity was ~4x more effective than cash transfers, would GWWC
rate it below one on which GiveWell had no opinion and Founders Pledge
thought was only slightly more cost-effective than cash transfers?
I wrote to GWWC, and there’s actually a good system behind these
badges. They don’t combine the ratings naively, but have their own
intermediate threshold at ~3x better than cash transfers. They’re
able to do this because while Founders Pledge doesn’t publish
efficacy estimates for charities they do have them internally, and
they coordinate with GWWC. This explains why they don’t mark
GiveDirectly as “top rated”: it’s part of the baseline by which the
other charities are measured. And it also explains why IGN does get
the badge: Founders Pledge thinks it’s at least 3x better than cash
transfers.
What it doesn’t explain, though, is why SCI isn’t listed. In November
2021 GiveWell estimated (sheet,
via)
that donations to SCI were about 13x as effective as cash transfers.
The
reason GiveWell stopped recommending SCI as a top charity was not
that this estimate changed, but that GiveWell changed their
criteria:
We have always supported deworming because we estimate it to be highly
cost-effective—deworming programs are very inexpensive, and we think
there’s a small chance they may lead to large income gains later in
life. But we’re more uncertain about deworming than we are about our
current top charities, so deworming doesn’t fulfill the second
criterion on our list—a high likelihood of substantial
impact. However, we expect to continue supporting deworming through
grants from our All Grants Fund, as there continue to be funding
opportunities in deworming that exceed our cost-effectiveness
threshold, and we encourage donors who have supported individual
deworming programs in the past to keep doing so
In their response to my questions GWWC pointed me to their note on
their page
for SCI where they said “We intend to reevaluate the status of all
of the charities listed on our platform that are working on deworming
programmes after one of our trusted evaluators, Founders Pledge,
publishes its review of deworming’s cost-effectiveness.” They also
told me that until Founders Pledge finishes their re-evaluation of
deworming they don’t want to list them as “top rated”. Unless they
have some reason to think that GiveWell’s estimate of SCI’s cost
effectiveness has fallen by a factor of four in the last year,
however, I don’t think the decision to withhold this designation makes
sense. GiveWell dropping them as a top charity was a decision about
how to handle risk and uncertainty, and GWWC accepts Founders Pledge
recommendations which both (a) include some reviews much older than
2021 and (b) are more accepting of risk than GiveWell was even before
the August
2022 changes.
Aside from the questions about how SCI and other deworming charities
should be badged, however, my main takeaway here is that I’d like to
see more public details from GWWC about this process in their inclusion
criteria. Specifically, I’d like to see them describe their ~3x
cash threshold and use of Founders Pledge internal estimates. I’d
also like to see Founders Pledge make their estimates public, or at
least publish which of their recommendations meet the 3x bar. The
idea driving this is that, as I wrote earlier this week, I think the
“recommending specific charities” portion of GWWC’s work should depend
only on public information.
Disclosure: my wife used to be
President of GWWC and is on the GiveWell board, but I haven’t run this post by her and I don’t know
what she thinks here. I sent a draft of this post to GWWC, Founders
Pledge, and GiveWell; thanks to Sjir at GWWC for his substantial
feedback, and Matt at Founders Pledge for his quick responses.
GWWC’s Handling of Conflicting Funding Bars
When recommending charities, Giving What We Can (GWWC) mirrors the recommendations of their trusted evaluators. For example, they list the Against Malaria Foundation because GiveWell considers it a top charity and the Humane League because Animal Charity Evaluators does. This approach makes sense: consolidating the reviews of multiple evaluators for the GWWC membership and the broader EA community.
One place where this gets tricky is when their evaluators work in the same area but have different funding bars. This is most apparent in global health and development, where GiveWell and Founders Pledge overlap. GiveWell is willing to recommend funding opportunities if they are
5x-8x[EDIT: 10x] better than cash transfers to very poor people while Founders Pledge uses a bar of at least as good as cash transfers.Reading GWWC’s website it wasn’t clear to me what they did. For example they badge the salt-iodization charity IGN as “top rated” but not the cash transfer charity GiveDirectly or the deworming charity SCI, even though Founders Pledge recommends all three.
If you look at the Founders Pledge recommendations for GiveDirectly and SCI they start with a note that “This is a Founders Pledge summary and interpretation of original research published by GiveWell.” So I initially thought that what was happening was that they would mark the charity as “top rated” if Founders Pledge recommends it based on their own evaluation, but not if Founders Pledge is just relying on GiveWell’s evaluation and GiveWell doesn’t recommend them.
This seemed bad: even if GiveWell, Founders Pledge, and GWWC all agreed a charity was ~4x more effective than cash transfers, would GWWC rate it below one on which GiveWell had no opinion and Founders Pledge thought was only slightly more cost-effective than cash transfers?
I wrote to GWWC, and there’s actually a good system behind these badges. They don’t combine the ratings naively, but have their own intermediate threshold at ~3x better than cash transfers. They’re able to do this because while Founders Pledge doesn’t publish efficacy estimates for charities they do have them internally, and they coordinate with GWWC. This explains why they don’t mark GiveDirectly as “top rated”: it’s part of the baseline by which the other charities are measured. And it also explains why IGN does get the badge: Founders Pledge thinks it’s at least 3x better than cash transfers.
What it doesn’t explain, though, is why SCI isn’t listed. In November 2021 GiveWell estimated (sheet, via) that donations to SCI were about 13x as effective as cash transfers. The reason GiveWell stopped recommending SCI as a top charity was not that this estimate changed, but that GiveWell changed their criteria:
In their response to my questions GWWC pointed me to their note on their page for SCI where they said “We intend to reevaluate the status of all of the charities listed on our platform that are working on deworming programmes after one of our trusted evaluators, Founders Pledge, publishes its review of deworming’s cost-effectiveness.” They also told me that until Founders Pledge finishes their re-evaluation of deworming they don’t want to list them as “top rated”. Unless they have some reason to think that GiveWell’s estimate of SCI’s cost effectiveness has fallen by a factor of four in the last year, however, I don’t think the decision to withhold this designation makes sense. GiveWell dropping them as a top charity was a decision about how to handle risk and uncertainty, and GWWC accepts Founders Pledge recommendations which both (a) include some reviews much older than 2021 and (b) are more accepting of risk than GiveWell was even before the August 2022 changes.
Aside from the questions about how SCI and other deworming charities should be badged, however, my main takeaway here is that I’d like to see more public details from GWWC about this process in their inclusion criteria. Specifically, I’d like to see them describe their ~3x cash threshold and use of Founders Pledge internal estimates. I’d also like to see Founders Pledge make their estimates public, or at least publish which of their recommendations meet the 3x bar. The idea driving this is that, as I wrote earlier this week, I think the “recommending specific charities” portion of GWWC’s work should depend only on public information.
Disclosure: my wife used to be President of GWWC and is on the GiveWell board, but I haven’t run this post by her and I don’t know what she thinks here. I sent a draft of this post to GWWC, Founders Pledge, and GiveWell; thanks to Sjir at GWWC for his substantial feedback, and Matt at Founders Pledge for his quick responses.