Surely if you thought that EA AWF was a good evaluator or donation opportunity for donors, you would just let them manage the entirety of the fund? As then EA AWF would be able to distribute to THL if they actually thought THL was the most effective use of funds on the margin. And if not, even better, as they can give to more effective opportunities.
The short answer is ânoâ: we donât think we can currently justify the claim that giving to the AWF is better than giving to THLâs corporate campaigns, or vice versa. We did indeed conclude from our evaluation that the AWF can likely use marginal funds cost-effectively, but that isnât the same as deferring to them on all fronts (including because we also found significant room for improvement, as explained in the AWF report), nor does it imply the AWF is better at allocating extra capital than THL is.
Iâm also curious why you felt the need to recommend at least one competitive alternative to the AWF, when the AWF itself is a fairly diversified fund? Arguably, you marked ACE down for similar reasoning in your evaluation of their Movement Grants (that they were spreading their grants across many groups rather than focusing mostly on the most effective groups)
Our goal is to provide recommendations to help donors maximise their impact from the perspective of a variety of worldviews, and itâs in that light that we decided to also recommend THLâs corporate campaigns: consider that someone else could have made an (I think justifiable) comment that is entirely the opposite of what you are saying, i.e. that we should only recommend THL because there we actually have some independent evidence of their intervention working and being highly cost-effective, which is lacking for many if not most of the projects the AWF funds (given the early stage of the AW charity evaluation space).
We criticized ACE MG not for making grants to multiple groups but for doing so at the seeming expense of expected impact. As mentioned above, we donât think THLâs corporate campaigns are a worse donation opportunity than the AWF, and we think there may be donors who think itâs more cost-effective in expectation, for instance because they put less weight on the individual judgement of grantmakers (or on our judgement in evaluating AWF to be a good donation opportunity!) and think the publicly available evidence for THL is stronger.
Statements like this make me worry that this evaluation focused too much on the certainty of some positive impact, rather than maximising expected impact (i.e. measurability bias).
I think youâre right to worry about thisâI do as well! - as I would say there is some implicit measurability bias in our recommendations. Most notably, we ended up recommending THLâs corporate campaigns over other ACE recommendations not because we have strong evidence that they are a better donation opportunity than any other individual ACE recommendation, but because they are the only one where we think we have sufficient evidence to justify recommending them.
However, this is importantly different from us prioritising certainty of some positive impact over maximum expected impact: THLâs corporate campaigns is our best guess donation opportunity to maximise expected impact (alongside the AWF). If we thought we could have easily justified any one of ACEâs other recommendations was betterâor even just as goodâfrom that perspective, we would have recommended them, but we currently canât. And please note that âjustifyingâ here isnât about finding âcertainty of positive impactâ: we are looking for the expected value case (as we do for the AWF and our other recommendations as well).
As mentioned in the comment above, you would struggle to find many experienced animal advocates who would confidently recommend THL as the single best marginal giving opportunity. In reality, they would likely either advocate for a spread of groups using different approaches or just simply give to a fund (e.g. EA AWF or ACE).
This is a much stronger claim than we are making (THLâs corporate campaigns being the âsingle best marginal giving opportunityâ): we think itâs one of the two best donation opportunities we can, from the information we have available, recommend to a broad set of donors to maximise their expected impact. We are not claiming that nobody could do better (certainly by their individual values/âworldview!), and encourage donors to do their own (further) research if they have the time and expertise available. This is also why we host a broader selection of promising programs donors can look into and support on our donation platform.
THLâs corporate campaigns is our best guess donation opportunity to maximise expected impact (alongside the AWF). If we thought we could have easily justified any one of ACEâs other recommendations was betterâor even just as goodâfrom that perspective, we would have recommended them, but we currently canât. And please note that âjustifyingâ here isnât about finding âcertainty of positive impactâ: we are looking for the expected value case (as we do for the AWF and our other recommendations as well).
Based on your paragraph below from the ACE Report, Iâm inferring that you only looked at three (out of 11) ACE recommendations, which only included charities evaluated in 2023, rather than 2022? So by default, GFI, Sinergia, Fish Welfare Initiative, Kafessiz and DVF were all excluded from potentially being identified (which seems illogical, as there is no obvious reason to think that charities evaluated in 2022 would be less cost-effective).[1]
ACE helpfully â and on very short notice â provided us with private documentation to elaborate on the cases for three of its 2023 charity recommendations [emphasis mine]. Unfortunately â potentially in part because of time constraints ACE had â we still didnât find these cases to provide enough evidence on the marginal cost-effectiveness of the charities to justify relying on them for our recommendations.
Given you only looked at three of the ACE 2023 recommendations (and you didnât say which ones), Iâm wondering how you can make such a strong claim for all of ACEâs recommended charities?
If we thought we could have easily justified any one of ACEâs other recommendations [emphasis mine] was betterâor even just as goodâfrom that perspective, we would have recommended them, but we currently canât.
On a slightly unrelated point: For the referral from OP, I would be curious to hear if you asked them âWhat is the most cost-effective marginal giving opportunity for farmed animal welfareâ (to which they replied THLâs corporate campaigns) or something closer to âDo you think THL is a cost-effective giving opportunity on the margin?â
This is a much stronger claim than we are making (THLâs corporate campaigns being the âsingle best marginal giving opportunityâ): we think itâs one of the two best donation opportunities we can, from the information we have available, recommend to a broad set of donors to maximise their expected impact.
Fair enough! I should have said âOne of the top 2 marginal giving opportunitiesâ but I still think I stand by my point that many experienced animal advocates would disagree with this claim, and itâs not clear that your charity recommendation work has sufficient depth to challenge that (e.g. you didnât evaluate groups yourself), in which case itâs not clear why folks should defer to you over subject-matter experts (e.g. AWF, OP or ACE).
You might say there is weaker evidence of their cost-effectiveness as itâs been a year since they were evaluated but since you said you focused on the expected value case rather than certainty of positive impact, I assume this wasnât your issue.
So by default, GFI, Sinergia, Fish Welfare Initiative, Kafessiz and DVF were all excluded from potentially being identified (which seems illogical, as there is no obvious reason to think that charities evaluated in 2022 would be less cost-effective)
Yes they were, as were any other charities than the three charities we asked ACE to send us more information on (based on where they thought they could make the strongest case by our lights). Among those, we think ACE provided the strongest case for THLâs corporate campaigns, and with the additional referral from Open Phil + the existing public reports by FP and RP on corporate campaigns, we think this is enough to justify a recommendation. This is what I meant by there indeed being a measurability bias in our recommendation (which we think is a bullet worth biting here!): we ended up recommending THL in large part because there was sufficient evidence of cost-effectiveness readily and publicly available. We donât have the same evidence for any of these other charities, so they could in principle be as or even more cost-effective than THL (but also much less!), and without the evidence to support their case we donât (yet) feel justified recommending them. We donât have capacity to directly evaluate individual charities ourselves (including THL!), but continue to host many promising charities on our donation platform, so donors who have time to look into them further can choose to support them.
To put this differently, the choice for us wasnât between âevaluating all of ACEâs recommendationsâ and âevaluating only THL /â three charitiesâ (as we didnât have capacity to do any individual charity evaluations). The choice for us was between âonly recommending the AWFâ and ârecommending both the AWF and THLâs corporate campaignsâ because there happened to already be sufficiently strong evidence/âevaluations available for THLâs corporate campaigns. For reasons explained earlier, we stand by our decision to prefer the latter over the former, even though that means that many other promising charities donât have a chance to be recommended at this point (but note that this is the case in charity evaluation across cause areas!).
Given you only looked at three of the ACE 2023 recommendations (and you didnât say which ones), Iâm wondering how you can make such a strong claim for all of ACEâs recommended charities?
Could you clarify which âstrong claim for all of ACEâs recommended charitiesâ you are referring to? From the executive summary of our report on ACE:
We also expect the gain in impact from giving to any ACE-recommended charity over giving to a random animal welfare charity is much larger than any potential further gain from giving to the AWF or THLâs corporate campaigns over any (other) ACE-recommended charity, and note that we havenât evaluated ACEâs recommended charities individually, but only ACEâs evaluation process.
On a slightly unrelated point: For the referral from OP, I would be curious to hear if you asked them âWhat is the most cost-effective marginal giving opportunity for farmed animal welfareâ (to which they replied THLâs corporate campaigns) or something closer to âDo you think THL is a cost-effective giving opportunity on the margin?â
The latter, because a referral by OP on its own wouldnât have been sufficient for us to make a recommendation (as we havenât evaluated OP): for recommending THLâs corporate campaigns, we really relied on these four separate pieces of evidence being available.
I should have said âOne of the top 2 marginal giving opportunitiesâ but I still think I stand by my point that many experienced animal advocates would disagree with this claim, and itâs not clear that your charity recommendation work has sufficient depth to challenge that (e.g. you didnât evaluate groups yourself), in which case itâs not clear why folks should defer to you over subject-matter experts (e.g. AWF, OP or ACE).
Weâre not even claiming it is one of the top 2 marginal giving opportunities, just that it is the best recommendation we can make to donors based on the information available to us from evaluators. If you could point us to any alternative well-justified recommendations/âevaluators for us to evaluate, weâd be all ears.
And we donât claim people should defer to us directly on charity evaluations (again, we donât currently do these ourselves!). Ultimately, our recommendations (including THL!) are based on the recommendations of the subject-matter experts you reference. The purpose of our evaluations and reports is to help donors make better decisions based on the recommendations and information these experts provide.
Thanks for your comments and questions, James.
The short answer is ânoâ: we donât think we can currently justify the claim that giving to the AWF is better than giving to THLâs corporate campaigns, or vice versa. We did indeed conclude from our evaluation that the AWF can likely use marginal funds cost-effectively, but that isnât the same as deferring to them on all fronts (including because we also found significant room for improvement, as explained in the AWF report), nor does it imply the AWF is better at allocating extra capital than THL is.
Our goal is to provide recommendations to help donors maximise their impact from the perspective of a variety of worldviews, and itâs in that light that we decided to also recommend THLâs corporate campaigns: consider that someone else could have made an (I think justifiable) comment that is entirely the opposite of what you are saying, i.e. that we should only recommend THL because there we actually have some independent evidence of their intervention working and being highly cost-effective, which is lacking for many if not most of the projects the AWF funds (given the early stage of the AW charity evaluation space).
We criticized ACE MG not for making grants to multiple groups but for doing so at the seeming expense of expected impact. As mentioned above, we donât think THLâs corporate campaigns are a worse donation opportunity than the AWF, and we think there may be donors who think itâs more cost-effective in expectation, for instance because they put less weight on the individual judgement of grantmakers (or on our judgement in evaluating AWF to be a good donation opportunity!) and think the publicly available evidence for THL is stronger.
I think youâre right to worry about thisâI do as well! - as I would say there is some implicit measurability bias in our recommendations. Most notably, we ended up recommending THLâs corporate campaigns over other ACE recommendations not because we have strong evidence that they are a better donation opportunity than any other individual ACE recommendation, but because they are the only one where we think we have sufficient evidence to justify recommending them.
However, this is importantly different from us prioritising certainty of some positive impact over maximum expected impact: THLâs corporate campaigns is our best guess donation opportunity to maximise expected impact (alongside the AWF). If we thought we could have easily justified any one of ACEâs other recommendations was betterâor even just as goodâfrom that perspective, we would have recommended them, but we currently canât. And please note that âjustifyingâ here isnât about finding âcertainty of positive impactâ: we are looking for the expected value case (as we do for the AWF and our other recommendations as well).
This is a much stronger claim than we are making (THLâs corporate campaigns being the âsingle best marginal giving opportunityâ): we think itâs one of the two best donation opportunities we can, from the information we have available, recommend to a broad set of donors to maximise their expected impact. We are not claiming that nobody could do better (certainly by their individual values/âworldview!), and encourage donors to do their own (further) research if they have the time and expertise available. This is also why we host a broader selection of promising programs donors can look into and support on our donation platform.
Based on your paragraph below from the ACE Report, Iâm inferring that you only looked at three (out of 11) ACE recommendations, which only included charities evaluated in 2023, rather than 2022? So by default, GFI, Sinergia, Fish Welfare Initiative, Kafessiz and DVF were all excluded from potentially being identified (which seems illogical, as there is no obvious reason to think that charities evaluated in 2022 would be less cost-effective).[1]
Given you only looked at three of the ACE 2023 recommendations (and you didnât say which ones), Iâm wondering how you can make such a strong claim for all of ACEâs recommended charities?
On a slightly unrelated point: For the referral from OP, I would be curious to hear if you asked them âWhat is the most cost-effective marginal giving opportunity for farmed animal welfareâ (to which they replied THLâs corporate campaigns) or something closer to âDo you think THL is a cost-effective giving opportunity on the margin?â
Fair enough! I should have said âOne of the top 2 marginal giving opportunitiesâ but I still think I stand by my point that many experienced animal advocates would disagree with this claim, and itâs not clear that your charity recommendation work has sufficient depth to challenge that (e.g. you didnât evaluate groups yourself), in which case itâs not clear why folks should defer to you over subject-matter experts (e.g. AWF, OP or ACE).
You might say there is weaker evidence of their cost-effectiveness as itâs been a year since they were evaluated but since you said you focused on the expected value case rather than certainty of positive impact, I assume this wasnât your issue.
Yes they were, as were any other charities than the three charities we asked ACE to send us more information on (based on where they thought they could make the strongest case by our lights). Among those, we think ACE provided the strongest case for THLâs corporate campaigns, and with the additional referral from Open Phil + the existing public reports by FP and RP on corporate campaigns, we think this is enough to justify a recommendation. This is what I meant by there indeed being a measurability bias in our recommendation (which we think is a bullet worth biting here!): we ended up recommending THL in large part because there was sufficient evidence of cost-effectiveness readily and publicly available. We donât have the same evidence for any of these other charities, so they could in principle be as or even more cost-effective than THL (but also much less!), and without the evidence to support their case we donât (yet) feel justified recommending them. We donât have capacity to directly evaluate individual charities ourselves (including THL!), but continue to host many promising charities on our donation platform, so donors who have time to look into them further can choose to support them.
To put this differently, the choice for us wasnât between âevaluating all of ACEâs recommendationsâ and âevaluating only THL /â three charitiesâ (as we didnât have capacity to do any individual charity evaluations). The choice for us was between âonly recommending the AWFâ and ârecommending both the AWF and THLâs corporate campaignsâ because there happened to already be sufficiently strong evidence/âevaluations available for THLâs corporate campaigns. For reasons explained earlier, we stand by our decision to prefer the latter over the former, even though that means that many other promising charities donât have a chance to be recommended at this point (but note that this is the case in charity evaluation across cause areas!).
Could you clarify which âstrong claim for all of ACEâs recommended charitiesâ you are referring to? From the executive summary of our report on ACE:
We also expect the gain in impact from giving to any ACE-recommended charity over giving to a random animal welfare charity is much larger than any potential further gain from giving to the AWF or THLâs corporate campaigns over any (other) ACE-recommended charity, and note that we havenât evaluated ACEâs recommended charities individually, but only ACEâs evaluation process.
The latter, because a referral by OP on its own wouldnât have been sufficient for us to make a recommendation (as we havenât evaluated OP): for recommending THLâs corporate campaigns, we really relied on these four separate pieces of evidence being available.
Weâre not even claiming it is one of the top 2 marginal giving opportunities, just that it is the best recommendation we can make to donors based on the information available to us from evaluators. If you could point us to any alternative well-justified recommendations/âevaluators for us to evaluate, weâd be all ears.
And we donât claim people should defer to us directly on charity evaluations (again, we donât currently do these ourselves!). Ultimately, our recommendations (including THL!) are based on the recommendations of the subject-matter experts you reference. The purpose of our evaluations and reports is to help donors make better decisions based on the recommendations and information these experts provide.