Hi Vince. Do you have any plans to estimate the marginal cost-effectiveness of charities? I agree withGiving What We Can (GWWC) that this would be a major improvement to ACE’s evaluations.
We believe that ACE’s Charity Evaluation Program’s current approach does not sufficiently emphasise marginal cost-effectiveness as the most decision-relevant factor in evaluation decisions. For example, ACE primarily evaluates on the basis of organisations’ existing programs, rather than explicitly focusing their evaluation on those programs that are most likely to be funded on the margin. This is in direct contrast to ACE’s Movement Grants, which explicitly evaluates programs that ACE would be influencing funding to on the margin.
ACE has been putting more emphasis on selecting programs for cost-effectiveness (and theory of change) analysis based on marginal plans and how likely we are fund them to, so we are estimating marginal cost-effectiveness.
That being said, I don’t think our methods in 2025 were far off from where they are now. We continue to think that current cost-effectiveness can be the best proxy for marginal future cost-effectiveness in many cases, and that focusing narrowly on where we predict our funding will be going would a mistake.
For example, The Humane League’s future plans are a prioritized list of 24 ways that they would expand with additional funding. Not only is it not feasible for us to construct so many forward-looking CEAs, but the resulting estimates would also carry higher uncertainty. Additionally, many plans are expansions of existing programs or are general operational investments, which depend on current cost-effectiveness.
Funding is also fungible; dollars we direct to THL won’t cleanly fund items 1-3 versus 4-6 on the list. Actual deployment depends on what other funders do, internal decisions, and new opportunities that emerge. Trying to predict which specific items our marginal dollar pays for is false precision, and we’ve seen charities greatly change their marginal plans within the two-year span of their recommendation.
Instead, we use a mix of current cost-effectiveness, our understanding of potential diminishing returns, qualitative judgments about effectiveness and strategic prioritization, and speculative, forward-looking CEAs to make our decisions. Current cost-effectiveness captures an organization’s demonstrated ability to convert dollars into outcomes and that is often what is most likely to carry over to the next dollar. Large shifts in marginal productivity is likely low, especially for established orgs running stable programs with strong prioritization, so current cost-effectiveness is usually within a reasonable range of marginal future cost-effectiveness.
For example, The Humane League’s (THL’s) future plans are a prioritized list of 24 ways that they would expand with additional funding. Not only is it not feasible for us to construct so many forward-looking CEAs
ACE’s Recommended Charity Fund granted 399 k$ (= (85.4 + 314)*10^3) to THL in 2024, and the 3 highest priority future plans from THL have a cost of 455 k$. So, assuming that the amount granted to THL covers most of the additional funding it received as a result of your recommendation, you would only need to model the cost-effectiveness of 3 programs, or maybe just 2, as the 2 highest priority projects cost 78.7 % (= 358⁄455) as much as the 3 highest priority projects? In your last cost-effectiveness analysis of THL, you looked into 4 programs.
Funding is also fungible; dollars we direct to THL won’t cleanly fund items 1-3 versus 4-6 on the list. Actual deployment depends on what other funders do, internal decisions, and new opportunities that emerge.
Makes sense. At the same time, I still think assessing the cost-effectiveness of marginal projects is valuable. Imagine THL ends up using ACE’s funds to support projects 4 to 6 instead of projects 1 to 3, which had the highest priority at the time of your evaluation, and were the only ones covered by your cost-effectiveness analyses. One should expect the cost-effectiveness of projects 4 to 6 to be closer to what ACE estimated for projects 1 to 3 than to the cost-effectiveness of THL as a whole? I think so. Mostly because projects 4 to 6 were much closer to being marginal at the time of your evaluation than random funds from THL. And partly because THL prioritising projects 4 to 6 would be an update towards projects 1 to 3 having a lower cost-effectiveness than you estimated, and projects 4 to 6 having a higher cost-effectiveness than you estimated.
Trying to predict which specific items our marginal dollar pays for is false precision, and we’ve seen charities greatly change their marginal plans within the two-year span of their recommendation.
I agree it would not make sense to predict changes in what charities will prioritise. I would just assess the cost-effectiveness of the highest priority projects covering the expected increase in funding resulting from your recommendation.
Current cost-effectiveness captures an organization’s demonstrated ability to convert dollars into outcomes and that is often what is most likely to carry over to the next dollar.
I suspect the marginal cost-effectiveness of large organisations may be significantly lower than the cost-effectiveness of the whole spending. My intuition having worked for the last 9 months at Anima International is that our marginal cost-effectiveness is 10 % to 50 % of the cost-effectiveness of our whole spending. Relatedly, @abrahamroweguessed 16 months ago corporate campaigns for chicken welfare not funded by Coefficient Giving (CG), including ones run by organisations funded by CG, were as cost-effective as burning money, despite some of the ones funded by CG being beneficial. Here are some related reflections from Abraham. I was surprised by Abraham’s claim at the time, but I have meanwhile come closer to his position, although I think I am still significatly more optimistic than he was at the time.
What you describe is roughly how we decide each Recommended Charity Fund distribution, but unfortunately recommendation decisions have more complexity. Some examples of this complexity include ACE influencing significantly more funding than we distribute directly, this influence varying by charity, and it being especially difficult to estimate for charities we haven’t recommended before.
So far, we’ve estimated cost-effectiveness for specific programs, not a charity’s overall cost-effectiveness, so comparisons are relative to those programs. For THL’s plans to expand the Open Wing Alliance, for instance, we assessed the cost-effectiveness (and theory of change) of the OWA itself, not THL’s overall cost-effectiveness.
Overall, I largely agree, and we already do what you’re suggesting in some cases, though not this one. For these reasons and others, I’m not sure investigating the marginal cost-effectiveness of OWA Europe and Africa specifically would have been the best use of limited evaluation time, but we do that kind of analysis when we think it’s decision-relevant.
ACE influencing significantly more funding than we distribute directly, this influence varying by charity
You have estimates for the additional funding charities get as a result of your recommendation (accounting for additional grants from ACE, and donations from individual donors)? If so, you could analyse the cost-effectiveness of that additional funding.
it being especially difficult to estimate for charities we haven’t recommended before
Have you considered assessing the marginal cost-effectiveness of charities you have recommended in the past, and the overall cost-effectiveness of charities you have never recommended?
[...] For THL’s plans to expand the Open Wing Alliance, for instance, we assessed the cost-effectiveness (and theory of change) of the OWA itself, not THL’s overall cost-effectiveness.
[...] For these reasons and others, I’m not sure investigating the marginal cost-effectiveness of OWA Europe and Africa specifically would have been the best use of limited evaluation time, but we do that kind of analysis when we think it’s decision-relevant.
The programs from THL you assessed had a cost of 7.96 M$ (= (7.65 + 0.311)*10^6), 19.9 (= 7.96*10^6/(399*10^3)) times the amount you granted to THL in 2025. I understand you made more funding go to THL than what you granted, but still significantly less than 19.9 times as much? If so, I think the cost-effectiveness analysis could have focussed on more marginal spending. I believe the marginal cost-effectiveness of large programs could be significantly lower than their overall cost-effectiveness. THL’s programs are quite large. So I would have thought that assessing their marginal cost-effectiveness is especially important.
Hi Vince. Do you have any plans to estimate the marginal cost-effectiveness of charities? I agree with Giving What We Can (GWWC) that this would be a major improvement to ACE’s evaluations.
ACE has been putting more emphasis on selecting programs for cost-effectiveness (and theory of change) analysis based on marginal plans and how likely we are fund them to, so we are estimating marginal cost-effectiveness.
That being said, I don’t think our methods in 2025 were far off from where they are now. We continue to think that current cost-effectiveness can be the best proxy for marginal future cost-effectiveness in many cases, and that focusing narrowly on where we predict our funding will be going would a mistake.
For example, The Humane League’s future plans are a prioritized list of 24 ways that they would expand with additional funding. Not only is it not feasible for us to construct so many forward-looking CEAs, but the resulting estimates would also carry higher uncertainty. Additionally, many plans are expansions of existing programs or are general operational investments, which depend on current cost-effectiveness.
Funding is also fungible; dollars we direct to THL won’t cleanly fund items 1-3 versus 4-6 on the list. Actual deployment depends on what other funders do, internal decisions, and new opportunities that emerge. Trying to predict which specific items our marginal dollar pays for is false precision, and we’ve seen charities greatly change their marginal plans within the two-year span of their recommendation.
Instead, we use a mix of current cost-effectiveness, our understanding of potential diminishing returns, qualitative judgments about effectiveness and strategic prioritization, and speculative, forward-looking CEAs to make our decisions. Current cost-effectiveness captures an organization’s demonstrated ability to convert dollars into outcomes and that is often what is most likely to carry over to the next dollar. Large shifts in marginal productivity is likely low, especially for established orgs running stable programs with strong prioritization, so current cost-effectiveness is usually within a reasonable range of marginal future cost-effectiveness.
Thanks for the reply, Vince.
ACE’s Recommended Charity Fund granted 399 k$ (= (85.4 + 314)*10^3) to THL in 2024, and the 3 highest priority future plans from THL have a cost of 455 k$. So, assuming that the amount granted to THL covers most of the additional funding it received as a result of your recommendation, you would only need to model the cost-effectiveness of 3 programs, or maybe just 2, as the 2 highest priority projects cost 78.7 % (= 358⁄455) as much as the 3 highest priority projects? In your last cost-effectiveness analysis of THL, you looked into 4 programs.
Makes sense. At the same time, I still think assessing the cost-effectiveness of marginal projects is valuable. Imagine THL ends up using ACE’s funds to support projects 4 to 6 instead of projects 1 to 3, which had the highest priority at the time of your evaluation, and were the only ones covered by your cost-effectiveness analyses. One should expect the cost-effectiveness of projects 4 to 6 to be closer to what ACE estimated for projects 1 to 3 than to the cost-effectiveness of THL as a whole? I think so. Mostly because projects 4 to 6 were much closer to being marginal at the time of your evaluation than random funds from THL. And partly because THL prioritising projects 4 to 6 would be an update towards projects 1 to 3 having a lower cost-effectiveness than you estimated, and projects 4 to 6 having a higher cost-effectiveness than you estimated.
I agree it would not make sense to predict changes in what charities will prioritise. I would just assess the cost-effectiveness of the highest priority projects covering the expected increase in funding resulting from your recommendation.
I suspect the marginal cost-effectiveness of large organisations may be significantly lower than the cost-effectiveness of the whole spending. My intuition having worked for the last 9 months at Anima International is that our marginal cost-effectiveness is 10 % to 50 % of the cost-effectiveness of our whole spending. Relatedly, @abrahamrowe guessed 16 months ago corporate campaigns for chicken welfare not funded by Coefficient Giving (CG), including ones run by organisations funded by CG, were as cost-effective as burning money, despite some of the ones funded by CG being beneficial. Here are some related reflections from Abraham. I was surprised by Abraham’s claim at the time, but I have meanwhile come closer to his position, although I think I am still significatly more optimistic than he was at the time.
What you describe is roughly how we decide each Recommended Charity Fund distribution, but unfortunately recommendation decisions have more complexity. Some examples of this complexity include ACE influencing significantly more funding than we distribute directly, this influence varying by charity, and it being especially difficult to estimate for charities we haven’t recommended before.
So far, we’ve estimated cost-effectiveness for specific programs, not a charity’s overall cost-effectiveness, so comparisons are relative to those programs. For THL’s plans to expand the Open Wing Alliance, for instance, we assessed the cost-effectiveness (and theory of change) of the OWA itself, not THL’s overall cost-effectiveness.
Overall, I largely agree, and we already do what you’re suggesting in some cases, though not this one. For these reasons and others, I’m not sure investigating the marginal cost-effectiveness of OWA Europe and Africa specifically would have been the best use of limited evaluation time, but we do that kind of analysis when we think it’s decision-relevant.
You have estimates for the additional funding charities get as a result of your recommendation (accounting for additional grants from ACE, and donations from individual donors)? If so, you could analyse the cost-effectiveness of that additional funding.
Have you considered assessing the marginal cost-effectiveness of charities you have recommended in the past, and the overall cost-effectiveness of charities you have never recommended?
The programs from THL you assessed had a cost of 7.96 M$ (= (7.65 + 0.311)*10^6), 19.9 (= 7.96*10^6/(399*10^3)) times the amount you granted to THL in 2025. I understand you made more funding go to THL than what you granted, but still significantly less than 19.9 times as much? If so, I think the cost-effectiveness analysis could have focussed on more marginal spending. I believe the marginal cost-effectiveness of large programs could be significantly lower than their overall cost-effectiveness. THL’s programs are quite large. So I would have thought that assessing their marginal cost-effectiveness is especially important.