Hi Sanjay, thanks for writing this. As we have discussed, I agree with some of this and disagree with other parts.
1. On whether the pledged funds will be forthcoming. I agree that the pessimistic estimate of funds forthcoming was probably too high, though I haven’t looked at how much money has actually come out in the past year. However, I don’t think this that big an effect on the CEA because the pessimistic estimate also assumes a cost per tonne of $30 (vs the $5 per tonne that you assume here) to abate CO2 through deforestation prevention. In the model, this offsets the potential overestimate of the forthcoming funds by a factor of 6, which makes the end estimate similar to the one you produce. I’m also not sure it is right to anchor so much on how much money has been disbursed so far, given that the model assesses the money that will be disbursed through REDD+ over all time, and not just the preceding year.
2. On the counterfactual impact of funds. I agree that this is in principle a gap in the CEA. However, this criticism also applies to almost all CEAs I have ever seen. Accounting for all counterfactuals in CEA models is very hard. Moreover, as you note, we do try to account for the counterfactual in the model by trying to estimate how much of the additional funding for REDD+ counterfactually contributes to additional CO2 reductions. We do this in the section where we discuss the interaction between carbon pricing and the effect of freeing up relatively cheap forestry offsets. The argument is that carbon is priced at a very low level worldwide (<$10/t), so opening up <$10/t offsets does free up additional funds for climate change that would not otherwise have gone to climate change. This also applies for planting trees, since REDD+ in principle covers such activity, so I don’t think that could be a reason to downgrade CfRN’s cost-effectiveness.
I agree that the funds spent on REDD+ could have gone to global development and this isn’t accounted for in the model, but (1) to put this criticism in context, this is also true of almost all other CEAs that I have seen—you could do this in a CEA for FHI for example—money to them could have gone to global development. It becomes very unwieldy to measure such things. (2) Standard EA wisdom is that a lot of govt global development spending isn’t very impactful. It is also of course hard to know how to trade off CO2 and global development metrics, but this seems to me at most a reason to very modestly reduce your estimate of CfRN’s cost-effectiveness. I personally think that climate change is clearly better than global development from a long-termist point of view, so directing money to the former is far better than the directing money to the latter.
On counterfactual private sector funds, I’m not sure I agree with this. The government compulsion we refer to in the report is assuming that they impose a carbon price of <$10/t. For the reasons mentioned above, I don’t think there are many other <$10/t offsets aside from forests.
3. Insufficient incentive funds. This is definitely a concern about REDD+ and I had hoped it would have (1) picked up more over the last year (maybe it has I haven’t checked) and (2) constrained Bolsonaro’s policies more due to the financial incentives (though I haven’t looked into this this year either).
I’m not sure I agree that this is a good reason not to support CfRN. One could also argue that this makes it especially important to make sure REDD+ does not collapse and get replaced by nothing/something worse. It is (I think we agree) in principle a good idea, but there is a fair way to go on the implementation side. But I can also see the force of your argument.
4. Future private sector demand for compliance-grade offsets. I agree that the rationale surrounding the chart was a mistake and could have included other reasons for the decline in demand. However, as you say, this isn’t the only piece of evidence that we produce for this estimate. The argument is that carbon pricing will incentivise private companies to buy high-grade offsets. I still think this is true. I agree that it is unlikely that corporates will buy such offsets for extra security of having an impact, though this was not part of our argument for the private sector funding projections.
The idea is not that CfRN ensure that the private funding goes through the registry and exchange but rather that REDD+ offsets are recognised as high enough quality to be included in carbon pricing schemes, incentivising corporates to buy such offsets.
5. On it being overly generous to assign all of these benefits to CfRN. I think this is a philosophical difference in measuring counterfactual impact. Some evaluators give orgs a portion of ‘the credit’ for some amount of impact, but I don’t think this is correct. We measure the impact of CfRN as a speed up in deforestation prevention, rather than giving them a portion of the credit, which I don’t think is an idea that makes conceptual sense.
I do think it is plausible that if CfRN had not existed, agreement on a system for forestry protection would have been delayed for 2-5 years and arguably much longer (it is extremely hard to say). (This also means that Paris Agreement would probably also have been delayed by many years). So, I do think it is plausible that CfRN have counterfactually released massive amounts of money for forests despite having a small budget. It is important to remember that CfRN are unusual in that they are an intergovernmental org and have a seat at the table at climate negotiations where they represent all of the world’s largest rainforest countries except Brazil.
These disagreements aside, I encourage more efforts at checking charity recommendations rather than taking them on faith, so thanks again for doing this. Also, Founders Pledge has hired a new climate policy expert and we will be revisiting our climate research over the next few months and will assess our old recommendations and hopefully add new ones.
Thanks to Sanjay and you, I found this very interesting to follow!
Just a quick note on your point 2:
On the counterfactual impact of funds. I agree that this is in principle a gap in the CEA. However, this criticism also applies to almost all CEAs I have ever seen. Accounting for all counterfactuals in CEA models is very hard.
It’s quite possible that I’m missing something here, because I’m not familiar with the context, and haven’t tried to dig into the details. - But FWIW, my initial reaction was that this response doesn’t quite speak to what I understood to be the specific concern in the OP, i.e.:
To be clear, this section is *not* considering the opportunity cost of your (the donor’s) money going to CfRN. Rather, CfRN enables other funds to be raised and donated to REDD+ – this section is considering the opportunity cost on those funds.
I agree with you (Halstead) that the opportunity cost of the donor’s money is almost never accounted for in a CEA. (This also wouldn’t be required conceptually—choosing the donation target with maximal cost-effectiveness across all CEAs would be sufficient to minimize opportunity cost, conditional on the CEAs being individually correct and jointly comprehensive.)
I also agree that “accounting for all counterfactuals in CEA models is very hard”.
However, from my (possibly uninformed) perspective, the force of the OP’s argument is due to an appeal to a somewhat specific, contingent property of the intervention under consideration—namely that this CEA is assessing the cost-effectiveness of a donation the primary purpose of which is to cause a change in the allocation of other (here, largely government) funds.
I think this situation is not at all analogous to “money to [FHI] could have gone to global development”. I in fact think it’s similar to why e.g. 80,000 Hours considers plan changes as their main metric. Or, to give a hypothetical example, consider a health intervention aiming to make people buy more vegetables; when assessing the impact of this intervention, I would want to know whether people who end up buying more vegetables have reduced their expenses for grains or chocolate or clothing, whether they’ve taken out a loan to buy more vegetables etc. - And this concern is quite distinct from the concern that “donations to fund this intervention could also have been donated elsewhere”.
Hello, my response was about the counterfactual value of funds to REDD+ - i.e. what govts and the private sector would spend money on. It is analogous to a donation to FHI: Sanjay is proposing that we should discount money to REDD+ projects because part of the money would otherwise have gone to global development. In the same way, one could argue that money donated to FHI would otherwise have gone to global development and discount by that. This is in principle correct, but it tends not to be done.
Max_Daniel is on the nose about what we were trying to convey.
Let’s imagine another hypothetical example from global development:
imagine that the Fairly Good Charity is 50% as good as AMF
and imagine that Warm Fuzzy Fundraising encourages current AMF donors to donate to Fairly Good Charity instead of AMF
Would we consider Fairly Good Charity to have negative impact because the funds could have gone to AMF? Arguably we could do, but in practice we don’t.
What about a donation to support the operations of Warm Fuzzy Fundraising? I think this is a negative impact. (there’s a parallel with Max_Daniel’s vegetable example)
And coming back to climate change, if we thought that funds going to REDD+ were displacing higher-impact uses of the money, then CfRN too would have net negative impact.
I agree with that. My response is (1) to contextualise this by saying that this feature is true of almost all CEAs, (2) to say that I don’t think the counterfactual use of funds is very good in comparison to effective spending on deforestation prevention.
Hi Sanjay, thanks for writing this. As we have discussed, I agree with some of this and disagree with other parts.
1. On whether the pledged funds will be forthcoming. I agree that the pessimistic estimate of funds forthcoming was probably too high, though I haven’t looked at how much money has actually come out in the past year. However, I don’t think this that big an effect on the CEA because the pessimistic estimate also assumes a cost per tonne of $30 (vs the $5 per tonne that you assume here) to abate CO2 through deforestation prevention. In the model, this offsets the potential overestimate of the forthcoming funds by a factor of 6, which makes the end estimate similar to the one you produce. I’m also not sure it is right to anchor so much on how much money has been disbursed so far, given that the model assesses the money that will be disbursed through REDD+ over all time, and not just the preceding year.
2. On the counterfactual impact of funds. I agree that this is in principle a gap in the CEA. However, this criticism also applies to almost all CEAs I have ever seen. Accounting for all counterfactuals in CEA models is very hard. Moreover, as you note, we do try to account for the counterfactual in the model by trying to estimate how much of the additional funding for REDD+ counterfactually contributes to additional CO2 reductions. We do this in the section where we discuss the interaction between carbon pricing and the effect of freeing up relatively cheap forestry offsets. The argument is that carbon is priced at a very low level worldwide (<$10/t), so opening up <$10/t offsets does free up additional funds for climate change that would not otherwise have gone to climate change. This also applies for planting trees, since REDD+ in principle covers such activity, so I don’t think that could be a reason to downgrade CfRN’s cost-effectiveness.
I agree that the funds spent on REDD+ could have gone to global development and this isn’t accounted for in the model, but (1) to put this criticism in context, this is also true of almost all other CEAs that I have seen—you could do this in a CEA for FHI for example—money to them could have gone to global development. It becomes very unwieldy to measure such things. (2) Standard EA wisdom is that a lot of govt global development spending isn’t very impactful. It is also of course hard to know how to trade off CO2 and global development metrics, but this seems to me at most a reason to very modestly reduce your estimate of CfRN’s cost-effectiveness. I personally think that climate change is clearly better than global development from a long-termist point of view, so directing money to the former is far better than the directing money to the latter.
On counterfactual private sector funds, I’m not sure I agree with this. The government compulsion we refer to in the report is assuming that they impose a carbon price of <$10/t. For the reasons mentioned above, I don’t think there are many other <$10/t offsets aside from forests.
3. Insufficient incentive funds. This is definitely a concern about REDD+ and I had hoped it would have (1) picked up more over the last year (maybe it has I haven’t checked) and (2) constrained Bolsonaro’s policies more due to the financial incentives (though I haven’t looked into this this year either).
I’m not sure I agree that this is a good reason not to support CfRN. One could also argue that this makes it especially important to make sure REDD+ does not collapse and get replaced by nothing/something worse. It is (I think we agree) in principle a good idea, but there is a fair way to go on the implementation side. But I can also see the force of your argument.
4. Future private sector demand for compliance-grade offsets. I agree that the rationale surrounding the chart was a mistake and could have included other reasons for the decline in demand. However, as you say, this isn’t the only piece of evidence that we produce for this estimate. The argument is that carbon pricing will incentivise private companies to buy high-grade offsets. I still think this is true. I agree that it is unlikely that corporates will buy such offsets for extra security of having an impact, though this was not part of our argument for the private sector funding projections.
The idea is not that CfRN ensure that the private funding goes through the registry and exchange but rather that REDD+ offsets are recognised as high enough quality to be included in carbon pricing schemes, incentivising corporates to buy such offsets.
5. On it being overly generous to assign all of these benefits to CfRN. I think this is a philosophical difference in measuring counterfactual impact. Some evaluators give orgs a portion of ‘the credit’ for some amount of impact, but I don’t think this is correct. We measure the impact of CfRN as a speed up in deforestation prevention, rather than giving them a portion of the credit, which I don’t think is an idea that makes conceptual sense.
I do think it is plausible that if CfRN had not existed, agreement on a system for forestry protection would have been delayed for 2-5 years and arguably much longer (it is extremely hard to say). (This also means that Paris Agreement would probably also have been delayed by many years). So, I do think it is plausible that CfRN have counterfactually released massive amounts of money for forests despite having a small budget. It is important to remember that CfRN are unusual in that they are an intergovernmental org and have a seat at the table at climate negotiations where they represent all of the world’s largest rainforest countries except Brazil.
These disagreements aside, I encourage more efforts at checking charity recommendations rather than taking them on faith, so thanks again for doing this. Also, Founders Pledge has hired a new climate policy expert and we will be revisiting our climate research over the next few months and will assess our old recommendations and hopefully add new ones.
Thanks to Sanjay and you, I found this very interesting to follow!
Just a quick note on your point 2:
It’s quite possible that I’m missing something here, because I’m not familiar with the context, and haven’t tried to dig into the details. - But FWIW, my initial reaction was that this response doesn’t quite speak to what I understood to be the specific concern in the OP, i.e.:
I agree with you (Halstead) that the opportunity cost of the donor’s money is almost never accounted for in a CEA. (This also wouldn’t be required conceptually—choosing the donation target with maximal cost-effectiveness across all CEAs would be sufficient to minimize opportunity cost, conditional on the CEAs being individually correct and jointly comprehensive.)
I also agree that “accounting for all counterfactuals in CEA models is very hard”.
However, from my (possibly uninformed) perspective, the force of the OP’s argument is due to an appeal to a somewhat specific, contingent property of the intervention under consideration—namely that this CEA is assessing the cost-effectiveness of a donation the primary purpose of which is to cause a change in the allocation of other (here, largely government) funds.
I think this situation is not at all analogous to “money to [FHI] could have gone to global development”. I in fact think it’s similar to why e.g. 80,000 Hours considers plan changes as their main metric. Or, to give a hypothetical example, consider a health intervention aiming to make people buy more vegetables; when assessing the impact of this intervention, I would want to know whether people who end up buying more vegetables have reduced their expenses for grains or chocolate or clothing, whether they’ve taken out a loan to buy more vegetables etc. - And this concern is quite distinct from the concern that “donations to fund this intervention could also have been donated elsewhere”.
Hello, my response was about the counterfactual value of funds to REDD+ - i.e. what govts and the private sector would spend money on. It is analogous to a donation to FHI: Sanjay is proposing that we should discount money to REDD+ projects because part of the money would otherwise have gone to global development. In the same way, one could argue that money donated to FHI would otherwise have gone to global development and discount by that. This is in principle correct, but it tends not to be done.
Max_Daniel is on the nose about what we were trying to convey.
Let’s imagine another hypothetical example from global development:
imagine that the Fairly Good Charity is 50% as good as AMF
and imagine that Warm Fuzzy Fundraising encourages current AMF donors to donate to Fairly Good Charity instead of AMF
Would we consider Fairly Good Charity to have negative impact because the funds could have gone to AMF? Arguably we could do, but in practice we don’t.
What about a donation to support the operations of Warm Fuzzy Fundraising? I think this is a negative impact. (there’s a parallel with Max_Daniel’s vegetable example)
And coming back to climate change, if we thought that funds going to REDD+ were displacing higher-impact uses of the money, then CfRN too would have net negative impact.
I agree with that. My response is (1) to contextualise this by saying that this feature is true of almost all CEAs, (2) to say that I don’t think the counterfactual use of funds is very good in comparison to effective spending on deforestation prevention.