(Written on a plane to Christmas holidays, so apologies for lack of links, and further engagement being delayed to 2023). Thanks for writing this! I am looking forward to more work on this and to find the best thing in this space, but I am extremely skeptical of the claim that we do not know anything about how anti-deforestation compares to other things we can fund in climate.
As my team and I have spent thousands of hours at this point working on what gives rise to relative cost effectiveness in climate (which is what this claim is about) and I also conducted a 50h+ investigation on CfRN and REDD+ in 2020 when reevaluting (we were unable to publish this for organizational reasons, FWIW our estimates where in the upper end of the range cited here and I’d be happy to share in DM), I wanted to explain why (not sorted by importance, but by explanatory logic moving from more outside to more inside view):
1. First, in some trivial sense, it is of course true that we do not know with 100% certainty. But the claim that we know nothing about the relative cost effectiveness of anti-deforestation feels akin to the following statements: (a) “We do not know for certain that giving to the Center for Health Security (CHS) or donating PPE to your local hospital is more effective to reducing biorisk” , (b) “We do not know for certain that giving to the Good Food Institute is better to reduce animal suffering than giving to your cat shelter next door”. These statements are trivially true, but I assume almost everyone reading this would feel quite confident in CHS > PPE for local hospitals and GFI > local cat shelter and would not be distraught by the lack of RCT evidence or unified identical methodology across evaluators. My tldr point is we should feel the same about this case, because—like in those cases—we know a lot about the structure of the problem including philanthropic biases, but also about the effectiveness of different strategies.
2. The reason for that, to me, seems a Bayesian mindset of “what should we believe given the evidence?” rather than what are we 100% certain of, a state we will never reach in an intervention space covering decades, uncertain futures, and global human and natural systems.
3. So what is the evidence against ignorance on cost-effectiveness? At its most outside-view-y the fact that three evaluators that have looked at this independently (FP, SoGive, Giving Green) and none of them has anti-deforestation as their top recommendation is evidence against this. The fact that FP stopped recommending CfRN is additional evidence of this kind, as is SoGive’s work that led EA away from early recs in this space and the recent work by Vasco et al on tree planting cost-effectiveness.
4. This evidence is stronger when one takes into account significant donor pressures to find anti-deforestation good. I am sure we could all advise a lot more money if we had rainforest charities in our top recs, the opportunity cost of not recommending in this space is high in terms of advised money. (This is also the main reason I am writing this reply now despite heading out for holidays, I think such a claim of ignorance is quite negative for effective giving in climate because a lot of donors want to believe this as much as Mulder wants to believe in aliens).
5. Going one step less meta, Vasco et al’s recent piece has an appendix that discusses considerations on what a reasonable prior on this might be. I am certainly biased in that they rely on our report to form that prior, but I think it gives a good brief overview of the key considerations why one should approach cost-effectiveness of forest-related interventions with skepticism compared to alternatives.
6. The most important reason against to me seems the utter non-neglectednees of anti-deforestation (even Republicans like planting trees!). I literally get emails from people I have not talked to in years who know I work in climate that ask me how to save the rainforest. Less anecdotally, natural solutions is one of the largest pots in climate philanthropy and the Bezos Earth Fund and several other for the other largest climate philanthropists have strongly increased giving in this space, so it is not only one of the largest spaces, it is also one of the fastest growing ones. What is more, philanthropy focused on conservation, also including interventions like this, is several X larger than climate philanthropy (last I checked at least 2-3). This should make one very sceptical to be able to find (under reasonable search cost, including opportunity cost) high quality room for funding that would not otherwise be quickly filled unless one has a systematic theory of why all large donors overlook this. (Maybe such an opportunity exists with small organizations that large donors consider peanuts, but for country level REDD+ this special consideration does not apply). Almost any charity will almost always offer room for funding and a good story so analysis of philanthropy data and donor behavior patterns is crucial to analyze in a space with as much money influx as climate when considering counterfactually adjusted cost-effectiveness which is what we are after.
7. More broadly, there has been a multi-decade global effort to find tractable cost effective interventions around avoiding deforestation. Given the salience of conservation before climate, I assume they preceed climate, but since I am more familiar with climate I will focus on this. Essentially ever since the birth of international climate policy in the early 1990s the international community has tried to solve the problem of rewarding poor countries for taking cheap abatement options, often in the context of forests and with REDD(+) since the early 2000s. Everyone wants this to work, whole governments (Norway in particular) spend enormous resources to make this work. Yet, as of mid-2020, when I last looked at the literature in depth, I think it was pretty clear this by and large did not work in a way that achieves additional and permanent anti-deforestation. It’s of course possible something fundamentally changed since then but I haven’t heard of it despite following climate news pretty closely.
8. I think we have good reasons to expect this to remain intractable because the two biggest problems in the space are irresolvable.Permanence can never be guaranteed until we figure out how to construe contracts that bind future governments of sovereign countries. Additionality of avoided deforestation can never be guaranteed until we can run and compare multiple worlds. These are not techical or governance issues that can be solved with better satellites or slightly improved international frameworks, these are deep reasons to be skeptical on the tractability of avoided deforestation vs more tractable strategies.
9. More subtly but quite importantly, the effectiveness of investing in anti-deforestation is negatively correlated with its value, anti-deforestation is least effective in the futures where it would matter most. This is the opposite of what we want, we would want at least robustness (independence of effectiveness and value) and ideally, for lack of a better term, hedginess (positive correlation between effectiveness and value; I discuss this much more in my SERI talk on the structure or climate risk and its implications for high-impact climate philanthropy).
Much of the uncertainty on cost-effectiveness of anti-deforestation is driven by uncertainty about permanence and additionality, which is arguably related to international and national political conditions. Anti-deforestation via REDD+ will be most cost-effective in futures where international cooperation is good and we get a functioning and, crucially, lasting global anti-deforestation market. These are worlds where the social cost of carbon, the value of additional abatement, is likely quite low because these conditions correspond with good climate progress overall. Conversely, in worlds where additional abatement is particularly valuable, with poor international coordination and low climate policy and lots of energy-intensive growth, REDD+ is unlikely to work. This sounds like an absurdly technical point, but given the shape of climate damage it is actually quite important and it makes reduced emissions in expectation the wrong metric for evaluating climate impact as long as we know something about how different interventions correlate with warming trajectories (and thus marginal value of abatement).
Crucially, energy innovation as the generally considered most cost-effective intervention (at least naively, when not adjusting for recent crowding), does not have this weakness. Somewhat roughly, solar will still be relatively cheap when climate policy breaks down, the world forms blocs, and there’s lots of energy-intensive growth in emerging economies, but REDD+ will likely fail.
10. I could go on but this comment is already too long. To summarize, it seems to me we have all kinds of relevant evidence that should make us quite skeptical of the claim that anti-deforestation is likely to be particularly cost-effective, in particular in relative terms. It looks bad on the outside view, on neglectedness, it looks bad on tractability and on many more inside-view considerations. Moreover we do have more evidence than the post grants on the relative cost-effectiveness of other interventions. Since I first made the argument for innovation advocacy for neglected technologies in EA in 2016/2017 this was vetted by Let’s Fund, Founders Pledge (before I joined), SoGive, and Giving Green. The fact that this theory of change / intervention strategy has stood the test of time across evaluators does carry information. While I am not sure it’s still the best thing to do on the margin (given how philanthropic additionality is changing, I think it’s possible that interventions to avoid carbon lock-in might beat it), it seems very unlikely for all the reasons discussed above that we should be in a state of ignorance when we compare it to REDD+.
I am still curious to see what you find and I think that when one is willing to incur a comparatively very large search cost there might be good things that can be found. But I don’t think we are in a state of ignorance until then with regards to what we should expect on its relative cost-effectiveness, especially compared to interventions that have been investigated and found cost-effective by several evaluators.
Thanks for your very thoughtful comments, Johannes.
Please note that we do not have the bandwidth to respond in detail here to each of the 10 points you lay out in your comment (nor to all Forum comments on our reports), but we will do our best to respond to those we find most critical to the conversation and would love to find a time to further discuss these and your remaining points over separate correspondence.
With regard to the Bayesian mindset to which you allude (“what should we believe, given the evidence?), we agree with this approach. Of course, the evidence in the space of climate economics and policy is vast, so we hope we can agree that this approach is a significant undertaking that would benefit from coordination and collaboration across relevant orgs in EA. Given the vastness and complexity of various climate-relevant literatures, we believe that there is a real risk of jumping to conclusions too hastily, hence our interest in introducing some form of time-efficient peer review process among the major EA climate research organizations.
Our aim in this project was to summarize and discuss the best evidence we could find on the (cost-)effectiveness of REDD+ over about 8 weeks of two full-time researchers’ time, and to draw reasonable conclusions about the interventions’ potential impact and CE. We do not claim that this research is fully comprehensive and we caveat that it was undertaken early in 2022 (e.g., before the recent presidential election in Brazil or the 30x30 agreement, the implications of which we have not researched in any depth).
That said, we do hope that the EA community can use the research as a starting point for coming to a consensus on the CE of REDD+ as a climate intervention. We aim to be quite clear about our main uncertainties in the report and provide some suggestions for how researchers could usefully spend more time. After finding little in the way of comparable estimates in the EA climate literature, we concluded that it would be quite difficult to confidently claim that REDD+ was a relatively cost-effective intervention relative to other climate interventions, and even more difficult when broadening the comparison to global health and development interventions.
It is very possible that there is more consensus (using a comparably rigorous research approach, with comparable outcome measures) than we are aware of regarding the cost-effectiveness and potential impact of the myriad climate interventions that could plausibly “pass” an ITN shallow review, and that this EA consensus would be compatible with broader climate expert consensus. However, many of the EA-recommended climate charities we came across were related to climate advocacy, where cost-effectiveness estimates are understandably difficult to pin down given extreme sensitivity to modeler assumptions and relatively short-term political contexts, making direct comparison or ranking quite difficult.
We hope to discuss how to improve intervention comparability with you and others in the EA climate space in the near future! If there is indeed such consensus and comparability, perhaps we could collaborate to compile this information for ease of comparison, and we will update our report accordingly.
On the subject of neglectedness, the Bezos Earth Fund is indeed focused on natural climate solutions, though they have committed (i) “only” $10B over 10 years (where we anticipate RFMF an order of magnitude larger — and annually — over the next ten years for REDD+), and (ii) not specifically to the REDD+ framework, to our knowledge.
Regarding your point about historic REDD+ not having achieved additionality and permanence, we refer you to the report, where we largely agree with this perspective and hence recommend a focus on jurisdictional REDD+ programs that are gaining significant momentum, as well as continuous results-based payments for the entire duration of the desired avoided emissions. We also touch on reasons we expect additionality and permanence concerns to diminish going forward.
Finally, we have recently begun to consider whether it may make sense to recommend various options for different “types” of potential donors in the climate space (also for risk diversification). As you suggest, it is possible that a whole lot of potential philanthropic dollars are not being allocated (or are being allocated to other causes) due to donors’ preferences for near-term / more certain vs. long-term / more risky solutions, or for carbon savings alone vs. perhaps lower carbon savings with significant (human, gender, biodiversity, other) co-benefits, etc. Unless those potential philanthropic funds otherwise get allocated toward other high-impact charities (i.e. not “the cat shelter next door”, which seems plausible for individuals desiring near-term, tangible impact), neglecting to provide donation recommendations that align with donor preferences represents a missed opportunity for impact.
We have written some additional (and generally more specific) responses to some of your points, which we look forward to discussing with you in separate correspondence. Thanks again, and looking forward to continuing the conversation!
(Written on a plane to Christmas holidays, so apologies for lack of links, and further engagement being delayed to 2023). Thanks for writing this! I am looking forward to more work on this and to find the best thing in this space, but I am extremely skeptical of the claim that we do not know anything about how anti-deforestation compares to other things we can fund in climate.
As my team and I have spent thousands of hours at this point working on what gives rise to relative cost effectiveness in climate (which is what this claim is about) and I also conducted a 50h+ investigation on CfRN and REDD+ in 2020 when reevaluting (we were unable to publish this for organizational reasons, FWIW our estimates where in the upper end of the range cited here and I’d be happy to share in DM), I wanted to explain why (not sorted by importance, but by explanatory logic moving from more outside to more inside view):
1. First, in some trivial sense, it is of course true that we do not know with 100% certainty. But the claim that we know nothing about the relative cost effectiveness of anti-deforestation feels akin to the following statements: (a) “We do not know for certain that giving to the Center for Health Security (CHS) or donating PPE to your local hospital is more effective to reducing biorisk” , (b) “We do not know for certain that giving to the Good Food Institute is better to reduce animal suffering than giving to your cat shelter next door”. These statements are trivially true, but I assume almost everyone reading this would feel quite confident in CHS > PPE for local hospitals and GFI > local cat shelter and would not be distraught by the lack of RCT evidence or unified identical methodology across evaluators. My tldr point is we should feel the same about this case, because—like in those cases—we know a lot about the structure of the problem including philanthropic biases, but also about the effectiveness of different strategies.
2. The reason for that, to me, seems a Bayesian mindset of “what should we believe given the evidence?” rather than what are we 100% certain of, a state we will never reach in an intervention space covering decades, uncertain futures, and global human and natural systems.
3. So what is the evidence against ignorance on cost-effectiveness? At its most outside-view-y the fact that three evaluators that have looked at this independently (FP, SoGive, Giving Green) and none of them has anti-deforestation as their top recommendation is evidence against this. The fact that FP stopped recommending CfRN is additional evidence of this kind, as is SoGive’s work that led EA away from early recs in this space and the recent work by Vasco et al on tree planting cost-effectiveness.
4. This evidence is stronger when one takes into account significant donor pressures to find anti-deforestation good. I am sure we could all advise a lot more money if we had rainforest charities in our top recs, the opportunity cost of not recommending in this space is high in terms of advised money. (This is also the main reason I am writing this reply now despite heading out for holidays, I think such a claim of ignorance is quite negative for effective giving in climate because a lot of donors want to believe this as much as Mulder wants to believe in aliens).
5. Going one step less meta, Vasco et al’s recent piece has an appendix that discusses considerations on what a reasonable prior on this might be. I am certainly biased in that they rely on our report to form that prior, but I think it gives a good brief overview of the key considerations why one should approach cost-effectiveness of forest-related interventions with skepticism compared to alternatives.
6. The most important reason against to me seems the utter non-neglectednees of anti-deforestation (even Republicans like planting trees!). I literally get emails from people I have not talked to in years who know I work in climate that ask me how to save the rainforest. Less anecdotally, natural solutions is one of the largest pots in climate philanthropy and the Bezos Earth Fund and several other for the other largest climate philanthropists have strongly increased giving in this space, so it is not only one of the largest spaces, it is also one of the fastest growing ones. What is more, philanthropy focused on conservation, also including interventions like this, is several X larger than climate philanthropy (last I checked at least 2-3). This should make one very sceptical to be able to find (under reasonable search cost, including opportunity cost) high quality room for funding that would not otherwise be quickly filled unless one has a systematic theory of why all large donors overlook this. (Maybe such an opportunity exists with small organizations that large donors consider peanuts, but for country level REDD+ this special consideration does not apply). Almost any charity will almost always offer room for funding and a good story so analysis of philanthropy data and donor behavior patterns is crucial to analyze in a space with as much money influx as climate when considering counterfactually adjusted cost-effectiveness which is what we are after.
7. More broadly, there has been a multi-decade global effort to find tractable cost effective interventions around avoiding deforestation. Given the salience of conservation before climate, I assume they preceed climate, but since I am more familiar with climate I will focus on this. Essentially ever since the birth of international climate policy in the early 1990s the international community has tried to solve the problem of rewarding poor countries for taking cheap abatement options, often in the context of forests and with REDD(+) since the early 2000s. Everyone wants this to work, whole governments (Norway in particular) spend enormous resources to make this work. Yet, as of mid-2020, when I last looked at the literature in depth, I think it was pretty clear this by and large did not work in a way that achieves additional and permanent anti-deforestation. It’s of course possible something fundamentally changed since then but I haven’t heard of it despite following climate news pretty closely.
8. I think we have good reasons to expect this to remain intractable because the two biggest problems in the space are irresolvable. Permanence can never be guaranteed until we figure out how to construe contracts that bind future governments of sovereign countries. Additionality of avoided deforestation can never be guaranteed until we can run and compare multiple worlds. These are not techical or governance issues that can be solved with better satellites or slightly improved international frameworks, these are deep reasons to be skeptical on the tractability of avoided deforestation vs more tractable strategies.
9. More subtly but quite importantly, the effectiveness of investing in anti-deforestation is negatively correlated with its value, anti-deforestation is least effective in the futures where it would matter most. This is the opposite of what we want, we would want at least robustness (independence of effectiveness and value) and ideally, for lack of a better term, hedginess (positive correlation between effectiveness and value; I discuss this much more in my SERI talk on the structure or climate risk and its implications for high-impact climate philanthropy).
Much of the uncertainty on cost-effectiveness of anti-deforestation is driven by uncertainty about permanence and additionality, which is arguably related to international and national political conditions. Anti-deforestation via REDD+ will be most cost-effective in futures where international cooperation is good and we get a functioning and, crucially, lasting global anti-deforestation market. These are worlds where the social cost of carbon, the value of additional abatement, is likely quite low because these conditions correspond with good climate progress overall. Conversely, in worlds where additional abatement is particularly valuable, with poor international coordination and low climate policy and lots of energy-intensive growth, REDD+ is unlikely to work. This sounds like an absurdly technical point, but given the shape of climate damage it is actually quite important and it makes reduced emissions in expectation the wrong metric for evaluating climate impact as long as we know something about how different interventions correlate with warming trajectories (and thus marginal value of abatement).
Crucially, energy innovation as the generally considered most cost-effective intervention (at least naively, when not adjusting for recent crowding), does not have this weakness. Somewhat roughly, solar will still be relatively cheap when climate policy breaks down, the world forms blocs, and there’s lots of energy-intensive growth in emerging economies, but REDD+ will likely fail.
10. I could go on but this comment is already too long. To summarize, it seems to me we have all kinds of relevant evidence that should make us quite skeptical of the claim that anti-deforestation is likely to be particularly cost-effective, in particular in relative terms. It looks bad on the outside view, on neglectedness, it looks bad on tractability and on many more inside-view considerations. Moreover we do have more evidence than the post grants on the relative cost-effectiveness of other interventions. Since I first made the argument for innovation advocacy for neglected technologies in EA in 2016/2017 this was vetted by Let’s Fund, Founders Pledge (before I joined), SoGive, and Giving Green. The fact that this theory of change / intervention strategy has stood the test of time across evaluators does carry information. While I am not sure it’s still the best thing to do on the margin (given how philanthropic additionality is changing, I think it’s possible that interventions to avoid carbon lock-in might beat it), it seems very unlikely for all the reasons discussed above that we should be in a state of ignorance when we compare it to REDD+.
I am still curious to see what you find and I think that when one is willing to incur a comparatively very large search cost there might be good things that can be found. But I don’t think we are in a state of ignorance until then with regards to what we should expect on its relative cost-effectiveness, especially compared to interventions that have been investigated and found cost-effective by several evaluators.
Thanks for your very thoughtful comments, Johannes.
Please note that we do not have the bandwidth to respond in detail here to each of the 10 points you lay out in your comment (nor to all Forum comments on our reports), but we will do our best to respond to those we find most critical to the conversation and would love to find a time to further discuss these and your remaining points over separate correspondence.
With regard to the Bayesian mindset to which you allude (“what should we believe, given the evidence?), we agree with this approach. Of course, the evidence in the space of climate economics and policy is vast, so we hope we can agree that this approach is a significant undertaking that would benefit from coordination and collaboration across relevant orgs in EA. Given the vastness and complexity of various climate-relevant literatures, we believe that there is a real risk of jumping to conclusions too hastily, hence our interest in introducing some form of time-efficient peer review process among the major EA climate research organizations.
Our aim in this project was to summarize and discuss the best evidence we could find on the (cost-)effectiveness of REDD+ over about 8 weeks of two full-time researchers’ time, and to draw reasonable conclusions about the interventions’ potential impact and CE. We do not claim that this research is fully comprehensive and we caveat that it was undertaken early in 2022 (e.g., before the recent presidential election in Brazil or the 30x30 agreement, the implications of which we have not researched in any depth).
That said, we do hope that the EA community can use the research as a starting point for coming to a consensus on the CE of REDD+ as a climate intervention. We aim to be quite clear about our main uncertainties in the report and provide some suggestions for how researchers could usefully spend more time. After finding little in the way of comparable estimates in the EA climate literature, we concluded that it would be quite difficult to confidently claim that REDD+ was a relatively cost-effective intervention relative to other climate interventions, and even more difficult when broadening the comparison to global health and development interventions.
It is very possible that there is more consensus (using a comparably rigorous research approach, with comparable outcome measures) than we are aware of regarding the cost-effectiveness and potential impact of the myriad climate interventions that could plausibly “pass” an ITN shallow review, and that this EA consensus would be compatible with broader climate expert consensus. However, many of the EA-recommended climate charities we came across were related to climate advocacy, where cost-effectiveness estimates are understandably difficult to pin down given extreme sensitivity to modeler assumptions and relatively short-term political contexts, making direct comparison or ranking quite difficult.
We hope to discuss how to improve intervention comparability with you and others in the EA climate space in the near future! If there is indeed such consensus and comparability, perhaps we could collaborate to compile this information for ease of comparison, and we will update our report accordingly.
On the subject of neglectedness, the Bezos Earth Fund is indeed focused on natural climate solutions, though they have committed (i) “only” $10B over 10 years (where we anticipate RFMF an order of magnitude larger — and annually — over the next ten years for REDD+), and (ii) not specifically to the REDD+ framework, to our knowledge.
Regarding your point about historic REDD+ not having achieved additionality and permanence, we refer you to the report, where we largely agree with this perspective and hence recommend a focus on jurisdictional REDD+ programs that are gaining significant momentum, as well as continuous results-based payments for the entire duration of the desired avoided emissions. We also touch on reasons we expect additionality and permanence concerns to diminish going forward.
Finally, we have recently begun to consider whether it may make sense to recommend various options for different “types” of potential donors in the climate space (also for risk diversification). As you suggest, it is possible that a whole lot of potential philanthropic dollars are not being allocated (or are being allocated to other causes) due to donors’ preferences for near-term / more certain vs. long-term / more risky solutions, or for carbon savings alone vs. perhaps lower carbon savings with significant (human, gender, biodiversity, other) co-benefits, etc. Unless those potential philanthropic funds otherwise get allocated toward other high-impact charities (i.e. not “the cat shelter next door”, which seems plausible for individuals desiring near-term, tangible impact), neglecting to provide donation recommendations that align with donor preferences represents a missed opportunity for impact.
We have written some additional (and generally more specific) responses to some of your points, which we look forward to discussing with you in separate correspondence. Thanks again, and looking forward to continuing the conversation!