Hello everyone. Well, this forum has blown up, and we (GG) have taken some punches. I want to list a few take-aways on my end:
One thing we’re hearing loud and clear is that there is a lot of worry among this group that having recommendations in categories that are not the most cost-effective will do more harm than good. I think this is worth considering, though I don’t totally agree. What I do agree with is that our site could be designed to lead people to the most cost-effective recommendations, and make deviations from this ideal more obvious. Based on this feedback, we’ve made some changes already to the site, changing ordering and adding text to be clear that we believe policy is the most effective donation category. We would like to make deeper changes in the future.
One argument in this thread that particularly resonated with me is that there would really be a lot of value for conducting some more rigorous cost-benefit analysis for the offset recommendations. While we still believe at this moment that our offset recommendations are solid, I think there is value in performing more explicit cost-effectiveness analysis and using this to re-evaluate our choices. This is something we intend to do in the near term.
There’s clearly a lot of disagreement over activism and our recommendation of the Sunrise Movement Education Fund (TSM) in particular. I’m still on the side of defending this recommendation, but hear and respect the arguments put forth and believe they deserve consideration. At the very least, we need to re-work our materials from this recommendation to lay out our assumptions and arguments better. I’ll write briefly more about activism and TSM a bit below.
Do you want to help? We’re currently at a place of limited funding, so we are not actively conducting research at the moment. That makes it hard to do all the improvements we’d like (and have promised above). Our plan was to spend the next couple of months fundraising before turning back to research, but this thread is making us reconsider that a bit. We’d be interested in taking on a volunteer or short term contract from the EA community to help put some of the legwork in shoring up our existing research (and looking at a couple of new things.) Interested? Have some experience in climate issues and thinking through cost-effectiveness in an EA framework? Reach out at givinggreen@idinsight.org
One thing that I’d like to make clear: our mission at Giving Green is to create the most social impact we can through our recommendations. This, notably, does not necessarily mean that we just find the most cost-effective charity and recommend it solely. I truly believe that by offering ‘best in class’ recommendations in different categories, we can reach a wider audience and increase the pie of money that we can influence. That’s why we do offsets. That is (partially) why we felt it important to search for donation options outside of the narrow mechanism of action advocated by Founder’s Pledge (“(i) advocates (ii) focused on accelerating (iii) neglected yet critical decarbonisation and carbon removal technologies” in the US). While we find FP’s arguments clear and compelling, I don’t share the confidence that this is the mechanism of action with the highest expected value of donations (more later on this). This lack of confidence plus the ability to widen the pool of donors we influence is why we think there is a lot of value in exploring other donation categories. I do understand the concern about dilution from the most cost-effective places, but believe this can be mitigated through clear (better than now) design and information on the website.
Now, just a little bit more on the meat of the disagreement, which is the TSM recommendation. If I understand Johannes’ argument, he agrees with the importance of activism for making policy, both for putting electoral pressure on legislators as well as agenda-setting for current legislators. But, if I were to summarize, there are two main arguments against activism: timing, neglectedness, and one more argument against TSM in particular: strategy.
First timing: Johannes declares “Biden has declared climate as one of his top four priorities,” as an argument that activist energy is no longer important in the current situation. First, I think it’s pretty clear the TSM can take huge credit for this being the case. They flexed their political muscles in the last election, won a seat at the Bernie-Biden task force, heavily influenced Biden’s climate strategy, and succeeded in putting it at the top of the agenda. While I understand the argument that this victory is now in place, I don’t think we should take this climate priority as a given and assume that it will stay in place without further activist pressure. Yes Biden has taken some executive actions on climate already, but there’s no concerted push (as far as I can tell yet) for concerted legislation. At one point Johannes asked what success looks like for TSM. My answer: It’s ensuring climate policy is a priority (in executive and regulatory action, as well as legislation), and indeed shifting the Overton window of climate policy. (Agree with Johannes that the “Green New Deal” is not a realistic piece of legislation.) And for that you need continued pressure.
Second, Neglectedness: Ok, we hear you that some of those numbers in our report on activism spending are out of date. Sorry. The late 2020 numbers from the ClimateWorks Foundation helps, but it appears that the quoted category of “Public engagement” spans a lot more than policy-focused activism, so I’m not sure this is so helpful either. We should probably do some work to get some better numbers, but overall I’m not so convinced that the amount of money spent tells us all that much about the marginal benefit of the extra dollar. For TSM in particular, their budget really ballooned in 2020 (financial reports aren’t out but we heard almost $10 million), so that’s a reason to give some pause. That being said, I don’t see a compelling case for diminishing returns at this level of funding, especially since activism relies on sheer numbers to be effective. But yes, I hear this argument and I think it’s something we should consider more closely.
Finally, strategy: A lot of the previous arguments have revolved around TSM potentially having a negative effect due to its polarization of the climate policy. I really think strategy (ie polarization vs compromise) is a deeply difficult issue that is the subject of many debates within US politics. It’s important to realize that this is a feature, not a bug. TSM has chosen to take a “radical” position and linked with other progressive priorities as a way of building a strong and cohesive political movement. It’s a strategy that has historical precedent, and has borne dividends at numerous other times in US History (civil rights movement, tea party). Does that mean it will work this time? Unclear. Is it risky? Sure. But to argue that you need a bipartisan strategy to influence policy in DC I believe ignores political realities. I agree that sometimes the solutions put forth by TSM (no CCS, etc.) are not ideal, but again this is part of the strategy to build a fired-up coalition. Like Johannes said, Sunrise are not going to be the ones at the table hammering out policy anyway. It’s just their job to get people to the table, and put power in the hands of the climate negotiators.
Our recommendation of TSM also stems on our philosophy of supporting multiple theories of change. There are lots of different ways that political change happens, and I think that the best way to enact the desired change is to accept this, be humble about deciding ex-ante which theory is right, and support multiple avenues.
All of this doesn’t really address Johannes’ main concern, which is that one needs to make the case not just for TSM to be “Good”, but to really be better than CATF at the margin. Based on our approach of providing recommendations in multiple categories and subscribing to multiple theories of change, I don’t think we need to show perfectly that TSM is better or equal to CATF. But we should be able to provide a compelling argument that impact is in the same ballpark. Johannes thinks it is not. I think it is. But also I think we need to do a better job of crafting this argument in our documents. That’s something we are going to work on.
Finally, one more thing. I think that there is a little bit of risk as an EA community of over-committing to a single method of action (“(i) advocates (ii) focused on accelerating (iii) neglected yet critical decarbonisation and carbon removal technologies” in the US) and very small set of organizations dedicated to this method. Like I said, I find FP’s arguments compelling and think their central recommendation of CATF is very solid. But I would be careful about overconfidence here. In our research, we interviewed 50+ experts from various sectors of the climate space. At some point in the conversation we always asked “If you could allocate money in the most cost-effective matter to fight climate change, what would you do?” we got a lot of different answers, but notably no one said CATF unprompted. We usually followed up and asked about CATF directly. When prompted, some people said “Oh yeah, they are great.” But others didn’t agree, and pointed out various issues with their approach or influence. I know this is not a scientific approach and don’t want to put too much weight on it, but I do just think it’s important for EAs as a community to approach this space with humility.
Hi, this is Dan from Giving Green. As you might imagine, I have a lot to say here.
First though, let me thank Alex for going about this criticism in what I would consider the right way: he brought his concerns to us, we had a discussion, and he changed some things based on the discussion. He also offered us a chance to comment on his draft to ensure he hadn’t said anything blatantly factually inaccurate. And then he aired his disagreements in a respectful post. So thanks for that Alex.
That being said, I fundamentally disagree with the majority of Alex’s points, and believe that the judgement calls we have made at Giving Green allow us to be impactful to a wider audience.
But let’s start with something else: Giving Green is a young organization, and I think we have a lot of room to improve and pivot. So criticism is welcomed, and some of Alex’s suggestions did resonate with us.
First, I think we could do a better job in promoting the donation options we think are “better” (ie policy, instead of offsets.) I think the offset research is valuable (as described below), but I agree that it’s not totally obvious to users of the website that we recommend policy over offsets, so that’s something we’d like to improve.
Second, although I do think we have some fundamental disagreements about the value of modeling uncertain situations, I do think there would be value in modeling the cost-effectiveness of offsets more explicitly. I think this is a case where the modeling assumptions are tractable, and we could provide users useful cost-effectiveness data, and may even promote certain offsets over others. This is something we’ve wanted to do for a while, but haven’t had the time to implement. (As Alex noted, we have limited funding and have relied heavily on “side of the desk” work to create Giving Green.)
Now onto the disagreements. I think to respond to every point I would have to write a book, but let me tackle the main ones.
Recommending Offsets: I made an argument defending recommending offsets (even though we believe they are less cost-effective than policy charities) on a comment previously on this post. The main idea is that there’s a tradeoff between certainty and high-risk, high-reward options, and I think there’s a market for both. I’ll paste the most fun part of the argument below.
“Finally, at the risk of going down a rabbit hole, one more point. There are a lot of parallels to this offset debate within international development/global health, an area in which EA is much more developed. Within EA communities, most people are quite comfortable with the recommendations from GiveWell, which are all direct-delivery of health services, and therefore things that can be measured with a high level of certainty. (Like offsets!) So why don’t big international development agencies (World Bank, etc) concentrate only on direct delivery of health services? It’s not because they are just stupid. It’s because they think they can have more bang for their buck investing in systemic changes that can’t be well-quantified with an RCT (like institution-building, macroeconomic stability, infrastructure, etc). Kinda like...funding charities that work on climate policy. So I would find it curious if the final consensus from EAs on global health is all about certainty, but in environment it is firmly for less-certain policy interventions. My argument would be that there is a clear place for both. “
Quantitative modeling: Alex is of the opinion that because we haven’t explicitly quantitatively modeled some of the tradeoffs we face, that the analysis isn’t to be trusted. I think we just have a fundamental difference of opinion on the value of modeling in situations of extreme uncertainty. Look, I’m a trained economist and am pro-modeling in general. But if you’re going to make a model where the outcomes are decided by key parameters that you have to make uninformed judgement calls on, what is the value of the model? Why not just make your judgement call on the outcome?
I know that modeling is in vogue in the EA community so perhaps this makes us outsiders, but I fundamentally believe that modeling in these circumstances leads only to science-y false precision, and does not actually give more clarity.
Let’s take an example, which leads into a discussion below. Let’s say we were trying to weigh the value of a donation to the Sunrise Movement Education Fund (TSM) vs Clean Air Task Force (CATF). Ok, you could model it, but at some point you’re going to have to make a judgement call on the fundamental tradeoff: CATF is more likely to cause incremental change (though some would argue that this is at the expense of entrenching fossil interests and hurting long-term progress), while TSM has a lower chance of causing more fundamental change (though at the potential expense of increasing polarization and jeopardizing incremental progress). So tell me, how are you going to get an unbiased, data-driven estimate of this key parameter that will determine the outcome of your model? I don’t think it’s possible, so don’t want to go down that rabbit hole.
Recommendation of Sunrise Movement Education Fund (TSM): Understanding how donations to organizations lead to policy change is an exercise in fundamental uncertainty, and is going to involve tough judgement calls. I understand that people could make a different judgement call on the tradeoffs with TSM and come to a different conclusion. To be honest, we’ll know a lot more over the next couple of years, as now is the time for TSM to flex its muscles and get climate on the agenda of the Biden administration (and democratic congress.) But for now, we stand by our research and think it’s a good bet. You can read our justification on the site.
A couple of specific points: it’s true that TSM’s budget has grown massively over the last few years (as has CATF’s for that matter), but I think that’s a poor proxy for neglectedness. I think that there is very little effective climate activism happening out there, and there’s huge room for effective growth.
I’m really not compelled by the “uncertainty about the sign of impact” argument, though i don’t really have a way to argue against it quantitatively since it’s theoretically possible. I would just say that this argument is lobbed at a lot of organizations, since people have different theories of political change. For instance, above I linked to an article making a similar argument about the 45Q tax credit, which is one of CATF’s big claimed accomplishments. It’s messy.
Burn Recommendation: I really think that much of the criticism is off the mark here. Berkouer and Dean (2020) focuses a lot of their analysis on credit and demand curves and other fancy economics because that’s how economists get papers published, but underpinning the paper is a strong RCT that convincingly estimates the effect of purchasing a BURN stove on fuel use. Yes, it would be nice if the sample size used for long-term follow-up was larger. And yes, this is just one study but it’s important to realize that it’s a carbon offset certification (which has a number of validation criteria) plus an RCT, which is rare and gives multiple layers of certainty. Given the difficulty of many carbon offsets, I think this is a unique level of rigor that justifies our recommendation of BURN.
The worry that purchasing offsets will not actually lead to more stoves getting distributed is more valid, as this is very hard to verify. But I’m fundamentally willing to believe that if a company like BURN gets more revenue from every stove they sell, they will sell more stoves. In other words, I think the supply curve slopes up, like it usually does.
Climeworks:
This one is a little tougher. Like Alex said, we did not take cost into account when recommending offsets, because we were just looking for any offsets that we felt offered near-certainty. And Climeworks really does offer unparalleled certainty and permanence. But yes, Climeworks is expensive (and we are up front about that on the site). In order for it to be worth it, you have to believe that direct air capture and storage of CO2 is going to be an important part of the climate solution in the future. I don’t find those Metaculus numbers Alex listed too relevant, since you are betting on the technology, not the company. But I can see how reasonable people could disagree here.