Are systems change nonprofits effective? A shallow estimate of the efficacy of nonprofit legal work
Sections
1. Intro
2. Napkin calculations
-Math (skippable)
3. Ramifications
1. INTRO
This post currently serves as a demonstration of the efficacy of the environmental law group Earthjustice and the high ceiling for systems change nonprofits. I previously argued that working for Earthjustice is likely more effective than earning to give, but was convinced otherwise by Brad’s comment and lobotomized this section. I would recommend reading this post if you think that the niche of environmental law is not one that could in theory have extremely high expected value opportunities. However, if you think that the niche certainly is high impact, but simply saturated, there is probably nothing new here.
I now have a high level of confidence that working for Earthjustice < earning to give, and suspect that funding for the organization at the margin may be ineffective compared to incubating new nonprofits in neglected fields. However, I think it is worth noting that Earthjustice is a success story and that it has had extremely high impact during its existence. This may serve as a useful prior when evaluating nonprofit law groups in novel areas.
Comparisons to earning to give and AMF in this post were done without regard for the counterfactual, so this analysis serves mainly to establish that Earthjustice existing is very good, not that further employees or funding would be beneficial.
2. NAPKIN CALCULATIONS
This analysis will be a napkin calculation in the truest sense of the phrase and aims to get within an order of magnitude of the impact of Earthjustice on a very specific measure (temperature-related deaths caused by climate change, as in, being too physically hot or cold and dying as a result). This number will be used to extrapolate the overall impact of working for Earthjustice in comparison with donating to the Against Malaria Foundation.
The relevant study in this case is a 2020 paper aiming to estimate the mortality cost per ton of carbon from the present until 2100, only with relation to temperature-related deaths. Of course, 2100 is far enough away that all predictions are subject to massive uncertainty. By that time, climate change could be made less relevant by technological innovation, social or political change, or human extinction. Applying temporal discounting in general should lead us to downweight the relationship between each metric ton of carbon in the atmosphere and utility.
However, there is an equally obvious upweighting of this relationship to be done given that the study is looking exclusively at deaths, and deaths directly due to temperature at that. I’ll use a conservative estimate that temperature-related deaths account for 10%[1] of all climate change impacts, but, given that 2100 is quite far away, divide the deaths due to each ton of carbon by five to account for temporal discounting. I have no idea if this is drastic discounting, the opposite, or par for the course, but it seems about par for the course if 3% annual discounting is standard.
The 2020 paper found that 4434 metric tons of carbon added to the atmosphere equates to one human killed by 2100. So, multiplying by five and dividing by ten, 2217 metric tons of carbon reduced will be treated as value equivalent to one human life saved in the near future, for purposes of comparison with Against Malaria Foundation, which reportedly saves one human life with each $4500 donated. In other words, reducing carbon in the atmosphere by 2217 metric tons of carbon is being treated as equivalently good to contributing $4500 to AMF.
I am now going to look at a couple of 2022 cases/”victories” on the Earthjustice site that can easily be translated into estimated carbon reduction and extrapolate the annual positive impact of Earthjustice from this value. I will be even more conservative here, estimating the impact of two victories and making the assumption that each of the other fifty-three victories listed average just 5% of the impact of the evaluated victories.
[MATH ENSUES]
Looking through the “recent wins” page of Earthjustice, the largest environmental justice employer, is a case that is estimated to have saved 970,000,000-1,800,000,000 tons of carbon by 2050. Earthjustice can’t take full credit for this—they were just part of a large legal team including city and state governments. Let’s say their expertise was responsible for 1% of the win. Taking the midpoint of the carbon estimate, 1,385,000,000 was saved, of which Earthjustice was responsible for 13,850,000.
13,850,000 / 2217 = 6,246 lives saved.
To use career-length amounts of time, twenty years donating $200,000 to AMF results in 45 * 20 = 900 lives saved. So if there were 10 lawyers on that particular case (no idea how many there were in actuality), each of them got two thirds of the way to the 20-year impact of donating $200,000/yr for malaria nets (624 lives saved per lawyer). This is pretty impressive. The case started in 2020 and concluded in 2022, so at that rate each lawyer on the case would have hit equivalence in just three years.
This may be one of the highest-impact cases Earthjustice has recently done though, so it’s not fair to assume all 200 of their lawyers have comparable impact. Another way to find expected equivalency for all Earthjustice lawyers as a whole would be to tally up the impact of all Earthjustice wins in 2022, divide that by 200 lawyers (maybe add 100 because they say they also employ supporting personnel), and see if that beats 45 lives saved per person.
I don’t have the time to try to break this down, especially because many of their wins don’t have handy carbon estimates or primarily address clean water/air or biodiversity preservation. But I can break down another high-carbon win from 2022, and see how far that gets them.
Another 2022 win: 66,160 electric mail trucks replacing 8mpg gasoline trucks over the next few years, directly as a result of suing over a plan that would have purchased 106,000 entirely gasoline trucks over that time. They also got them to guarantee that all new vehicles would be electric from 2026 onwards.
If a mail truck does a 30-mile route, and idling and driving back account for 3⁄4 of total fuel usage, then 120miles/8mph = 15 gallons of gas would have been burned. Every gallon is 8.887kg of carbon. So over a year, assuming the vehicle is driven for 320 days, that’s 42,658kg or 42.7 metric tons. Let’s say the electricity still pollutes 2⁄3 of the amount as gasoline, because in a lot of the US it’s primarily generated from fossil fuels, plus both sources of energy have transportation costs and whatnot. That’s 14.2 metric tons saved, per vehicle, per year. Let’s say the vehicles are in use for 15 years each. 66,160 vehicles * 15 years * 14.2 tons/vehicle/year = 14,092,080 metric tons saved. And that’s not even to mention the guarantee of only electric after 2026, or the fact that the electricity will probably start to come from cleaner sources.
So actually, this case is likely even bigger than the other. It exceeded the amount of claimed carbon savings by 2050 from the energy efficiency case in 2041, plus I didn’t even account for the all-electric guarantee after 2026, which probably is where a significant proportion of the value lies.
14,092,080 carbon saved / 2217 carbon per life = 6,356 lives saved. So to add the two 2022 wins I’ve looked at, that’s 6,246 + 6,356 = 12,547. Divide that by 200 lawyers to get 42 lives saved in 2022 from those victories alone, or 42⁄45 = 93.33% of the impact of each of those personnel donating $200,000.
So, those two 2022 wins alone (there were fifty-five wins listed) added up to each lawyer employee of Earthjustice donating $186,666 over the course of the year. If each of the other fifty-three wins averaged just 5% of the impact of these two wins, each lawyer at Earthjustice (assuming that the lawyers are the entire team at Earthjustice, which is not true, and this will be noted later) did equivalent good to donating $681,331 to AMF in 2022.
Notably, there are 647 employees listed on LinkedIn. So if we are taking a “Estimated Victory Impact / Number of Total Employees = Earthjustice Impact Per Employee” approach, my estimate yields $210,612 equivalent AMF donations for every single Earthjustice employee. Some Earthjustice employees do research and public awareness work with value that is likely not included in the victories listed, so using this method substantially underestimates the total impact of the organization. Crucially, the lawyers are probably the highest-value parts of the organization and even the whole-organization analysis yielded results better than $200,000/year to AMF, so it seems that any way you cut it the napkin calculations come out starkly in favor of Earthjustice over earning to give. But, again, it is worth reiterating that these calculations are probably at best an order of magnitude within the actual impact of Earthjustice, and although I am willing to bet they are a significant underestimate, others with more research experience may not be.
[MATH ENDS]
3. RAMIFICATIONS
The reason why I feel confident in Earthjustice’s efficacy despite the fact that my numbers-crunching is probably within an order of magnitude at best of the organization’s actual direct impact is the amount of value that the existence of a strong deterrent to breaking environmental laws likely provides. Without Earthjustice and similar groups, there would not be a robust system for ensuring that large companies and governments are held accountable through the legal system, and the provision of high quality lawyers and funding to such organizations likely has a huge impact both directly and through deterrence.
This leads me to think it would be worth researching Earthjustice’s funding situation to see if more funding could be translated into relatively proportionally more impact. Their 2022 Annual Report shows a $27.5mil deficit and they are currently running a fundraising campaign on their website, so there is some evidence that more funding is necessary to maintain the current level of output and/or increase it.
However, even if further research estimates that new donations to Earthjustice are only mildly better than AMF and that the organization’s funding gap can be closed relatively easily, l think that both pre- and post-fulfillment of that gap there exists an external benefit to providing explicit endorsements to strong interventions in the climate field.
I think doing so expands the appeal of EA to those who would dismiss it out of hand for not doing enough about climate change (I think many people who are smart, dedicated, and generally a good fit for EA fit this description). In turn, I think this would contribute to a movement that inspires a greater number of people to adopt aspects of EA methodology/ideas, affects more systems change than it would otherwise, and could work even more productively on global health and x-risks due to more recruiting options and more general funding/interest/clout.
I also think it improves epistemics by setting a precedent for examining specific interventions in fields that contain many duds (which, after all, is all fields).
Overall, I think that researching, endorsing, working for, and possibly donating to systems-change related nonprofits is a promising and potentially underrated route for some EAs to take in order to maximize their impact, usual qualifiers of counterfactuals, personal fit, and other routes notwithstanding.
- ^
From said article: “A Lancet report concluded that “Climate change is the biggest global health threat of the 21st century”. Yet, climate-mortality damages are currently limited in the most widely used IAMs [integrated-assessment models]. In FUND, mortality costs account for ~3% of total damages. In DICE-2016, mortality impacts are not updated to the latest scientific understanding and less than 5% of the damages come from mortality“
In looking for the referenced reports I found a bunch of broken links, so I’m not sure if these IAMs were looking at mortality solely from temperature or by other mechanisms (wildfires, storms, etc). But if they included other mechanisms, then “damages from temperature-related deaths” likely would be estimated at <3% of all impacts by these IAMs.
In order for the lawyers to be credited that utility, we need to look at the counterfactual.
Need to look at the marginal effect of adding one more attorney to the field. The number of attorney/vaiue curve is likely to be logarithmic, because the first attorneys will likely be going after the low hanging fruit in litigation. If you are truly outstanding and able to provide better expected value than an alternative in the role, there might be more value...
ETG is likely much higher value, imo, because the counterfactual of the person making lots of money is someone who likely donates less and/or ineffectively, if at all.
In fact the limiting factor may be funding for high value litigation opportunities, so maybe even in the Civil Rights battles, you may have had higher impact funding litigation and other high EV activities than direct work.
Hi Brad,
The counterfactual is definitely something that I think I should examine in more detail.
Agreed that marginal effect would be fairly logarithmic and I probably should have considered the fact that there is quite a lot of competition for employment at Earthjustice (i.e. need to be top 0.001% of lawyers to have counterfactual impact).
I am pretty completely convinced by the argument that seeking to work for Earthjustice is worse than ETG actually, so I might go and make some rather sweeping modifications to the post.
I think that the exercise does at least stand as a demonstration of the potential impact of systems change nonprofits with new/neglected focus and that Earthjustice is a success story in this realm.
Do you have a high level of confidence that Earthjustice is too large/established for it to compete with funding new and/or neglected projects?
I have moderate to high confidence that it is too diversified to be a viable big funding target, compared to the alternative of standing up our own org devoted to effective climate litigation. I surmise that the amount of its resources EJ wants to spend on (e.g.) anti-carbon litigation would be fairly insensitive to our org’s existence, because it wants to be seen as active in that area. Thus, I think there would be significantly less internal funging than for bringing EJ on as a grantee.
On the funding side, you’d need to consider that marginal increases in funding likely enables cases with lower expected impact generally, and that funding more anti-carbon work may trigger funging of non-EA resources toward other types of environmental work that might be less effective.
Hi Jason, thanks for the response.
Agree that marginal increases have lower impact. I assume GiveWell-style research on the inner workings of the organization would be needed to see if funding efficacy is actually currently comparable to AMF, and I don’t presume to have that level of know-how. I’m just hoping to bring more attention to this area.
What tools are used to assess likely funging? Is a large deficit as % of operating costs a sign that funging would be relatively low, or are most organizations that don’t have the explicit goal of continuing to scale assumed to have very high funging costs of say 50% or higher?
I think it’s more art than science. You’ve got two issues here—funging within Earthjustice (after receiving a big restricted donation, they will probably spend fewer unrestricted funds on stuff we think is high-impact) and funging by other Earthjustice donors. I think the more important factor for the former is often how much unrestricted funding was going into the activity we like.