Also, my wild guess is that if the existential risk intervention came out as cost effective for the present generation, then it may pass your test even with continued exponential growth in utility.
Sorry—was writing too late at night for me—I got confused.
Sorry—was writing too late at night for me—I got confused.
Thanks for the update. I was not aware that people could apply for EA funds. The FAQ seems to be just for donors, and I hadn’t seen anything on the EA forum about applying.
Again I want to stress that the point here is not to debate these numbers (I was told that a decrease of the extinction risk of 0.1 percentage point for an investment of 0.1% of the world GDP was reasonable, but found it difficult to find references; I would appreciate comments pointing to relevant references).
If I understand you correctly, this is a one time expenditure, so we are talking about ~$80 billion. This is a model that considered $3 billion being spent on AGI safety. It was a marginal analysis, but I think many would agree that it would address a large fraction of the AGI risk, which is a large fraction of the total existential risk. So if it reduced existential risk overall by one percentage point, that would be 2.5 orders of magnitude more cost-effective than you have assumed, which is much better than your growth assumption. Investment into nuclear winter resilience has similar or even better returns. So I think we could be spending a lot more money on existential risk mitigation that would still be no regrets even with continued exponential growth of utility.
But, to give some substance to it, if we very crudely conflate our utility function with world GDP, then I think it is reasonable to place a return of at least a factor of 10 on some of the better growth investments.
If I understand you correctly, this one-time investment of 0.1% of GDP increases the GDP by 1% above the business as usual for all time. So if you look over one century without discounting, that looks like you have gotten a benefit to cost ratio of 1000. I think there has been discussion about how we have increased our R&D dramatically over the past few decades, but GDP growth has not increased. So maybe someone can jump in with the marginal returns for R&D. Or maybe you had something else in mind?
Incidentally, this would go some way into mitigating Fermi’s paradox: maybe other advanced civilizations have not come to visit us because they are mostly busy optimizing their surrounding environment, and don’t care all that much about colonizing space.
This sounds like an argument in the Age of Em, that once we accelerate our thought processes, expanding into space would be too painfully slow.
As per my comment on HaukeHillebrandt’s comment below—The trouble with these estimates is that I’m not convinced they do a good job of considering how costs change as a technology is scaled. For example, we’ve seen this with solar—http://solarsouthwest.co.uk/solar-panel-cost/. Do you have a recommended source which does somehow take account of these effects? If not, we’re not really comparing costs properly.
Also, I want to specifically comment on this. Unless you believe that very large scale CO2 air-capture is going to be economically/technologically/land-use viable, we don’t have time to wait for people to get richer. The CO2 being emitted today, is committing humanity to a particular temperature rise for centuries to come. The cheapest time to deal with that is right now, to avoid putting the CO2 in the atmosphere in the first place.
It is true we generally see reduction in costs as cumulative production increases (this is called learning in economics). But then this means it might be cheaper to reduce CO2 emissions in the future (at least at the margin for EA, and even for the world as a whole if some of the learning occurs in related fields that does not require spending money on CO2 mitigation now). It is possible that renewable energy will become less expensive than fossil fuels in the near future, though usually the comparison is made with fossil fuel electricity. It is much more difficult for renewable energy to be lower cost than fuels used directly. Furthermore, if we want to go back to 350 PPM, we would need to do some form of air capture, which I think will be expensive for quite a while. So overall, with learning, it would reduce the cost of solving the problem, but I think it is harder to imagine it being less than $1 trillion present value with low discounting. You are right that there is a trade off. If we spend money on saving lives at $3000 per life now with health interventions instead of reducing CO2 emissions, that means more CO2 in the atmosphere in 100 years. So the question is whether that harm to the relatively richer people in 100 years is greater than the harm you avert by spending money on global health now if your time horizon only extends about 100 years.
The final thing which makes this all more complex, is that climate change is something which we are on a very well defined trajectory towards—where inaction results in terrible consequences. However, things like nuclear war are risks which may never materialize. If we invest effort into averting credible but potential risks, we’ll never be sure whether that investment actually mattered. If we invest effort in averting climate change, we’ll be much more sure that the effort was worthwhile.
Full-scale nuclear war may very well not happen this century. However, when you include additional catastrophes such as extreme weather on multiple continents (which a UK government study estimated had an ~80% likelihood this century), regional nuclear war, etc., it appears to be more likely than not that we will have one of these catastrophes this century. But it is possible that we will not have one of these catastrophes. As I said in my 80,000 Hours interview, if you are someone who has paid for insurance where they have gotten no payout from it whether they have wasted their money, they say “no” because it makes sense to insure things we can’t afford. So I think of this as an insurance policy for the world. And actually in terms of probabilities, I would say one of these agricultural catastrophes is actually more likely than median or worse slow climate change, so the probability of the investment paying off is actually higher for alternate food preparedness.
Thanks—I wasn’t sure it was OK to reproduce it.
Thanks very much for the thoughtful engagement.
I don’t actually see a detailed calculation of human impacts in that paper. I agree that full extinction seems unlikely, but hugely catastrophic impacts seem very plausible.
Correct—the estimate of long term future impact was from the survey cited here.
Also, a temperature decrease is definitely not guaranteed to have a symmetric impact with a temperature increase, so the comparison doesn’t seem entirely valid.
I agree that it is not necessarily symmetric-cooling is generally worse for plants than warming. Also, we would have roughly 50% reduction in solar radiation in the nuclear winter case, which further makes it worse. Furthermore, I believe more people die from the cold than from the heat.
This is a statement which quickly points out a difference in our ethics. 10C of warming would likely require the evacuation of huge areas of land around the equator. That’s not the same as extinction, but I still consider it to be a deeply unacceptable outcome. Survival alone isn’t good enough for me. I’m not sure how to formalize this viewpoint within utilitarian calculations.
I agree that 10°C warming over a century would be bad. But would you agree that 8°C cooling and 50% reduction in solar radiation in one year would be worse?
It is important to note that 10°C global warming would have less than 10°C warming near the equator. This says that plants were doing pretty well near the equator when the Earth was about 12°C warmer. But as I note below, evolution takes a long time, so species would need to be relocated. As for humans, I’m not sure that a lot of relocation would be required. It is true that humans would not be able to go outside very long without technology. But I would say that is true at 40° latitude in the winter now. It is true that the technology of an insulating coat is fairly simple. But if it is hot, we could use the fairly simple technology of an ice vest like this. More complicated technology could involve a system which burns fuel and then uses absorption chilling to cool the body if one needed to stay cool for many hours. Of course this technology would not be affordable by many people in the tropics now, but 100 years out, I think the situation will be different.
Also, less abstractly, I’m not confident that the natural ecosystem which we rely on would be able to adapt to 10C in warming over a century. This suggests to me that we would see a huge amount of species being pushed into extinction, and again I consider this to be an extremely negative consequence, even if we are able to figure out ways to feed ourselves from a limited number of crops that we still manage to cultivate.
Though life was doing well when the earth was 10°C warmer than now, it is true that a change over 100 years is very short considering evolutionary timescales. In order to avoid a mass extinction, I think humans would need to help relocate plants (and indeed people are working on this now) (animals can generally move quickly enough, though still might need help in some circumstances). Also, relocation would not save all species, because some would no longer have a habitat cool enough, and could only be saved with captivity. I personally think it is unlikely that humans would allow the climate to warm 10°C, and instead would do solar radiation management. However, solar radiation management has its own risks, such as a double catastrophe where some other catastrophe hits us and then we are not able to maintain solar radiation management, and then we have two problems. I think it is possible to have a more robust solar radiation management to avoid this problem. But it is possible we do not use solar radiation management. Overall, I agree that there is intrinsic value in biodiversity, but that would require another discussion to find the most cost-effective ways of preserving it.
It seems to me like a huge leap of faith is required to believe that the global impact of 10C of warming (over a century) is on the same order of magnitude as an abrupt 10% agricultural shortfall. You’d need to lay out much more of an argument for me to believe that. As it stands, I think you are either predicting a much lower impact from climate change than I am, or putting more faith in technological/economic growth to mitigate the impacts. In either case, it’s clear we disagree.
I am focusing my analysis on the impact on the long term future, which means the reduction in the long term potential of humanity (out thousands or millions of years). But I think it is a reasonable proxy to look at the mortality. In this paper, I estimated the expected mortality of an abrupt 10% food shortfall from something like India Pakistan nuclear war was about 500 million. Technically speaking, adaptation and relocation in response to a century long 10°C rise should involve the loss of many fewer lives, but it could go very badly, even up to including full-scale nuclear war, which could kill billions of people. So I think it is in the same order of magnitude in expectation as an abrupt 10% food shortfall. What would your estimate be of the expected mortality a century long 10°C rise?
This only considers the impact of 10C of warming. If we don’t have 10C of warming, we are still going to have an amount less than that. There’s a currently a >25% of >4C of warming, without fully accounting for climate tipping points. 4C of warming is already expected to have serious consequences. However, these haven’t yet been well quantified because even 4C of warming has impacts on so many aspects of the world society, economy, and ecology, that it’s incredibly difficult to model.
It is true that this analysis is not taking into account the smaller warmings, but these are less likely to have an impact on the long-term future, so I think they are unlikely to change the order of magnitude of the result. I am concerned about a possible tipping point that would be a runaway greenhouse effect. But since the earth was about 14°C warmer about 50 million years ago, and the sun’s radiation is not that much higher than it was then, I think we only have to start worrying about this at over ~10°C warming.
This presumes that mitigating climate change through alternative foods is a morally acceptable outcome. As per my statement above, for me, it isn’t.
I agree it would be better to prevent climate change rather than just mitigate some of the impacts if we had unlimited resources. But EA is about prioritizing-we have limited resources and we want to make the largest impact possible. We have other competing demands I have not talked about, such as reducing the risk from natural and engineered pandemics. I do hope that we can bring many more resources to EA and then we can work our way further down the existential risk mitigation curve to climate emissions mitigation.
This statement actually very neatly encapsulates my main objection to long-termism. It feels very much like a case of Pascal’s Mugging. Over a long enough view of the potential future of humanity (e.g. 10s of millions of years), none of today’s problems really matter.
I don’t think the comparison I am making is Pascal’s Mugging. I think Pascal’s Mugging could be considering the immense potential value of the long-term future and then demanding some sacrifice now. However, since I am looking at the reduction in the long-term future due to climate change and due to nuclear winter, they are on equal footing and do not depend on the precise value of the long-term future.
As for prioritizing the present generation, my analysis indicates that prioritizing current global poverty is a couple orders of magnitude more effective than reducing emissions at carbon costs required to solve the whole problem (largely because the current poor will likely be richer when the main climate change impacts hit). However, if you believe the Cool Earth numbers (not counting opportunity costs of the value of the land for farming) and if you don’t think they will be taken by someone else, then it could be competitive. However, I think alternative foods are even better from the present generation perspective.
Can you clarify what exactly you mean by “from the perspective of the long-term future”? What time-horizon do you have in mind, and what kind of discount rate are you applying, if any?
I think the only discount rate I would apply would be that due to existential risk. So it is something like maximizing the total expected utility of sentient beings as long as we can keep them going. But other long-termists would say it differently (different ethical theories than consequentialism could still result in highly valuing the longterm future). Also note that even if one has a few percent discount rate, if one puts a non-negligible probability mass on some sort of technological singularity within a century or two with the potential for a huge number of computer consciousnesses, that is another way of getting at the overwhelming importance of making it through to the longterm.
I think my comment here applies to this charity as well: generally where trees grow, you can farm, and the opportunity cost of not farming makes the total $/ton much higher than the obvious cost of planting trees.
Would CEA be willing to accept donations and route them to EA hotel so the donors in the US or UK can get the tax advantage?
This appears to be very a different conclusion from the survey where most people said that future generations matter just as much as the present (no requirement for utopia).
Nuclear winter would be approximately 8°C change in only one year, and this is unlikely to cause extinction. 10°C climate warming over a century would be much lower impact, because there is time to relocate infrastructure and people (and nuclear winter also reduces solar radiation). So I have put it in the intensity category of an abrupt 10% agricultural shortfall. Based on a survey of GCR researchers, this has a mean long-term reduction in far future potential of approximately 5%. This combined with a probability of about 2% gives about a 0.1% reduction in the far future potential. Full scale nuclear war is estimated to have a 17% reduction in long term future potential. There is great uncertainty in the probability of full-scale nuclear war, but I think 0.1% per year or 10% in the next 100 years is reasonably conservative.* Therefore, full scale nuclear war is more likely than extreme climate change and also significantly greater consequences if it were to happen. But then the question is how much would it cost to significantly mitigate the problems. Since solar radiation management is risky, the present value of the cost of largely solving the climate change problem by reducing emissions is around $10 trillion (there was an EA forum post on value of information of this, but I can’t seem to find it). I have researched both energy efficiency and renewable energy for years, and I do think there is still some low hanging fruit of energy efficiency that pays for itself. However, to actually solve the problem will cost a lot of money. On the other hand, reducing the far future impact of nuclear winter by about 17% would cost around $100 million by investing in response plans and research and development of alternative foods. Therefore, since alternative foods address a roughly 15 times bigger problem, at 100,000 times lower cost and with 1⁄5 the threat reduction (if we assume the $10 trillion on emissions reductions completely solves the problem), this works out to approximately 300,000 times higher cost effectiveness for alternative foods versus emissions reductions.
Fortunately, alternative foods also mitigates climate related catastrophes such as abrupt regional climate change, coincident extreme weather on multiple continents, and slow 10°C change (which makes the cost effectiveness of alternative foods even higher than the numbers calculated above). There may be other low hanging fruit that address climate change such as Cool Earth (though see this criticism) and energy efficiency (though even if energy efficiency pays for itself, it still costs donor money to advocate for it). But even at a cost of $0.38 per ton CO2, it is still a few orders of magnitude lower cost effectiveness than alternative foods or artificial general intelligence safety from the perspective of the long-term future. Of course it is better to do this probabilistically, which is why I have encouraged you to add climate change to an existing cost-effectiveness model of alternative foods and artificial intelligence.
Hopefully we can direct tens of billions of dollars more to EA, and then we can work our way further down the marginal cost effectiveness curves of existential risk mitigation, but I don’t think that reducing greenhouse gas emissions should be a priority for EA at this point.
*For the alternative food analysis, we only used at few decades effective time horizon but higher probability of nuclear war from here.
I think the EA community currently has a limited amount to say to anyone with power.
More broadly, CSER has these recommendations for governments for global catastrophic risks.
I think the EA community currently has a limited amount to say to anyone with power.
I think it would be useful for governments to have response plans for agricultural catastrophes such as nuclear and volcanic winter, and also for electricity/industry disrupting catastrophes including solar storms and high-altitude electromagnetic pulses (HEMPs). Governments could also fund research related resilience including alternative foods and backup communications systems.
Unfortunately I’m not familiar with that literature, but others feel free to jump in!
Very interesting. It sounds similar to Bolder Giving but more organized and focused and you don’t actually have to give 50%.
Another advantage of increased trade is greater economic interdependence, which I think reduces the probability of conflict. If that conflict were to manifest itself as nuclear war, this could have catastrophic consequences, plausibly reducing the long-term potential of humanity.
In addition to direct cost-effectiveness calculations for the present generation as StevenKaas recommends, I would recommend direct cost-effectiveness calculations for the long-term future. Here is an example where I compare AI and alternative foods to address agricultural catastrophes. It would not take very much work to use that framework for conventional emissions reductions for climate change. However, as others have pointed out, because emissions reductions are so expensive, they are unlikely to be competitive cost-effectiveness. Solar radiation management (SRM) (as opposed to non SRM geoengineering techniques such as CO2 air capture) has the potential of being much more cost effective, but it has its own risks, such as double catastrophe.
I like the big thinking! I agree that there are many tens of billions of dollars we could spend as we work our way down the marginal cost effectiveness curve of existential risk mitigation. Some other things to include are biosecurity interventions, preventing supervolcanic eruptions, comet detection and deflection (much more expensive than asteroid detection and deflection).