Note that Severin is a coauthor on this post, though I haven’t been able to find a way to add his EA Forum account on a crosspost from LessWrong.
StevenKaas
Karma: 418
We tried to write a related answer on Stampy’s AI Safety Info:
How could a superintelligent AI use the internet to take over the physical world?
We’re interested in any feedback on improving it, since this is a question a lot of people ask. For example, are there major gaps in the argument that could be addressed without giving useful information to bad actors?
Thanks for reporting the broken links. It looks like a problem with the way Stampy is importing the LessWrong tag. Until the Stampy page is fixed, following the links from LessWrong should work.
There’s an article on Stampy’s AI Safety Info that discusses the differences between FOOM and some other related concepts. FOOM seems to be used synonymously with “hard takeoff” or perhaps with “hard takeoff driven by recursive self-improvement”; I don’t think it has a technical definition separate from that. At the time of the FOOM debate, it was taken more for granted that a hard takeoff would involve recursive self-improvement, whereas now there seems to be more emphasis by MIRI people on the possibility that ordinary “other-improvement” (scaling up and improving AI systems) could result in large performance leaps before recursive self-improvement became important.
OK, thanks for the link. People can now use this form instead and I’ve edited the post to point at it.
Like you say, people who are interested in AI existential risk tend to be secular/atheists, which makes them uninterested in these questions. Conversely, people who see religion as an important part of their lives tend not to be interested in AI safety or technological futurism in general. I think people have been averse to mixing AI existential ideas with religious ideas, for both epistemic reasons (worries that predictions and concepts would start being driven by meaning-making motives) and reputational reasons (worries that it would become easier for critics to dismiss the predictions and concepts as being driven by meaning-making motives).
(I’m happy to be asked questions, but just so people don’t get the wrong idea, the general intent of the thread is for questions to be answerable by whoever feels like answering them.)
Thank you! I linked this from the post (last bullet point under “guidelines for questioners”). Let me know if you’d prefer that I change or remove that.
As I understand it, overestimation of sensitivity tails has been understood for a long time, arguably longer than EA has existed, and sources like Wagner & Weitzman were knowably inaccurate even when they were published. Also, as I understand it, although it has gotten more so over time, RCP8.5 has been considered to be much worse than the expected no-policy outcome since the beginning despite often being presented as the expected no-policy outcome. It seems to me that referring to most of the information presented by this post as “news” fails to adequately blame the EA movement and others for not having looked below the surface earlier.
What does an eventual warming of six degrees imply for the amount of warming that will take place in (as opposed to due to emissions in), say, the next century? The amount of global catastrophic risk seems like it depends more on whether warming outpaces humanity’s ability to adapt than on how long warming continues.
I was thinking e.g. of Nordhaus’s result that a modest amount of mitigation is optimal. He’s often criticized for his assumptions about discount rate and extreme scenarios, but neither of those is causing the difference in estimates here.
According to your link, recent famines have killed about 1M per decade, so for climate change to kill 1-5M per year through famine, it would have to increase the problem by a factor of 10-50 despite advancing technology and increasing wealth. That seems clearly wrong as a central estimate. The spreadsheet based on the WHO report says 85k-95k additional deaths due to undernutrition, though as you mention, there are limitations to this estimate. (And I guess famine deaths are just a small subset of undernutrition deaths?) Halstead also discusses this issue under “crops”.
I think the upper end of Halstead’s <1%-3.5% x-risk estimate is implausible for a few reasons:
1. As his paper notes and his climate x-risk writeup further discusses, extreme change would probably happen gradually instead of abruptly.
2. As his paper also notes, there’s a case that issues with priors and multiple lines of evidence imply the tails of equilibrium climate sensitivity are much less fat than those used by Weitzman. As I understand it, ECS > 10 would imply paleoclimate estimates are highly misleading and estimates based on the instrumental record are highly misleading and climate models are highly misleading. I don’t know how this sort of reasoning relates to Earth system feedbacks, but I guess the thresholds for them to become relevant would be less likely to be crossed.
3. Even if some of it were abrupt, a 10 degree rise would probably not be an existential disaster in the strict sense, though it would be horrible. (On the other hand, maybe a less than 10 degree rise would still have some risk of causing an existential disaster through some indirect effect on the stability of civilization.)
4. All estimates of the chance that a particular development will cause an existential disaster have to account for the possibility that some other development will have caused an existential disaster by that time and the possibility that some other development will have made humanity mostly immune to existential disasters.
Ah, it looks like I was myself confused by the “deaths/year” in line 20 and onward of the original, which represent an increase per year in the number of additional deaths per year. My apologies. At this point I don’t understand the GWWC article’s reasoning for not multiplying by years an additional time.
My prior was that, since economists argue over the relative value of mitigation (at least beyond low hanging fruit) and present consumption, and present consumption isn’t remotely competitive with global health interventions, a calculation that shows mitigation to be competitive with global health interventions is likely to be wrong. But after looking it over another time, I now think that’s accounted for mostly by:
1. The assumption that climate change increases all causes of death by the same percentage as the causes of death investigated here, which, as the article notes, seems very pessimistic. If 57 million people worldwide died in 2016 (and population is increasing but death rate is decreasing), then 5 million additional deaths per year in 2030-2050 seems implausibly large: almost one in ten deaths would be due to climate change.
2. Cool Earth being estimated here to be orders of magnitude more efficient than the kinds of mitigation that economists usually study. (I have no opinion on whether this is accurate.)
(edit: I no longer endorse this comment)
We don’t expect to be able to recapture most emitted CO2, so a very conservative value to use would be to attribute 50 years of increased deaths to each emission. Hence, this increases the estimate of lives saved by a factor of 50x.
This seems to be the key disagreement between your estimate and GWWC’s. As I understand it, if we reduce emissions for the year X by 1%, different things happen in the two calculations:
In GWWC’s calculation, every year Y for decades, we prevent 1% of the deaths during the year Y that would have been prevented by a delay of all climate change for one year (corresponding to the year X)
In your calculation, every year for decades, we prevent 1% of the deaths that would have been caused by climate change during the year Y
There are two “per year”s at play, “per year of deaths” and “per year of emissions”, and the “per year of deaths” is canceled out by “years of deaths”, leaving only the “per year of emissions”. GWWC treats a one-year-long stop to all emissions (in the present) as equivalent to a delay of warming by one year (in the future). I don’t quite understand why that is, but the units seem right. So if I’m not mistaken, you were understandably confused by the numbers being implicitly “per year per year” rather than just “per year”, and the factor 50 shouldn’t be there.
edit: To be more concrete, if you’re multiplying by 50 years in cell C44 of the updated sheet, then cell C34 should do something like divide the averted emissions by the total emissions over decades rather than by the emissions for just the year 2016.
A piece such as this should engage with the direct cost/benefit calculations that have been done by economists and EAs (e.g. Giving What We Can), which make it seem hard to argue that climate change is competitive with global health as a cause area.
How much it would take to stay under a mostly arbitrary probability of a mostly arbitrary level of temperature change is a less relevant statistic than how much future temperatures would change in response to reduced emissions.
My nonconfident best guess at an interpretation is that, according to these estimates, for every tonne of carbon:
Future Indians suffer damages utility-equivalent to the present population of India paying a total of $76
Future Americans suffer damages utility-equivalent to the present population of the USA paying a total of $48
Future Saudis suffer damages utility-equivalent to the present population of Saudi Arabia paying a total of $47
Next are China, Brazil, and the UAE, all with $24, and then a lot of other countries, and the sum of all these numbers is $417. So it’s as if the $417 is paid by this particular mix of the world’s people, making it iii), something in between. These numbers are totals that don’t divide by population, so an individual inhabitant of Saudi Arabia or the UAE pays a greater absolute amount than an individual American, who pays a greater absolute amount (but a smaller percentage of income) than an individual Indian.
I was about to say this and then saw your comment. My impression from the paper is the $417 is a sum of costs to different countries, and for each of them the cost is a present value to the people in that country, with discounting being applied based on the expected amount of economic growth in that country. So I don’t think it’s calibrated to present-day Americans, but I don’t think it’s calibrated to the world’s poorest either, and I agree the argument doesn’t go through.
There’s another problem with the quoted claim, which becomes clear if you pick a value like X = 1/1000. Paying $417,000 to avert a tonne of carbon is a huge net bad and not just a much smaller net good.
It seems to me another problem is that if the social cost of carbon comes from effects on growth, you have to compare that to the effects on growth of cash transfers. It’s generally easy for small changes in growth rate to outweigh small changes to level in the long run, so if you compare the growth effects of one intervention to the level effects of another intervention, it’s no surprise that the former would seem more effective.
A nuclear exchange may have the potential to … possibly lead to the extinction of life on Earth.
I haven’t seen anyone seriously argue for this claim and I don’t think it’s true or true-adjacent.
3. The goal for climate change mitigation should be getting to net zero emissions as fast as possible, as anything other than that still causes warming, and this goal is absent from many EA and the 80,000 Hours write-up.
If there’s already the goal of reducing emissions in general, with more reduction being better, is there any reason to add a goal about the zero level specifically? EA generally (and I think rightly) just cares about the expected amount of problem reduction, with exceptions where zero matters being things like diseases that can bounce back from a small number of cases.
What is wrong with it?
If the claims made here from p.13 on are true, it seems like the model can’t be reliable. This also disagrees. In general, it seems intuitively like it would be extremely hard to do this kind of statistics and extrapolate to the future with any serious confidence or rely on it for an estimate without a lot more thought. (I haven’t tried to look for critiques of the critiques and don’t claim to have a rigorous argument.)
Economic activity already goes to wherever it will be the most profitable. I don’t see why we would expect companies to predictably err.
I was thinking if climate has effects on growth rate, companies may not be capturing the full costs/benefits from that. My intuition that it could be extremely effective was something like “if an extremely blunt tool like global average temperature can have big effects on growth through improving local temperature in more places than it worsens local temperature, you can probably get much bigger effects by optimizing local temperature in a fine grained way through changing the locations of things.” Maybe that’s wrong, I don’t know.
Since somebody was wondering if it’s still possible to participate without having signed up through alignmentjam.com:
Yes, people are definitely still welcome to participate today and tomorrow, and are invited to head over to Discord to get up to speed.