Database of existential risk estimates

This post was writ­ten for Con­ver­gence Anal­y­sis, though the opinions ex­pressed are my own.

This post:

Key links rele­vant to ex­is­ten­tial risk estimates

  • Here’s a spread­sheet list­ing some key ex­is­ten­tial-risk-re­lated things peo­ple have es­ti­mated, with­out es­ti­mates in it.

    • The link makes a copy of the spread­sheet so that you can add your own es­ti­mates to it.

    • I men­tion this first so that you have the op­tion of pro­vid­ing some­what in­de­pen­dent es­ti­mates, be­fore look­ing at (more?) es­ti­mates from oth­ers.

    • Some dis­cus­sion of good tech­niques for fore­cast­ing, which may or may not ap­ply to such long-range and ex­treme-out­come fore­casts, can be found here, here, here, here, and here.

  • Here’s a database of all es­ti­mates of ex­is­ten­tial risks, or similarly ex­treme out­comes (e.g., re­duc­tion in the ex­pected value of the long-term fu­ture), which I’m aware of. I in­tend to add to it over time, and hope read­ers sug­gest ad­di­tions as well.

  • Here’s an EAGx talk I gave that’s ba­si­cally a bet­ter struc­tured ver­sion of this post (as I’ve now thought about this topic more), though with­out the links. So you may wish to watch that, and then just skim this post for links.

  • The ap­pendix of this ar­ti­cle by Beard et al. is where I got many of the es­ti­mates from, and it pro­vides more de­tail on the con­text and method­olo­gies of those es­ti­mates than I do in the database.

  • Beard et al. also crit­i­cally dis­cuss the var­i­ous method­olo­gies by which ex­is­ten­tial-risk-rele­vant es­ti­mates have been or could be de­rived. And Baum pro­vides ad­di­tional ex­cel­lent com­men­tary.

  • In this post, I dis­cuss some pros and cons of us­ing or stat­ing ex­plicit prob­a­bil­ities in gen­eral.

Why this database may be valuable

I’d bet that the ma­jor­ity of peo­ple read­ing this sen­tence have, at some point, seen one or more es­ti­mates of ex­tinc­tion risk by the year 2100, from one par­tic­u­lar source.[1] Th­ese es­ti­mates may in fact have played a role in ma­jor de­ci­sions of yours; I be­lieve they played a role in my own ca­reer tran­si­tion. That source is an in­for­mal sur­vey of global catas­trophic risk re­searchers, from 2008.

As Millett and Sny­der-Beat­tie note:

The dis­ad­van­tage [of that sur­vey] is that the es­ti­mates were likely highly sub­jec­tive and un­re­li­able, es­pe­cially as the sur­vey did not ac­count for re­sponse bias, and the re­spon­dents were not cal­ibrated be­fore­hand.

Ad­di­tion­ally, in any case, it was just one in­for­mal sur­vey, and is now 12 years old.[2] So why is it so fre­quently refer­enced? And why has it plau­si­bly (in my view) in­fluenced so many peo­ple?

I also ex­pect that es­sen­tially the same pat­tern is likely to re­peat, per­haps for an­other dozen years, but now with Toby Ord’s re­cent ex­is­ten­tial risk es­ti­mates. Why do I ex­pect this?

I origi­nally thought the an­swer to each of these ques­tions was es­sen­tially that we have so lit­tle else to go on, and the topic is so im­por­tant. It seemed to me there had just been so few at­tempts to ac­tu­ally es­ti­mate ex­is­ten­tial risks or similarly ex­treme out­comes (e.g., ex­tinc­tion risk, re­duc­tion in the ex­pected value of the long-term fu­ture).

I’d ar­gue that this causes two prob­lems:

  1. We have less in­for­ma­tion to in­form de­ci­sions such as whether to pri­ori­tise longter­mism over other cause ar­eas, whether to pri­ori­tise ex­is­ten­tial risk re­duc­tion over other longter­mist strate­gies, and es­pe­cially which ex­is­ten­tial risks to be most con­cerned about.

  2. We may an­chor too strongly on the very sparse set of es­ti­mates we are aware of, and get caught in in­for­ma­tion cas­cades.

In­deed, it seems to me prob­a­bly not ideal how many strate­gic de­ci­sions peo­ple con­cerned about ex­is­ten­tial risks (my­self in­cluded) have made so far with­out hav­ing first col­lected and cri­tiqued a wide ar­ray of such es­ti­mates.

This sec­ond is­sue seems all the more con­cern­ing given the many rea­sons we have for skep­ti­cism about es­ti­mates on these mat­ters, such as:

  • The lack of di­rectly rele­vant data (e.g., prior hu­man ex­tinc­tion events) to in­form the estimates

  • Es­ti­mates of­ten be­ing lit­tle more than quick guesses

  • It of­ten be­ing hard to in­ter­pret what is ac­tu­ally be­ing estimated

  • The pos­si­bil­ity of re­sponse bias

  • The es­ti­ma­tors typ­i­cally be­ing peo­ple who are es­pe­cially con­cerned about ex­is­ten­tial risks, and thus ar­guably be­ing es­sen­tially “se­lected for” above av­er­age pessimism

  • There’s limited ev­i­dence about how trust­wor­thy long-range fore­casts in gen­eral are, and some ev­i­dence of ex­pert fore­casts of­ten be­ing un­re­li­able even for events just a few years out (e.g., from Tet­lock)

Thus, we may be an­chor­ing on a small hand­ful of es­ti­mates which could in fact war­rant lit­tle trust.

But what else are we to do? One op­tion would be to each form our own, in­de­pen­dent es­ti­mates. This may be ideal, which is why I opened by link­ing to a spread­sheet you can du­pli­cate to do that in.

But it seems plau­si­ble that “ex­pert” es­ti­mates of these long-range, ex­treme out­comes would cor­re­late bet­ter with what the fu­ture ac­tu­ally has in store for us than the es­ti­mates most of us could come up with, at least with­out in­vest­ing a great deal of time into it. (Though I’m cer­tainly not sure of that, and I don’t know of any di­rectly rele­vant ev­i­dence. Plus, Tet­lock’s work may provide some rea­son to doubt the idea that “ex­perts” can be as­sumed to make bet­ter fore­casts in gen­eral.)

And in any case, most of us have prob­a­bly already been ex­posed to a hand­ful of rele­vant es­ti­mates, mak­ing it harder to gen­er­ate “in­de­pen­dent” es­ti­mates of our own.

So I de­cided to col­lect all quan­ti­ta­tive es­ti­mates I could find of ex­is­ten­tial risks or other similarly “ex­treme out­comes”, as this might:

  1. Be use­ful for de­ci­sion-mak­ing, be­cause I do think these es­ti­mates are likely bet­ter than noth­ing or than just phrases like “plau­si­ble”, plau­si­bly by a very large mar­gin.

  2. Re­duce a rel­a­tively “blind” re­li­ance on any par­tic­u­lar source of es­ti­mates, and per­haps even make it eas­ier for peo­ple to pro­duce effec­tively some­what in­de­pen­dent es­ti­mates. My hope is that pro­vid­ing a va­ri­ety of differ­ent es­ti­mates will throw an­chors in many di­rec­tions (so to speak) in a way that some­what “can­cels out”, and that it’ll also high­light the oc­ca­sion­ally ma­jor dis­crep­an­cies be­tween differ­ent es­ti­mates. I do not want you to take these es­ti­mates as gospel.

Above, I said “I origi­nally thought the an­swer to each of these ques­tions was es­sen­tially that we have so lit­tle else to go on, and the topic is so im­por­tant.” I still think that’s not far off the truth. But since start­ing this database, I dis­cov­ered Beard et al.’s ap­pendix, which has a va­ri­ety of other rele­vant es­ti­mates. And I won­dered whether that meant this database wouldn’t be use­ful.

But then I re­al­ised that many or most of the rele­vant es­ti­mates from that ap­pendix weren’t men­tioned in Ord’s book. And it’s still the case that I’ve seen the 2008 sur­vey es­ti­mates many times, and never seen most of the es­ti­mates in Beard et al.’s ap­pendix (see this post for an in­di­ca­tion that I’m not alone in that). And Beard et al. don’t men­tion Ord’s es­ti­mates (per­haps be­cause it was re­leased only a cou­ple months be­fore Ord’s book was), nor a va­ri­ety of other es­ti­mates I found (per­haps be­cause they’re from “in­for­mal sources” like 80,000 Hours ar­ti­cles).

So I still think col­lect­ing all rele­vant es­ti­mates in one place may be use­ful. And an­other benefit is that this database can be up­dated over time, in­clud­ing via sug­ges­tions from read­ers, so it can hope­fully be­come com­pre­hen­sive and stay up to date.

So please feel free to:

  • Make your own estimates

  • Take a look through the database as and when you’d find that useful

  • Com­ment in there re­gard­ing es­ti­mates or de­tails that could be added, cor­rec­tions if I made any mis­takes, etc.

  • Com­ment here re­gard­ing whether my ra­tio­nale for this database makes sense, re­gard­ing pros, cons, and best prac­tice for ex­is­ten­tial risk es­ti­mates in gen­eral, etc.

This post was sort-of in­spired by one sen­tence in a com­ment by MichaelStJules, so my thanks to him for that. My thanks also to David Kristoffers­son for helpful feed­back and sug­ges­tions on this post, and to Ofer Givoli, Juan Gar­cía Martínez, and Michael Dick­ens for com­ments or sug­ges­tions on the database it­self.

See also my thoughts on Toby Ord’s ex­is­ten­tial risk es­ti­mates, which are fea­tured in the database.

  1. For ex­am­ple, it’s referred to in these three 80,000 Hours ar­ti­cles. ↩︎

  2. Ad­di­tion­ally, Beard et al.’s ap­pendix says the sur­vey had only 13 par­ti­ci­pants. The origi­nal re­port of the sur­vey’s re­sults doesn’t men­tion the num­ber of par­ti­ci­pants. ↩︎