MichaelA

• MichaelA 12 Aug 2020 0:12 UTC

  2 points

  on: Pro­pose and vote on po­ten­tial tags

  Mo­ral Uncertainty

  Ar­gu­ment against:

  • Ar­guably a sub­set of Mo­ral Philosophy

  • Over­laps with Meta-Ethics

  Ar­gu­ment for:

  • Ar­guably an im­por­tant sub­set of Mo­ral Philosophy

  • I’d es­ti­mate there’s at least 10 posts on the topic

• MichaelA 11 Aug 2020 23:41 UTC

  2 points

  on: MichaelA’s Shortform

  The old de­bate over “giv­ing now vs later” is now some­times phrased as a de­bate about “pa­tient philan­thropy”. 80,000 Hours re­cently wrote a post us­ing the term “pa­tient longter­mism”, which seems in­tended to:

  • fo­cus only on how the de­bate over pa­tient philan­thropy ap­plies to longtermists

  • gen­er­al­ise the de­bate to also in­clude ques­tions about work (e.g., should I do a di­rectly use­ful job now, or build ca­reer cap­i­tal and do di­rectly use­ful work later?)

  They con­trast this against the term “ur­gent longter­mism”, to de­scribe the view that favours do­ing more dona­tions and work sooner.

  I think the terms “pa­tient longter­mism” and “ur­gent longter­mism” are both use­ful. One rea­son I think “ur­gent longter­mism” is use­ful is that it doesn’t sound pe­jo­ra­tive, whereas “im­pa­tient longter­mism” would.

  I sug­gest we also use three ad­di­tional terms:

  1. Pa­tient altruism

     1. Like “pa­tient philan­thropy” and un­like “pa­tient longter­mism”, this term is cause-neu­tral.

     2. But like “pa­tient longter­mism” and un­like “pa­tient philan­thropy”, this term clearly re­lates to both work and dona­tions, not merely to dona­tions.

        1. Dis­cus­sions about “pa­tient philan­thropy” do of­ten make some refer­ence to op­ti­mal timing of work, but it’s not usu­ally cen­tral. Also, the term “philan­thropy” is typ­i­cally used just for dona­tions.

  2. Ur­gent altruism

     1. Again, this is partly to avoid nega­tive con­no­ta­tions, as is my next sug­ges­tion.

  3. Ur­gent philanthropy

• MichaelA 11 Aug 2020 23:22 UTC

  2 points

  in reply to: JP Addison’s comment on: Pro­pose and vote on po­ten­tial tags

  Thanks for high­light­ing pa­tient philan­thropy as an op­tion, and good point that it’d be good for this tag to not just be about the de­bate but also how to im­ple­ment the pa­tient ap­proach.

  I’ve now made this tag, though with the name Pa­tient Altru­ism. I haven’t heard that term used, but it makes sense to me as a gen­er­al­i­sa­tion of pa­tient philan­thropy to also ac­count for how to use work, not just how to use dona­tions. I’ve now also writ­ten a short­form post ar­gu­ing for the term.

  One worry I have is that by say­ing Pa­tient Altru­ism rather than Pa­tient vs Ur­gent Altru­ism, this tag puts vir­tu­ous con­no­ta­tions on one side but not the other. But the ver­sion with “vs Ur­gent” is longer, it per­haps doesn’t as nat­u­rally in­clude posts about how to take the pa­tient ap­proach, and I’ve only heard the term “ur­gent longter­mism”, not “ur­gent philan­thropy” (though I do sug­gest use of the terms “ur­gent philan­thropy” and “ur­gent al­tru­ism” in that short­form post).

• MichaelA 11 Aug 2020 11:53 UTC

  5 points

  on: A New X-Risk Fac­tor: Brain-Com­puter Interfaces

  I haven’t had a chance to read this post yet, but just wanted to men­tion one pa­per I know of that does dis­cuss brain-com­puter in­ter­faces in the con­text of global catas­trophic risks, which there­fore might be in­ter­est­ing to you or other read­ers. (The pa­per doesn’t use the term ex­is­ten­tial risk, but I think the ba­sic points could be ex­trap­o­lated to them.)

  The pa­per is Assess­ing the Risks Posed by the Con­ver­gence of Ar­tifi­cial In­tel­li­gence and Biotech­nol­ogy. I’ll quote the most rele­vant sec­tion of text be­low, but table 4 is also rele­vant, and the pa­per is open-ac­cess and (in my view) in­sight­ful, so I’d recom­mend read­ing the whole thing.

  Beyond the risks as­so­ci­ated with med­i­cal de­vice ex­ploita­tion, it is pos­si­ble that in the fu­ture com­puter sys­tems will be in­te­grated with hu­man phys­iol­ogy and, there­fore, pose novel vuln­er­a­bil­ities. Brain-com­puter in­ter­faces (BCIs), tra­di­tion­ally used in medicine for mo­tor-neu­rolog­i­cal di­s­or­ders, are AI sys­tems that al­low for di­rect com­mu­ni­ca­tion be­tween the brain and an ex­ter­nal com­puter. BCIs al­low for a bidi­rec­tional flow of in­for­ma­tion, mean­ing the brain can re­ceive sig­nals from an ex­ter­nal source and vice versa.

  The neu­rotech­nol­ogy com­pany, Neu­ral­ink, has re­cently claimed that a mon­key was able to con­trol a com­puter us­ing one of their im­plants. This con­cept may seem far­fetched, but in 2004 a par­a­lyzed man with an im­planted BCI was able to play com­puter games and check email us­ing only his mind. Other stud­ies have shown a ‘‘brain­brain’’ in­ter­face be­tween mam­mals is pos­si­ble. In 2013, one re­searcher at the Univer­sity of Wash­ing­ton was able to send a brain sig­nal cap­tured by elec­troen­cephalog­ra­phy over the in­ter­net to con­trol the hand move­ments of an­other by way of tran­scra­nial mag­netic stim­u­la­tion. Ad­vances are oc­cur­ring at a rapid pace and many pre­vi­ous tech­ni­cal bot­tle­necks that have pre­vented BCIs from wide­spread im­ple­men­ta­tion are be­gin­ning to be over­come.

  Re­search and de­vel­op­ment of BCIs have ac­cel­er­ated quickly in the past decade. Fu­ture di­rec­tions seek to achieve a sym­bio­sis of AI and the hu­man brain for cog­ni­tive en­hance­ment and rapid trans­fer of in­for­ma­tion be­tween in­di­vi­d­u­als or com­puter sys­tems. Rather than hav­ing to spend time look­ing up a sub­ject, perform­ing a calcu­la­tion, or even speak­ing to an­other in­di­vi­d­ual, the trans­fer of in­for­ma­tion could be nearly in­stan­ta­neous. There have already been nu­mer­ous stud­ies con­ducted re­search­ing the use of BCIs for cog­ni­tive en­hance­ment in do­mains such as learn­ing and mem­ory, per­cep­tion, at­ten­tion, and risk aver­sion (one be­ing able to in­cite riskier be­hav­ior). Ad­di­tion­ally, stud­ies have ex­plored the mil­i­tary ap­pli­ca­tions of BCIs, and the field re­ceives a bulk of its fund­ing from US Depart­ment of Defense sources such as the Defense Ad­vanced Re­search Pro­jects Agency.

  While the com­mer­cial im­ple­men­ta­tion of BCIs may not oc­cur un­til well into the fu­ture, it is still valuable to con­sider the risks that could arise in or­der to high­light the need for se­cu­rity-by-de­sign think­ing and avoid path de­pen­dency, which could re­sult in vuln­er­a­bil­ities—like those seen with cur­rent med­i­cal de­vices—per­sist­ing in fu­ture im­ple­men­ta­tions. Cy­ber vuln­er­a­bil­ities in cur­rent BCIs have already been iden­ti­fied, in­clud­ing those that could cause phys­i­cal harm to the user and in­fluence be­hav­ior. In a fu­ture where BCIs are com­mon­place alongside ad­vanced un­der­stand­ings of neu­ro­science, it may be pos­si­ble for a bad ac­tor to achieve limited in­fluence over the be­hav­ior of a pop­u­la­tion or cause po­ten­tial harm to users. This is­sue high­lights the need to have ro­bust risk as­sess­ment prior to wide­spread tech­nolog­i­cal adop­tion, al­low­ing for reg­u­la­tion, gov­er­nance, and se­cu­rity mea­sures to take iden­ti­fied con­cerns into ac­count.

• MichaelA 11 Aug 2020 7:58 UTC

  7 points

  in reply to: jacobpfau’s comment on: What ques­tions would you like to see fore­casts on from the Me­tac­u­lus com­mu­nity?

  I’d also be in­ter­ested in fore­casts on these top­ics.

  I think Me­tac­u­lus could play a sort of san­ity-check­ing, out­side-view role for EA. Ques­tions like ‘Will EA see AI risk (cli­mate change/​bio-risk/​etc.) as less press­ing in 2030 than they do now?‘, or ‘Will EA in 2030 be­lieve that EA should’ve in­vested more and donated less over the 2020s?’

  It seems to me that there’d be a risk of self-fulfilling prophe­cies.

  That is, we’d hope that what’d hap­pen is:

  1. a bunch of fore­cast­ers pre­dict what the EA com­mu­nity would end up be­liev­ing af­ter a great deal of thought, de­bate, anal­y­sis, etc.

  2. then we can up­date our­selves closer to be­liev­ing that thing already, which could help us get to bet­ter de­ci­sions faster.

  ...But what might in­stead hap­pen is:

  1. a rel­a­tively small group of fore­cast­ers makes rel­a­tively un­founded forecasts

  2. then the EA com­mu­nity—which is rel­a­tively small, un­usu­ally con­nected to Me­tac­u­lus, and un­usu­ally in­ter­ested in fore­casts—up­dates overly strongly on those fore­casts, thus be­liev­ing some­thing that they wouldn’t oth­er­wise have be­lieved and don’t have good rea­sons to believe

  (Per­haps this is like a time-trav­el­ling in­for­ma­tion cas­cade?)

  I’m not say­ing the lat­ter sce­nario is more likely than the former, nor that this means we shouldn’t so­licit these fore­casts. But the lat­ter sce­nario seems likely enough to per­haps be an ar­gu­ment against so­lic­it­ing these fore­casts, and to at least be worth warn­ing read­ers about clearly and re­peat­edly if these fore­casts are in­deed so­lic­ited.

  Also, this might be es­pe­cially bad if EAs start notic­ing that com­mu­nity be­liefs are in­deed mov­ing to­wards the fore­casted fu­ture be­liefs, and don’t ac­count suffi­ciently well for the pos­si­bil­ity that this is just a self-fulfilling prophecy, and thus in­crease the weight they as­sign to these fore­casts. (There could per­haps be a feed­back loop.)

  I imag­ine there’s always some pos­si­bil­ity that fore­casts will in­fluence re­al­ity in a way that makes the fore­casts more or less likely to come true that they would’ve been oth­er­wise. But this seems more-than-usu­ally-likely when fore­cast­ing EA com­mu­nity be­liefs (com­pared to e.g. fore­cast­ing geopoli­ti­cal events).

• MichaelA 11 Aug 2020 4:22 UTC

  5 points

  on: Pro­pose and vote on po­ten­tial tags

  Now vs Later, or Op­ti­mal Timing, or Op­ti­mal Timing for Altru­ists, or some other name.

  This would be in­tended to cap­ture posts rele­vant to the de­bate over “giv­ing now vs later” and “pa­tient vs ur­gent longter­mism”, as well as re­lated de­bates like whether to do di­rect work now vs build ca­reer cap­i­tal vs move­ment-build, and how much to give/​work now vs later, and when to give/​work if not now (“later” is a very large cat­e­gory!).

  This tag would over­lap with Hinge of His­tory, but seems mean­ingfully dis­tinct from that.

  Not sure what the best name would be.

• MichaelA 11 Aug 2020 4:01 UTC

  2 points

  in reply to: Robert_de_Neufville’s comment on: Quan­tify­ing the prob­a­bil­ity of ex­is­ten­tial catas­tro­phe: A re­ply to Beard et al.

  Great, thanks for shar­ing that link! I’ve now ed­ited this post to link to the preprint rather than the pay­walled ver­sion, and to men­tion the blog post.

  (I’d already read the pay­walled ver­sion my­self.)

• MichaelA 11 Aug 2020 1:11 UTC

  2 points

  on: Col­lec­tion of good 2012-2017 EA fo­rum posts

  Really en­joyed this col­lec­tion.

  One more long-term-fu­ture post from that era that I’d recom­mend is Beck­stead’s A pro­posed ad­just­ment to the as­tro­nom­i­cal waste ar­gu­ment. I think that’s been in­fluen­tial in a lot of peo­ple’s think­ing (I see it cited of­ten), in­clud­ing mine.

  Also, re­gard­ing Beck­stead’s Im­prov­ing dis­aster shelters to in­crease the chances of re­cov­ery from a global catas­tro­phe (which you link to), he also wrote a good pa­per on the same topic.

• MichaelA 10 Aug 2020 23:06 UTC

  2 points

  in reply to: smclare’s comment on: I’m Linch Zhang, an am­a­teur COVID-19 fore­caster and gen­er­al­ist EA. AMA

  Yeah, I share the view that that sort of re­search could be very use­ful and seems worth try­ing to do, de­spite the challenges. (Though I hold that view with rel­a­tively low con­fi­dence, due to hav­ing rel­a­tively lit­tle rele­vant ex­per­tise.)

  Some po­ten­tially use­ful links: I dis­cussed the im­por­tance and challenges of es­ti­mat­ing ex­is­ten­tial risk in my EAGx light­ning talk and Un­con­fer­ence talk, provide some other use­ful links (in­clud­ing to pa­pers and to a database of all x-risk es­ti­mates I know of) in this post, and quote from and com­ment on a great re­cent pa­per here.

  I think there are at least two ap­proaches to in­ves­ti­gat­ing this topic: so­licit new fore­casts about the fu­ture and then see how cal­ibrated they are, or find past fore­casts and see how cal­ibrated they were. The lat­ter is what Muehlhauser did, and he found it very difficult to get use­ful re­sults. But it still seems pos­si­ble there’d be room for fur­ther work tak­ing that gen­eral ap­proach, so in a list of his­tory top­ics it might be very valuable to in­ves­ti­gate I men­tion:

  6. The his­tory of pre­dic­tions (es­pe­cially long-range pre­dic­tions and pre­dic­tions of things like ex­tinc­tion), mil­le­nar­i­anism, and how of­ten peo­ple have been right vs wrong about these and other things.

  Hope­fully some his­tor­i­cally minded EA has a crack at re­search­ing that some­day! (Though of course that de­pends on whether it’d be more valuable than other things they could be do­ing.)

  (One could also per­haps so­licit new fore­casts about what’ll hap­pen in some ac­tual his­tor­i­cal sce­nario, from peo­ple who don’t know what ended up hap­pen­ing. I seem to re­call Tet­lock dis­cussing this idea on 80k, but I’m not sure.)

• MichaelA 10 Aug 2020 9:14 UTC

  3 points

  in reply to: bshumway’s comment on: Ad­dress­ing Global Poverty as a Strat­egy to Im­prove the Long-Term Future

  Oh, ok. I knew of “gross na­tional hap­piness” as (1) a thing the Bhutan gov­ern­ment talked about, and (2) a thing some peo­ple men­tion as more im­por­tant than GDP with­out talk­ing pre­cisely about how GNH is mea­sured or what the con­se­quences of more GNH vs more GDP would be. (Those peo­ple were pri­mar­ily so­cial sci­ence teach­ers and text­book au­thors, from when I taught high school so­cial sci­ence.)

  I wasn’t aware GNH had been con­cep­tu­al­ised in a way that in­cludes things quite dis­tinct from hap­piness it­self. I don’t think the peo­ple I’d pre­vi­ously heard about it from were aware of that ei­ther. Know­ing that makes me think GNH is more likely to be a use­ful met­ric for x-risk re­duc­tion, or at least that it’s in the right di­rec­tion, as you sug­gest.

  At the same time, I feel that, in that case, GNH is quite a mis­lead­ing term. (I’d say similar about the Happy Planet In­dex.) But that’s a bit of a tan­gent, and not your fault (as­sum­ing you didn’t moon­light as the king of Bhutan in 1979).

Quan­tify­ing the prob­a­bil­ity of ex­is­ten­tial catas­tro­phe: A re­ply to Beard et al.

• MichaelA 10 Aug 2020 3:17 UTC

  3 points

  on: A List of EA Dona­tion Pledges (GWWC, etc)

  Thanks for putting this to­gether!

  Isn’t the GWWC Pledge too sim­plis­tic to fit ev­ery­one’s spe­cific situ­a­tion?

  Another thing that I think is worth men­tion­ing here is that the GWWC pledge is already less one-size-fits-all than many peo­ple re­al­ise. To illus­trate, here are some key points from their FAQ:

  The pledge is of course just a min­i­mum. Some mem­bers de­cide to go fur­ther than this and pledge to give a higher per­centage, such as 20% or even 50%.

  [...]

  What do you mean by in­come?

  The goal here is to help mem­bers stick to their plan of tak­ing sig­nifi­cant ac­tion to benefit oth­ers. All guidelines about how to calcu­late in­come should be thought of as serv­ing that goal. [Then there are more de­tails on this.]

  [...]

  Stu­dents, un­em­ployed peo­ple, and full-time parents

  Many stu­dents, un­em­ployed peo­ple and full-time par­ents have lit­tle or no in­come, but are largely sup­ported by money from fam­ily mem­bers, the gov­ern­ment or a stu­dent loan.

  The Pledge does not re­quire you to donate any of this fund­ing (al­though it does com­mit you re­gard­ing any fu­ture in­come). How­ever, in the in­ter­ests of all of our mem­bers giv­ing what they can, we feel that the spirit of the Pledge re­quires them to donate at least 1% of their spend­ing money.

  We define spend­ing money as money re­ceived for the pur­pose of spend­ing on items such as food, rent, travel, chil­dren, or per­sonal items. It does not in­clude spend­ing on tu­ition fees. If a cou­ple with shared fi­nances both wish to join, then they can sim­ply donate 10% of their com­bined earn­ings and not worry about spend­ing money.

  Of course, peo­ple who earn some in­come but de­pend on other help for their liv­ing ex­penses may choose to donate 10% of their earn­ings if they want to go above and be­yond.

  [...]

  How of­ten should mem­bers donate?

  The spirit of the Pledge is to donate on an on­go­ing ba­sis, rather than let­ting “dona­tion debt” build up over many years. We check in with mem­bers ev­ery year and en­courage them to log their dona­tions. How­ever, you don’t have to donate on a strictly an­nual ba­sis. Mem­bers who con­soli­date dona­tions into cer­tain years (for ex­am­ple for tax ad­van­tages, or in case of tem­po­rary fi­nan­cial hard­ship) are wel­come to do so.

  Do dona­tions have to be to reg­istered char­i­ties?

  The com­mit­ment is to donate to “the most effec­tive or­gani­sa­tions”. Th­ese or­gani­sa­tions could be char­i­ties, but could also be en­tities not offi­cially reg­istered as tax-de­ductible char­i­ties (for ex­am­ple a char­ity in the early stages of get­ting reg­istered, or an ad­vo­cacy or lob­by­ing group that is not a char­ity.)

• MichaelA 10 Aug 2020 2:14 UTC

  2 points

  in reply to: bshumway’s comment on: Ad­dress­ing Global Poverty as a Strat­egy to Im­prove the Long-Term Future

  A use­ful con­cept here might be that of an “en­vi­ron­men­tal Kuznets curve”:

  The en­vi­ron­men­tal Kuznets curve (EKC) is a hy­poth­e­sized re­la­tion­ship be­tween en­vi­ron­men­tal qual­ity and eco­nomic de­vel­op­ment[17]: var­i­ous in­di­ca­tors of en­vi­ron­men­tal degra­da­tion tend to get worse as mod­ern eco­nomic growth oc­curs un­til av­er­age in­come reaches a cer­tain point over the course of de­vel­op­ment.[18][19] The EKC sug­gests, in sum, that “the solu­tion to pol­lu­tion is eco­nomic growth.”

  There is both ev­i­dence for and against the EKC. I’m guess­ing the ev­i­dence varies for differ­ent as­pects of en­vi­ron­men­tal qual­ity and be­tween re­gions. I’m not an ex­pert on this, but that Wikipe­dia sec­tion would prob­a­bly be a good place for some­one in­ter­ested in the topic to start.

  I already think tech­nol­ogy is at a point where welfare does not have to de­pend on fos­sil fuel con­sump­tion.

  I think I broadly agree, but that it’s also true that pre­sent-day welfare is cheaper if we use fos­sil fuels than low/​no car­bon fuels (if we’re ig­nor­ing things like car­bon taxes or re­new­ables sub­sidies that were put in place speci­fi­cally to ad­dress the ex­ter­nal­ities). I think car­bon miti­ga­tion is well worth the price (in­clud­ing the price of en­act­ing e.g. car­bon taxes) when we con­sider fu­ture gen­er­a­tions, and per­haps even when we con­sider pre­sent gen­er­a­tions’ en­tire lifes­pans (though I haven’t looked into that). But there are some real ten­sions there, for peo­ple who are in prac­tice fo­cused on near-term effects.

• MichaelA 10 Aug 2020 2:01 UTC

  2 points

  in reply to: bshumway’s comment on: Ad­dress­ing Global Poverty as a Strat­egy to Im­prove the Long-Term Future

  Epistemic sta­tus: I’ve only spent per­haps 15 min­utes think­ing about these spe­cific mat­ters, though I’ve thought more about re­lated things.

  I’d guess that hap­piness lev­els (while of course in­trin­si­cally im­por­tant) wouldn’t be es­pe­cially valuable as a met­ric of how well a global health/​de­vel­op­ment in­ter­ven­tion is re­duc­ing ex­is­ten­tial risks. I don’t see a strong rea­son to be­lieve in­creased hap­piness (at least from the cur­rent mar­gin) leads to bet­ter han­dling of AI risk and biorisk. Hap­piness may cor­re­late with x-risk re­duc­tion, but if so, it’d prob­a­bly due to other vari­ables af­fect­ing both of those vari­ables.

  Met­rics that seem more use­ful to me might be things like:

  • qual­ity of rea­son­ing and ev­i­dence used for key poli­ti­cal and cor­po­rate de­ci­sion-making

    • Though op­er­a­tional­is­ing this is of course difficult

  • will­ing­ness to con­sider not just risks but also benefits of tech­nolog­i­cal and eco­nomic development

    • This is tricky be­cause I think peo­ple of­ten over­es­ti­mate or over­weight the risks from var­i­ous de­vel­op­ments (e.g., GMO crops), es­pe­cially if our fo­cus is on just the com­ing years or decades. So we’d want to some­how tar­get this met­ric to the “ac­tu­ally” risky de­vel­op­ments, or to “con­sid­er­ing” risks in a rea­son­able way rather than just in gen­eral.

  • lev­els of emissions

  • lev­els of corruption

  The last two of those met­rics might be “di­rectly” im­por­tant for ex­is­ten­tial risk re­duc­tion, but also might serve as a proxy for things like the first two met­rics or other things we care about.

• MichaelA 10 Aug 2020 1:56 UTC

  2 points

  in reply to: RayTaylor’s comment on: Ad­dress­ing Global Poverty as a Strat­egy to Im­prove the Long-Term Future

  Su­per cool to hear that UNDRR will be con­sid­er­ing ex­is­ten­tial risks!

  Are there any links to pub­lic state­ments or white pa­pers or what­ever men­tion­ing that yet?

  If not, could you say a lit­tle more about this new con­sid­er­a­tion might en­tail for their work?

  (I’d also un­der­stand if it’s too early days or if this info is some­what sen­si­tive for some rea­son.)

• MichaelA 10 Aug 2020 1:54 UTC

  5 points

  in reply to: RayTaylor’s comment on: Ad­dress­ing Global Poverty as a Strat­egy to Im­prove the Long-Term Future

  Just wanted to add the link and ab­stract to the bor­ing apoc­a­lypses pa­per, for any read­ers who may be in­ter­ested: Govern­ing Bor­ing Apoca­lypses: A new ty­pol­ogy of ex­is­ten­tial vuln­er­a­bil­ities and ex­po­sures for ex­is­ten­tial risk research

  In re­cent years, the study of ex­is­ten­tial risks has ex­plored a range of nat­u­ral and man-made catas­tro­phes, from su­per­vol­cano erup­tion to nu­clear war, and from global pan­demics to po­ten­tial risks from mis­al­igned AI. Th­ese risks share the prospect of caus­ing out­right hu­man ex­tinc­tion were they to oc­cur. In this ap­proach, such iden­ti­fied ex­is­ten­tial risks are fre­quently char­ac­ter­ised by rel­a­tively sin­gu­lar ori­gin events and con­crete path­ways of harm which di­rectly jeop­ar­dise the sur­vival of hu­man­ity, or un­der­cut its po­ten­tial for long-term tech­nolog­i­cal progress. While this ap­proach aptly iden­ti­fies the most cat­a­clys­mic fates which may be­fall hu­man­ity, we ar­gue that catas­trophic ‘ex­is­ten­tial out­comes’ may likely arise from a broader range of sources and so­cietal vuln­er­a­bil­ities, and through the com­plex in­ter­ac­tions of dis­parate so­cial, cul­tural, and nat­u­ral pro­cesses—many of which, taken in iso­la­tion, might not be seen to merit at­ten­tion as a global catas­trophic, let alone ex­is­ten­tial, risk.

  This ar­ti­cle ar­gues that an em­pha­sis on miti­gat­ing the haz­ards (dis­crete causes) of ex­is­ten­tial risks is an un­nec­es­sar­ily nar­row fram­ing of the challenge fac­ing hu­man­ity, one which risks pre­ma­turely cur­tailing the spec­trum of policy re­sponses con­sid­ered. In­stead, it ar­gues ex­is­ten­tial risks con­sti­tute but a sub­set in a broader set of challenges which could di­rectly or in­di­rectly con­tribute to ex­is­ten­tial con­se­quences for hu­man­ity. To illus­trate, we in­tro­duce and ex­am­ine a set of ex­is­ten­tial risks that of­ten fall out­side the scope of, or re­main un­der­stud­ied within, the field. By fo­cus­ing on vuln­er­a­bil­ity and ex­po­sure rather than ex­is­ten­tial haz­ards, we de­velop a new tax­on­omy which cap­tures fac­tors con­tribut­ing to these ex­is­ten­tial risks. La­tent struc­tural vuln­er­a­bil­ities in our tech­nolog­i­cal sys­tems and in our so­cietal ar­range­ments may in­crease our sus­cep­ti­bil­ity to ex­is­ten­tial haz­ards. Fi­nally, differ­ent types of ex­po­sure of our so­ciety or its nat­u­ral base de­ter­mine if or how a given haz­ard can in­ter­face with pre-ex­ist­ing vuln­er­a­bil­ities, to trig­ger emer­gent ex­is­ten­tial risks.

  We ar­gue that far from be­ing periph­eral foot­notes to their more di­rect and im­me­di­ately ter­mi­nal coun­ter­parts, these “Bor­ing Apoca­lypses” may well prove to be the more en­demic and prob­le­matic, drag­ging down and un­der­cut­ting short-term suc­cesses in miti­gat­ing more spec­tac­u­lar risks. If the car­di­nal con­cern is hu­man­ity’s con­tinued sur­vival and pros­per­ity, then fo­cussing aca­demic and pub­lic ad­vo­cacy efforts on re­duc­ing di­rect ex­is­ten­tial haz­ards may have the para­dox­i­cal po­ten­tial of ex­ac­er­bat­ing hu­man­ity’s in­di­rect sus­cep­ti­bil­ity to such out­comes. Adopt­ing law and policy per­spec­tives al­low us to fore­ground so­cietal di­men­sions that com­ple­ment and re­in­force the dis­course on ex­is­ten­tial risks.

  (That’s in­deed just the ab­stract; not sure why it’s three para­graphs long. Also, dis­claimer that I haven’t read be­yond the ab­stract my­self.)

• MichaelA 10 Aug 2020 1:33 UTC

  2 points

  on: I’m Linch Zhang, an am­a­teur COVID-19 fore­caster and gen­er­al­ist EA. AMA

  Thanks for do­ing this AMA! In case you still might an­swer ques­tions, I’m cu­ri­ous as to how much value you think there’d be in:

  • fur­ther re­search into fore­cast­ing techniques

  • im­prov­ing ex­ist­ing fore­cast­ing tools and platforms

  • de­vel­op­ing bet­ter tools and platforms

  E.g., if some­one asked you for ad­vice on whether to do work in academia similar to Tet­lock’s work, or build things like Me­tac­u­lus or cal­ibra­tion games, or do some­thing else EAs of­ten think is valuable, what might you say?

  (I ask in part be­cause you wrote about judge­men­tal fore­cast­ing be­ing “very much a nascent, pre-paradigm field”.)

• MichaelA 9 Aug 2020 4:48 UTC

  3 points

  in reply to: smclare’s comment on: I’m Linch Zhang, an am­a­teur COVID-19 fore­caster and gen­er­al­ist EA. AMA

  I think this is a very im­por­tant ques­tion. In case you haven’t seen it, here’s Luke Muehlhauser’s overview of his post How Fea­si­ble Is Long-range Fore­cast­ing? (I’d also highly recom­mend read­ing the whole post):

  How ac­cu­rate do long-range (≥10yr) fore­casts tend to be, and how much should we rely on them?

  As an ini­tial ex­plo­ra­tion of this ques­tion, I sought to study the track record of long-range fore­cast­ing ex­er­cises from the past. Un­for­tu­nately, my key find­ing so far is that it is difficult to learn much of value from those ex­er­cises, for the fol­low­ing rea­sons:

  1. Long-range fore­casts are of­ten stated too im­pre­cisely to be judged for ac­cu­racy. [More]

  2. Even if a fore­cast is stated pre­cisely, it might be difficult to find the in­for­ma­tion needed to check the fore­cast for ac­cu­racy. [More]

  3. De­grees of con­fi­dence for long-range fore­casts are rarely quan­tified. [More]

  4. In most cases, no com­par­i­son to a “baseline method” or “null model” is pos­si­ble, which makes it difficult to as­sess how easy or difficult the origi­nal fore­casts were. [More]

  5. In­cen­tives for fore­caster ac­cu­racy are usu­ally un­clear or weak. [More]

  6. Very few stud­ies have been de­signed so as to al­low con­fi­dent in­fer­ence about which fac­tors con­tributed to fore­cast­ing ac­cu­racy. [More]

  7. It’s difficult to know how com­pa­rable past fore­cast­ing ex­er­cises are to the fore­cast­ing we do for grant­mak­ing pur­poses, e.g. be­cause the fore­casts we make are of a differ­ent type, and be­cause the fore­cast­ing train­ing and meth­ods we use are differ­ent. [More]

  We plan to con­tinue to make long-range quan­tified fore­casts about our work so that, in the long run, we might learn some­thing about the fea­si­bil­ity of long-range fore­cast­ing, at least for our own case.

  (See also the com­ments on the EA Fo­rum link post.)

• MichaelA 8 Aug 2020 22:52 UTC

  3 points

  in reply to: fmoreno’s comment on: Pro­pose and vote on po­ten­tial tags

  If you mean un-tag­ging a page, you vote down its rele­vance by hov­er­ing over the tag on the page and click­ing the < ar­row. If the rele­vance score gets to or be­low 0, the tag is re­moved.

  If you mean delet­ing a tag en­tirely (not just from one page), I think you’d have to mes­sage the EA Fo­rum team?

  More info on tags here and here.

• MichaelA 8 Aug 2020 6:50 UTC

  2 points

  in reply to: MichaelA’s comment on: Pro­pose and vote on po­ten­tial tags

  (Up­date: I’ve now made this tag.)

  Space (or maybe Space Gover­nance, or Space Gover­nance & Colon­i­sa­tion, or some­thing along those lines)

  “Gover­nance of outer space” is men­tioned by 80k here.

  Would per­haps just be a sub­set of Long-Term Fu­ture. But per­haps a suffi­ciently large and im­por­tant sub­set to war­rant its own tag.

  Some posts this should in­clude:

  • Will we even­tu­ally be able to colonize other stars? Notes from a pre­limi­nary review

  • Space gov­er­nance is im­por­tant, tractable and neglected

  • Off-Earth Governance

  • An In­for­mal Re­view of Space Exploration

  • Maybe Lu­nar Colony

  • Maybe Does Utili­tar­ian Longter­mism Im­ply Directed Pansper­mia?