William_MacAskill

• William_MacAskill 13 Sep 2019 19:56 UTC

  2 points

  in reply to: WilliamKiely's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Thanks! :)

• William_MacAskill 13 Sep 2019 19:51 UTC

  9 points

  in reply to: Kit's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Sorry - ‘or oth­er­wise lost’ qual­ifier was meant to be a catch-all for any way of the in­vest­ment los­ing its value, in­clud­ing (bad) value-drift.

  I think there’s a de­cent case for (some) EAs do­ing bet­ter at avoid­ing this than e.g. typ­i­cal foun­da­tions:

  • If you have pre­cise val­ues (e.g. clas­si­cal util­i­tar­i­anism) then it’s eas­ier to trans­mit those val­ues across time—you can write your val­ues down clearly as part of the con­sti­tu­tion of the foun­da­tion, and it’s eas­ier to find and iden­tify younger peo­ple to take over the fund who also en­dorse those val­ues. In con­trast, for other foun­da­tions, the ul­ti­mate aims of the foun­da­tion are of­ten not clear, and too de­pen­dent on a par­tic­u­lar em­piri­cal situ­a­tion (e.g. Ben­jamin Fran­klin’s funds were to ‘to provide loans for ap­pren­tices to start their busi­nesses’ (!!)).

  • If you take a lot of time care­fully choos­ing who your suc­ces­sors are (and those peo­ple take a lot of time over who their suc­ces­sors are).

  Then to re­duce ap­pro­pri­a­tion, one could spread the funds across many differ­ent coun­tries and differ­ent peo­ple who share your val­ues. (Again, eas­ier if you en­dorse a set of val­ues that are leg­ible and non-idiosyn­cratic.)

  It might still be true that the chance of the fund be­com­ing val­ue­less gets large over time (if, e.g. there’s a 1% risk of it los­ing its value per year), but the size of the re­sources available also in­creases ex­po­nen­tially over time in those wor­lds where it doesn’t lose its value.

  Caveat also tricky ques­tions on when ‘value drift’ is a bad thing rather than the fu­ture fund own­ers just hav­ing a bet­ter un­der­stand­ing of the right thing to do than the founders did, which of­ten seems to be true for long-last­ing foun­da­tions.

• William_MacAskill 13 Sep 2019 1:14 UTC

  4 points

  in reply to: Carl_Shulman's comment on: Ask Me Any­thing!

  I think you might be mi­s­un­der­stand­ing what I was refer­ring to. An ex­am­ple of what I mean: Sup­pose Jane is de­cid­ing whether to work for Deep­mind on the AI safety team. She’s un­sure whether this speeds up or slows down AI de­vel­op­ment; her cre­dence is im­pre­cise, rep­re­sented by the in­ter­val [0.4, 0.6]. She’s con­fi­dent, let’s say, that speed­ing up AI de­vel­op­ment is bad. Be­cause there’s some pre­cisifi­ca­tion of her cre­dences on which tak­ing the job is good, and some on which tak­ing the job is bad, then if she uses a Liberal de­ci­sion rule (= it is per­mis­si­ble for you to perform any ac­tion that is per­mis­si­ble ac­cord­ing to at least one of the cre­dence func­tions in your set), it’s per­mis­si­ble for her to take the job or not take the job.

  The is­sue is that, if you have im­pre­cise cre­dences and a Liberal de­ci­sion rule, and are a longter­mist, then al­most all se­ri­ous con­tenders for ac­tions are per­mis­si­ble.

  So the neart­er­mist would need to have some way of say­ing (i) we can carve out the definitely-good part of the ac­tion, which is bet­ter than not-do­ing the ac­tion on all pre­cisifi­ca­tions of the cre­dence; (ii) we can ig­nore the other parts of the ac­tion (e.g. the flow-through effects) that are good on some pre­cisifi­ca­tions and bad on some pre­cisifi­ca­tions. It seems hard to make that the­o­ret­i­cally jus­tified, but I think it matches how peo­ple ac­tu­ally think, so at least has some com­mon-sense mo­ti­va­tion.

  But you could do it if you could ar­gue for a pseu­dodom­i­nance prin­ci­ple that says: “If there’s some in­ter­val of time t_i over which ac­tion x does more ex­pected good than ac­tion y on all pre­cisifi­ca­tions of one’s cre­dence func­tion, and there’s no in­ter­val of time t_j at which ac­tion y does more ex­pected good than ac­tion x on all pre­cisifi­ca­tions of one’s cre­dence func­tion, then you should choose x over y”.

  
  

  (In con­trast, it seems you thought I was refer­ring to AI vs some other pu­ta­tive great longter­mist in­ter­ven­tion. I agree that plau­si­ble longter­mist ri­vals to AI and bio are thin on the ground.)

• William_MacAskill 13 Sep 2019 1:07 UTC

  4 points

  in reply to: WilliamKiely's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Thanks, William!

  Yeah, I think I messed up this bit. I should have used the har­monic mean rather than the ar­ith­metic mean when av­er­ag­ing over pos­si­bil­ities of how many peo­ple will be in the fu­ture. Do­ing this brings the chance of be­ing among the most in­fluen­tial per­son ever close to the chance of be­ing the most in­fluen­tial per­son ever in a small-pop­u­la­tion uni­verse. But then we get the is­sue that be­ing the most in­fluen­tial per­son ever in a small-pop­u­la­tion uni­verse is much less im­por­tant than be­ing the most in­fluen­tial per­son in a big-pop­u­la­tion uni­verse. And it’s only the lat­ter that we care about.

  
  

  So what I re­ally should have said (in my too-glib ar­gu­ment) is: for sim­plic­ity, just as­sume a high-pop­u­la­tion fu­ture, which are the ac­tion-rele­vant fu­tures if you’re a longter­mist. Then take a uniform prior over all times (or all peo­ple) in that high-pop­u­la­tion fu­ture. So my claim is: “In the ac­tion-rele­vant wor­lds, the fre­quency of ‘most im­por­tant time’ (or ‘most im­por­tant per­son’) is ex­tremely low, and so should be our prior.”

• William_MacAskill 13 Sep 2019 1:05 UTC

  3 points

  in reply to: gwern's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Thanks for these links. I’m not sure if your com­ment was meant to be a crit­i­cism of the ar­gu­ment, though? If so: I’m say­ing “prior is low, and there is a healthy false pos­i­tive rate, so don’t have high pos­te­rior.” You’re point­ing out that there’s a healthy false nega­tive rate too — but that won’t cause me to have a high pos­te­rior?

  And, if you think that ev­ery gen­er­a­tion is in­creas­ing in in­fluen­tial­ness, that’s a good ar­gu­ment for think­ing that fu­ture gen­er­a­tions will be more in­fluen­tial and we should there­fore save.

• William_MacAskill 13 Sep 2019 1:02 UTC

  14 points

  on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  There were a cou­ple of re­cur­ring ques­tions, so I’ve ad­dressed them here.

  What’s the point of this dis­cus­sion — isn’t pass­ing on re­sources to the fu­ture too hard to be worth con­sid­er­ing? Won’t the money be stolen, or used by peo­ple with worse val­ues?

  In brief: Yes, los­ing what you’ve in­vested is a risk, but (at least for rel­a­tively small donors) it’s out­weighed by in­vest­ment re­turns.

  Longer: The con­cept of ‘in­fluen­tial­ness of a time’ is the same as the cost-effec­tive­ness (from a longter­mist per­spec­tive) of the best op­por­tu­ni­ties ac­cessible to longter­mists at a time. Sup­pose I think that the best op­por­tu­ni­ties in, say, 100 years, are as good as the best op­por­tu­ni­ties now. Then, if I have a small amount of money, then I can get (say) at least a 2% re­turn per year on those funds. But I shouldn’t think that the chance of my funds be­ing ap­pro­pri­ated (or oth­er­wise lost) is as high as 2% per year. So the ex­pected amount of good I do is greater by sav­ing.

  So if you think that hingey­ness (as I’ve defined it) is about the same in 100 years as it is now, or greater, then there’s a strong case for in­vest­ing for 100 years be­fore spend­ing the money.

  (Caveat that once we con­sider larger amounts of money, diminish­ing re­turns for ex­pen­di­ture be­comes an is­sue, and chance of ap­pro­pri­a­tion in­creases.)

  What’s your view on an­throp­ics? Isn’t that rele­vant here?

  I’ve been try­ing to make claims that aren’t sen­si­tive to tricky is­sues in an­thropic rea­son­ing. The claim that if there are n peo­ple, or­dered in terms of some re­la­tion F (like ‘more im­por­tant than’), then the claim that the prior prob­a­bil­ity that you are most F (‘most im­por­tant’) per­son is 1/​n doesn’t dis­t­in­guish be­tween an­thropic prin­ci­ples, be­cause I’ve already con­di­tioned on the num­ber of peo­ple in the world. So I think an­thropic prin­ci­ples aren’t di­rectly rele­vant for the ar­gu­ment I’ve made, though ob­vi­ously they are rele­vant more gen­er­ally.

• William_MacAskill 13 Sep 2019 0:49 UTC

  3 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  I don’t think I agree with this, un­less one is able to make a com­par­a­tive claim about the im­por­tance (from a longter­mist per­spec­tive) of these events rel­a­tive to fu­ture events’ im­por­tance—which is ex­actly what I’m ques­tion­ing.

  I do think that weight­ing ear­lier gen­er­a­tions more heav­ily is cor­rect, though; I don’t feel that much turns on whether one con­strues this as prior choice or an up­date from one’s prior.

• William_MacAskill 13 Sep 2019 0:44 UTC

  2 points

  in reply to: Toby_Ord's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Given this, if one had a hy­per­prior over differ­ent pos­si­ble Beta dis­tri­bu­tions, shouldn’t 2000 cen­turies of no event oc­cur­ring cause one to up­date quite hard against the (0.5, 0.5) or (1, 1) hy­per­pa­ram­e­ters, and in favour of a prior that was mas­sively skewed to­wards the per-cen­tury prob­a­bil­ity of no-lock-in-event be­ing very low?

  (And not­ing that, de­pend­ing ex­actly on how the propo­si­tion is speci­fied, I think we can be very con­fi­dent that it hasn’t hap­pened yet. E.g. if the propo­si­tion un­der con­sid­er­a­tion was ‘a val­ues lock-in event oc­curs such that ev­ery­one af­ter this point has the same val­ues’.)

• William_MacAskill 13 Sep 2019 0:38 UTC

  40 points

  in reply to: Toby_Ord's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Hi Toby,

  Thanks so much for this very clear re­sponse, it was a very satis­fy­ing read, and there’s a lot for me to chew on. And thanks for lo­cat­ing the point of dis­agree­ment — prior to this post, I would have guessed that the biggest differ­ence be­tween me and some oth­ers was on the weight placed on the ar­gu­ments for the Time of Per­ils and Value Lock-In views, rather than on the choice of prior. But it seems that that’s not true, and that’s very helpful to know. If so, it sug­gests (ad­ver­tise­ment to the Fo­rum!) that fur­ther work on prior-set­ting in EA con­texts is very high-value.

  I agree with you that un­der un­cer­tainty over how to set the prior, be­cause we’re clearly so dis­tinc­tive in some par­tic­u­lar ways (namely, that we’re so early on in civil­i­sa­tion, that the cur­rent pop­u­la­tion is so small, etc), my choice of prior will get washed out by mod­els on which those dis­tinc­tive fea­tures are im­por­tant; I char­ac­ter­ised these as out­side-view ar­gu­ments, but I’d un­der­stand if some­one wanted to char­ac­ter­ise that as prior-set­ting in­stead.

  I also agree that there’s a strong case for mak­ing the prior over per­sons (or per­son-years) rather than cen­turies. In your dis­cus­sion, you go via num­ber of per­sons (or per­son-years) per cen­tury to the com­par­a­tive im­por­tance of cen­turies. What I’d be in­clined to do is just change the claim un­der con­sid­er­a­tion to: “I am among the (say) 100,000 most in­fluen­tial peo­ple ever”. This means we still take into ac­count the fact that, though more pop­u­lous cen­turies are more likely to be in­fluen­tial, they are also harder to in­fluence in virtue of their larger pop­u­la­tion. If we frame the core claim in terms of be­ing among the most in­fluen­tial peo­ple, rather than be­ing at the most in­fluen­tial time, the core claim seems even more strik­ing to me. (E.g. a uniform prior over the first 100 billion peo­ple would give a prior of 1 in 1 mil­lion of be­ing in the 100,000 most in­fluen­tial peo­ple ever. Though of course, there would also be an ex­tra out­side-view ar­gu­ment for mov­ing from this prior, which is that not many peo­ple are try­ing to in­fluence the long-run fu­ture.)

  How­ever, I don’t cur­rently feel at­tracted to your way of set­ting up the prior. In what fol­lows I’ll just fo­cus on the case of a val­ues lock-in event, and for sim­plic­ity I’ll just use the stan­dard Laplacean prior rather than your sug­ges­tion of a Jeffreys prior.

  In sig­nifi­cant part my lack of at­trac­tion is be­cause the claims — that (i) there’s a point in time where al­most ev­ery­thing about the fate of the uni­verse gets de­cided; (ii) that point is ba­si­cally now; (iii) al­most no-one sees this apart from us (where ‘us’ is a very small frac­tion of the world) — seem ex­traor­di­nary to me, and I feel I need ex­traor­di­nary ev­i­dence in or­der to have high cre­dence in them. My prior-set­ting dis­cus­sion was one way of cash­ing out why these seem ex­traor­di­nary. If there’s some way of set­ting pri­ors such that claims (i)-(iii) aren’t so ex­traor­di­nary af­ter all, I feel like a rab­bit is be­ing pul­led out of a hat.

  Then I have some spe­cific wor­ries with the Laplacean ap­proach (which I *think* would ap­ply to the Jeffreys prior, too, but I’m yet to figure out what a Fischer in­for­ma­tion ma­trix is, so I don’t to­tally back my­self here).

  But be­fore I men­tion the wor­ries, I’ll note that it seems to me that you and I are cur­rently talk­ing about pri­ors over differ­ent propo­si­tions. You seem to be con­sid­er­ing the propo­si­tions, ‘there is a lock-in event this cen­tury’ or ‘there is an ex­tinc­tion event this cen­tury’; I’m con­sid­er­ing the propo­si­tion ‘I am at the most in­fluen­tial time ever’ or ‘I am one of the most in­fluen­tial peo­ple ever.’ As is well-known, when it comes to us­ing prin­ci­ple-of-in­differ­ence-es­que rea­son­ing, if you use that rea­son­ing over a num­ber of differ­ent propo­si­tions then you can end up with in­con­sis­tent prob­a­bil­ity as­sign­ments. So, at best, one should use such rea­son­ing in a very re­stricted way.

  The rea­son I like think­ing about my propo­si­tion (‘are we at the most im­por­tant time?’ or ‘are we one of the most in­fluen­tial peo­ple ever?’) for the re­stricted prin­ci­ple of in­differ­ence, is that:

  (i) I know the fre­quency of oc­cur­rence of ‘most in­fluen­tial per­son’, for each pos­si­ble to­tal pop­u­la­tion of civ­i­liza­tion (past, pre­sent and fu­ture). Namely, it oc­curs once out of the to­tal pop­u­la­tion. So I can look at each pos­si­ble pop­u­la­tion size for the fu­ture, look at my cre­dence in each pos­si­ble pop­u­la­tion oc­cur­ring, and in each case know the fre­quency of be­ing the most in­fluen­tial per­son (or, more nat­u­rally, in the 100,000 most in­fluen­tial peo­ple).

  (ii) it’s the most rele­vant propo­si­tion for the ques­tion of what I should do. (e.g. Per­haps it’s likely that there’s a lock-in event, but we can’t do any­thing about it and fu­ture peo­ple could, so we should save for a later date.)

  Any­way, the wor­ries about Laplacean (and Jeffreys) prior.

  First, the Laplacean prior seems to get the wrong an­swer for lots of similar pred­i­cates. Con­sider the claims: “I am the most beau­tiful per­son ever” or “I am the strongest per­son ever”, rather than “I am the most im­por­tant per­son ever”. If we used the Laplacean prior in the way you sug­gest for these claims, the first per­son would as­sign 50% cre­dence to be­ing the strongest per­son ever, even if they knew that there was prob­a­bly go­ing to be billions of peo­ple to come. This doesn’t seem right to me.

  Se­cond, it also seems very sen­si­tive to our choice of start date. If the propo­si­tion un­der ques­tion is, ‘there will be a lock-in event this cen­tury’, I’d get a very differ­ent prior de­pend­ing on whether I chose to be­gin count­ing from: (i) the dawn of the in­for­ma­tion age; (ii) the be­gin­ning of the in­dus­trial rev­olu­tion; (iii) the start of civil­i­sa­tion; (iv) the ori­gin of homo sapi­ens; (v) the ori­gin of the genus homo; (vi) the ori­gin of mam­mals, etc.

  Of course, the uniform prior has some­thing similar, but I think it han­dles the is­sue grace­fully. e.g. On pri­ors, I should think it’s 1 in 5 mil­lion likely that I’m the fun­niest per­son in Scot­land; 1 in 65 mil­lion that I’m the fun­niest per­son in Bri­tain, 1 in 7.5 billion that I’m the fun­niest per­son in the world. Similarly, with whether I’m the most in­fluen­tial per­son in the post-in­dus­trial era, the post-agri­cul­tural era, etc.

  Third, the Laplacean prior doesn’t add up to 1 across all peo­ple. For ex­am­ple, sup­pose you’re the first per­son and you know that there will be 3 peo­ple. Then, on the Laplacean prior, the to­tal prob­a­bil­ity for be­ing the most in­fluen­tial per­son ever is ½ + ½(⅓) + ½(⅔)(¼) = ¾. But I know that some­one has to be the most in­fluen­tial per­son ever. This sug­gests the Laplacean prior is the wrong prior choice for the propo­si­tion I’m con­sid­er­ing, whereas the sim­ple fre­quency ap­proach gets it right.

  So even if one feels skep­ti­cal of the uniform prior, I think the Laplacean way of prior-set­ting isn’t a bet­ter al­ter­na­tive. In gen­eral: I’m sym­pa­thetic to hav­ing a model where early peo­ple are more likely to be more in­fluen­tial, but a model which is uniform over or­ders of mag­ni­tude seems too ex­treme to me.

  
  

  (As a fi­nal thought: Doesn’t this form of prior-set­ting also suffer from the prob­lem of there be­ing too many hy­pothe­ses? E.g. con­sider the propo­si­tions:

  A—There will be a value lock-in event this cen­tu­ry
  B—There will be a lock-in of he­do­nis­tic util­i­tar­ian val­ues this century

  C—There will be a lock-in of prefer­ence util­i­tar­ian val­ues this century

  D—There will be a lock-in of Kan­tian val­ues this century

  E—There will be a lock-in of fas­cist val­ues this century

  On the Laplacean ap­proach, these would all get the same prob­a­bil­ity as­sign­ment—which seems in­con­sis­tent. And then just by stack­ing pri­ors over par­tic­u­lar lock-in events, we can get a prob­a­bil­ity that it’s over­whelm­ingly likely that there’s some lock-in event this cen­tury. I’ve put this com­ment in paren­the­ses, though, as I feel *even less* con­fi­dent about my worry here than my other wor­ries listed.)

• William_MacAskill 5 Sep 2019 4:02 UTC

  5 points

  in reply to: Gregory_Lewis's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  The way I’d think about it is that we should be un­cer­tain about how jus­tifi­ably con­fi­dent peo­ple can be that they’re at the HoH. If our cur­rent cre­dence in HoH is low, then the chance that it might be jus­tifi­ably much higher in the fu­ture should be the sig­nifi­cant con­sid­er­a­tion. At least if we put aside simu­la­tion wor­ries, I can imag­ine ev­i­dence which would lead me to have high con­fi­dence that I’m at the HoH.

  E.g., the prior is (say) 1/​mil­lion this decade, but if the ev­i­dence sug­gests it is 1%, per­haps we should drop ev­ery­thing to work on it, if we won’t ex­pect our cre­dence to be this high again for an­other mil­le­nia.

  I think if that were one’s cre­dences, what you say makes sense. But it seems hard for me to imag­ine a (re­al­is­tic) situ­a­tion where I think that it’s 1% chance of HoH this decade, but I’m con­fi­dent that the chance will much much lower than that for all of the next 99 decades.

  For what it’s worth, my in­tu­ition is that pur­su­ing a mixed strat­egy is best; some peo­ple aiming for im­pact now, in case now is a hinge, and some peo­ple aiming for im­pact in many many years, at some fu­ture hinge mo­ment.

• William_MacAskill 5 Sep 2019 3:48 UTC

  24 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  So I would say both the pop­u­la­tion and pre-emp­tion (by ear­lier sta­billiza­tion) fac­tors in­tensely fa­vor ear­lier eras in per re­source hingey­ness, con­strained by the era hav­ing any sig­nifi­cant lock-in op­por­tu­ni­ties and the pres­ence of longter­mists.

  I think this is a re­ally im­por­tant com­ment; I see I didn’t put these con­sid­er­a­tions into the out­side-view ar­gu­ments, but I should have done as they are make for pow­er­ful ar­gu­ments.

  The fac­tors you men­tion are analo­gous to the pa­ram­e­ters that go into the Ram­sey model for dis­count­ing: (i) a pure rate of time prefer­ence, which can ac­count for risk of pre-emp­tion; (ii) a term to ac­count for there be­ing more (and, pre­sum­ably, richer) fu­ture agents and some sort of diminish­ing re­turns as a func­tion of how many fu­ture agents (or to­tal re­sources) there are. Then given un­cer­tainty about these pa­ram­e­ters, in the long run the sce­nar­ios that dom­i­nate the EV calcu­la­tion are where there’s been no pre-emp­tion and the fu­ture pop­u­la­tion is not that high. e.g. There’s been some great so­cietal catas­tro­phe and we’re re­build­ing civ­i­liza­tion from just a few mil­lion peo­ple. If we think the in­verse re­la­tion­ship be­tween pop­u­la­tion size and hingey­ness is very strong, then maybe we should be sav­ing for such a pos­si­ble sce­nario; that’s the hinge mo­ment.

  For the later sce­nar­ios here you’re deal­ing with much larger pop­u­la­tions. If the plau­si­bil­ity of im­por­tant lock-in is similar for so­lar coloniza­tion and in­ter­galac­tic coloniza­tion eras, but the pop­u­la­tion of the lat­ter is billions of times greater, it doesn’t seem to be at all an op­tion that it could be the most HoH pe­riod on a per re­source unit ba­sis.

  I agree that other things be­ing equal a time with a smaller pop­u­la­tion (or: smaller to­tal re­sources) seems like­lier to be a more in­fluen­tial time. But ‘doesn’t seem to be at all an op­tion’ seems over­stated to me.

  Sim­ple case: con­sider a world where there just aren’t op­tions to in­fluence the very long-run fu­ture. (Agents can make short-run per­tur­ba­tions but can’t af­fect long-run tra­jec­to­ries; some sort of his­tor­i­cal de­ter­minism is true). Then the most in­fluen­tial time is just when we have the best knowl­edge of how to turn re­sources into short-run util­ity, which is pre­sum­ably far in the fu­ture.

  Or, more im­por­tantly, where hingey­ness is es­sen­tially 0 up un­til a cer­tain point far in the fu­ture. If our abil­ity to pos­i­tively in­fluence the very long-run fu­ture were no bet­ter than a dart-throw­ing chimp un­til we’ve got com­put­ers the size of so­lar sys­tems, then the most in­fluen­tial times would also in­volve very high populations

  More gen­er­ally, per-re­source hingey­ness in­creases with:

  • Availa­bil­ity of pivotal mo­ments one can in­fluence, and their pivotal­ity

  • Knowl­edge /​ un­der­stand­ing of how to pos­i­tively in­fluence the long-run future

  And hingey­ness de­creases with:

  • Pop­u­la­tion size

  • Level of ex­pen­di­ture on long-term influence

  • Chance of be­ing pre-empted already

  If knowl­edge or availa­bil­ity of pivotal mo­ments at a time is 0, then hingey­ness at the time is 0, and lower pop­u­la­tions can’t out­weigh that.

• William_MacAskill 5 Sep 2019 3:31 UTC

  20 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  I think this over­states the case. Diminish­ing re­turns to ex­pen­di­tures in a par­tic­u­lar time fa­vor a nonzero dis­burse­ment rate (e.g. with log­a­r­ith­mic re­turns to spend­ing at a given time 10x HoH lev­els would drive a 10x ex­pen­di­ture for a given pe­riod)

  Sorry, I wasn’t mean­ing we should be en­tirely punt­ing to the fu­ture, and in case it’s not clear from my post my ac­tual all-things-con­sid­ered views is that longter­mist EAs should be en­dors­ing a mixed strat­egy of some sig­nifi­cant pro­por­tion of effort spent on near-term longter­mist ac­tivi­ties and some pro­por­tion of effort spent on long-term longter­mist ac­tivi­ties.

  I do agree that, at the mo­ment, EA is mainly in­vest­ing (e.g. be­cause of Open Phil and be­cause of hu­man cap­i­tal and be­cause much ac­tual ex­pen­di­ture is field-build­ing-y, as you say). But it seems like at the mo­ment that’s pri­mar­ily be­cause of man­age­ment con­straints and weird­ness of bor­row­ing-to-give (etc), rather than a prin­ci­pled plan to spread giv­ing out over some (pos­si­bly very long) time pe­riod. Cer­tainly the vibe in the air is ‘ex­pen­di­ture (of money or labour) now is su­per im­por­tant, we should re­ally be fo­cus­ing on that’.

  (I also don’t think that diminish­ing re­turns is en­tirely true: there are fixed costs and economies of scale when try­ing to do most things in the world, so I ex­pect s-curves in gen­eral. If so, that would favour a lumpier dis­burse­ment sched­ule.)

• William_MacAskill 5 Sep 2019 3:28 UTC

  5 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  I would note that the cre­ation of nu­mer­ous simu­la­tions of HoH-type pe­ri­ods doesn’t re­duce the to­tal im­pact of the ac­tual HoH folk

  Agree that it might well be that even though one has a very low cre­dence in HoH, one should still act in the same way. (e.g. be­cause if one is not at HoH, one is a sim, and your ac­tions don’t have much im­pact).

  The sim-arg could still cause you to change your ac­tions, though. It’s some­what plau­si­ble to me, for ex­am­ple, that the chance of be­ing a sim if you’re at the very most mo­men­tous time is 1000x higher than the chance of be­ing a sim if you’re at the 20th most hingey time, but the most hingey time is not 1000x more hingey than the 20th most hingey time. In which case the hy­poth­e­sis that you’re at the 20th most hingey time has a greater rel­a­tive im­por­tance than it had be­fore.

• William_MacAskill 5 Sep 2019 3:27 UTC

  16 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  I agree we are learn­ing more about how to effec­tively ex­ert re­sources to af­fect the fu­ture, but if your defi­ni­tion is con­cerned with the effect of a marginal in­cre­ment of re­sources (rather than the to­tal ca­pac­ity of an era), then you need to wres­tle with the is­sue of diminish­ing re­turns.

  I agree with this, though if we’re un­sure about how many re­sources will be put to­wards longter­mist causes in the fu­ture, then the ex­pected value of sav­ing will come to be dom­i­nated by the sce­nario where very few re­sources are de­voted to it. (As hap­pens in the Ram­sey model for dis­count­ing if one in­cludes un­cer­tainty over fu­ture growth rates and the pos­si­bil­ity of catas­tro­phe.) This con­sid­er­a­tions gets stronger if one thinks the diminish­ing marginal re­turns curve is very steep.

  E.g. per­haps in 150 years’ time, EA and Open Phil and longter­mist con­cern will be dust; in which case those who saved for the fu­ture (and en­sured that there would be at least some suffi­ciently like­minded peo­ple to pass their re­sources onto) will have an out­sized re­turn. And per­haps re­turns diminish re­ally steeply, so that what mat­ters is guaran­tee­ing that there are at least some longter­mists around. If the out­sized re­turn in this sce­nario if large enough, then even a low prob­a­bil­ity of this sce­nario might be the dom­i­nant con­sid­er­a­tion.

  Found­ing fields like AI safety or pop­u­la­tion ethics is much bet­ter on a per cap­ita ba­sis than ex­pand­ing them by 1% af­ter they have de­vel­oped more.

  Strongly agree, though by in­duc­tion it seems we should think there will be more such fields in the fu­ture.

  The longter­mist of 1600 would in­deed have mostly ‘in­vested’ in build­ing a move­ment and even­tu­ally in things like fi­nan­cial as­sets when move­ment-build­ing re­turns fell be­low fi­nan­cial re­turns, but they also should have made con­crete in­ter­ven­tions like caus­ing the lev­er­aged growth of in­sti­tu­tions like sci­ence and the En­light­en­ment that looked to have a fair chance of con­tribut­ing to HoH sce­nar­ios over the com­ing cen­turies, and those could have paid off.

  You might think the coun­ter­fac­tual is un­fair here, but I wouldn’t re­gard it as ac­cessible to some­one in 1600 to know that they could make con­tri­bu­tions to sci­ence and the En­light­en­ment as a good way of in­fluenc­ing the long-run fu­ture.

  This is analo­gous to the gen­eral point in fi­nan­cial mar­kets that as­sets classes with sys­tem­at­i­cally high re­turns only have them be­fore those re­turns are widely agreed on to be valuable and ac­cessible...

  A world in which ev­ery­one has shared cor­rect val­ues and strong knowl­edge of how to im­prove things is one in which marginal longter­mist re­sources are gild­ing the lily.

  Though if we’re re­ally clue­less right now (per­haps not much bet­ter than the per­son in 1600) then per­haps that’s the best we can do.

  And it would seem that the re­ally high-value sce­nario is where (i) knowl­edge is very high but (ii) con­cern for the very long-run fu­ture is very low (but not nonex­is­tent, al­low­ing for re­sources to be passed onto those times.)

  In terms of the fi­nan­cial anal­ogy, that would be like how some­one with strange prefer­ences, who gets ex­traor­di­nary util­ity from eat­ing bread and pota­toes, gets a much higher re­turn (when mea­sured in util­ity gained) from a reg­u­lar salary than other peo­ple would.

  And in gen­eral I’m more in­clined to be­lieve sto­ries of us hav­ing ex­traor­di­nary im­pact if that pri­mar­ily re­sults from a differ­ence in what we care about com­pared with oth­ers, rather than from hav­ing greater in­sight.

  
  

  I will say, though: the ar­gu­ment “we’re at an un­usual pe­riod where longter­mist (/​im­par­tial con­se­quen­tial­ish) con­cern is very low but not nonex­is­tent” as a rea­son for now be­ing a par­tic­u­larly in­fluen­tial time seems pretty good to me, and wasn’t one that I in­cluded in my list of ar­gu­ments in favour of HoH.

• William_MacAskill 5 Sep 2019 3:18 UTC

  14 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  To talk about what they would have been one needs to con­sider a coun­ter­fac­tual in which we anachro­nis­ti­cally in­tro­duce at least some min­i­mal ver­sion of longter­mist al­tru­ism, and what one in­cludes in that in­ter­ven­tion will af­fect the re­sult one ex­tracts from the ex­er­cise.

  I agree there’s a tricky is­sue of how ex­actly one con­structs the coun­ter­fac­tual. The defi­ni­tion I’m us­ing is try­ing to get it as close as pos­si­ble to a coun­ter­fac­tual we re­ally face: how much to spend now vs how much to pass re­sources onto fu­ture al­tru­ists. I’d be in­ter­ested if oth­ers thought of very differ­ent ap­proaches. It’s pos­si­ble that I’m try­ing to pack too much into the con­cept of ‘most in­fluen­tial’, or that this con­cept should be kept sep­a­rate from the idea of mov­ing re­sources around to differ­ent times.

  I feel that in­volv­ing the anachro­nis­tic in­ser­tion of a longter­mist al­tru­ist into the past, if any­thing, makes my ar­gu­ment harder to make, though. If I can’t guaran­tee that the past per­son I’m giv­ing re­sources to would even be a longter­mist, that makes me less in­clined to give them re­sources. And if I in­clude the pos­si­bil­ity that longter­mism might be wrong and that the fu­ture-per­son that I pass re­sources onto will recog­nise this, that’s (at least some) ar­gu­ment to me in favour of pass­ing on re­sources. (Caveat sub­jec­tivist meta-ethics, pos­si­bil­ity of fu­ture peo­ple’s moral­ity go­ing way­ward, etc.)

• William_MacAskill 5 Sep 2019 3:16 UTC

  7 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  I would dis­pute this. Pos­si­bil­ities of AGI and global dis­aster were dis­cussed by pi­o­neers like Tur­ing, von Neu­mann, Good, Min­sky and oth­ers from the found­ing of the field of AI.

  Thanks, I’ve up­dated on this since writ­ing the post and think my origi­nal claim was at least too strong, and prob­a­bly just wrong. I don’t cur­rently have a good sense of, say, if I were liv­ing in the 1950s, how likely I would be to figure out AI as the thing, rather than fo­cus on some­thing else that turned out not to be as im­por­tant (e.g. the fo­cus on nan­otech by the Fore­sight In­sti­tute (a group of ideal­is­tic fu­tur­ists) in the late 80s could be a rele­vant ex­am­ple).

• William_MacAskill 5 Sep 2019 3:13 UTC

  21 points

  in reply to: Carl_Shulman's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Hi Carl,

  Thanks so much for tak­ing the time to write this ex­cel­lent re­sponse, I re­ally ap­pre­ci­ate it, and you make a lot of great points. I’ll di­vide up my re­ac­tions into differ­ent com­ments; hope­fully that helps ease of read­ing.

  I’d like to flag that I would re­ally like to see a more el­e­gant term than ‘hingey­ness’ be­come stan­dard for refer­ring to the ease of in­fluence in differ­ent pe­ri­ods.

  This is a good idea. Some op­tions: in­fluen­tial­ness; crit­i­cal­ity; mo­men­tous­ness; im­por­tance; pivotal­ity; sig­nifi­cance.

  I’ve cre­ated a straw poll here to see as a first pass what the Fo­rum thinks.

  [Edit: Re­sults:

  Pivotal­ity − 26% (17 votes)

  Crit­i­cal­ity − 22% (14 votes)

  Hingey­ness − 12% (8 votes)

  In­fluen­tial­ness − 11% (7 votes)

  Im­por­tance − 11% (7 votes)

  Sig­nifi­cance − 11% (7 votes)

  Mo­men­tous­ness − 8% (5 votes)]

• William_MacAskill 4 Sep 2019 4:59 UTC

  5 points

  in reply to: Stefan_Schubert's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Thanks—I agree that this dis­tinc­tion is not as crisp as would be ideal. I’d see re­li­gion-spread­ing, and move­ment-build­ing, as in prac­tice al­most always a mixed strat­egy: in part one is giv­ing re­sources to fu­ture peo­ple, and in part one is also di­rectly al­ter­ing how the fu­ture goes.

  But it’s more like buck-pass­ing than it is like di­rect work, so I think I should just not in­clude the Ax­ial age in the list of par­tic­u­larly in­fluen­tial times (given my defi­ni­tion of ‘in­fluen­tial’).

• William_MacAskill 4 Sep 2019 4:56 UTC

  5 points

  in reply to: lukeprog's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Huh, thanks for the great link! I hadn’t seen that be­fore, and had been un­der the im­pres­sion that though some peo­ple (e.g. Good, Tur­ing) had sug­gested the in­tel­li­gence ex­plo­sion, no-one re­ally wor­ried about the risks. Looks like I was just wrong about that.

• William_MacAskill 4 Sep 2019 4:56 UTC

  8 points

  in reply to: Pablo_Stafforini's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  Agreed, good point; I was think­ing just of the case where you re­duce ex­tinc­tion risk in one pe­riod but not in oth­ers.

  I’ll note, though, that re­duc­ing ex­tinc­tion risk at all fu­ture times seems very hard to do. I can imag­ine, if we’re close to a val­ues lock-in point, we could shift so­cietal val­ues such that they care about fu­ture ex­tinc­tion risk much more than they would oth­er­wise have done. But if that’s the path­way, then the Time of Per­ils view wouldn’t provide an ar­gu­ment for HoH in­de­pen­dent of the Value Lock-In view.