Linch

• Linch 25 Oct 2019 8:00 UTC

  7 points

  in reply to: Linch's comment on: What are your top pa­pers of the 2010s?

  The de­gree that EA thought re­lies on cut­ting-edge* re­search in eco­nomics, philos­o­phy, etc, from the last 10 years is kinda sur­pris­ing if you think about it.

  It’s kinda weird that not just Su­per­in­tel­li­gence but also Poor Eco­nomics, Com­pas­sion by the Pound, in­for­ma­tion haz­ards, unilat­er­al­ist’s curse, and other things we just kinda as­sume to be “in the wa­ter sup­ply” rely mostly on ar­gu­ments or re­search that’s not even a decade old!

  
  

  *the less po­lite way to put it is “likely to be over­turned” :P

[Question] Is there a clear writeup sum­ma­riz­ing the ar­gu­ments for why deep ecol­ogy is wrong?

• Linch 25 Oct 2019 7:31 UTC

  11 points

  on: What are your top pa­pers of the 2010s?

  I talked about it be­fore sev­eral times, but the biggest one is:

  The Pos­si­bil­ity of an On­go­ing Mo­ral Catas­tro­phe by Evan G. Willi­ams, which I sum­ma­rized here.

  Other than that, in philos­o­phy mostly stuff by Bostrom:

  The Unilat­er­al­ist’s Curse

  In­for­ma­tion Hazards

  (Also flag­ging Will’s work on moral un­cer­tainty, though it’s un­clear to me that his PhD the­sis is the best pre­sen­ta­tion)

  In CS:

  Ad­ver­sar­ial Ex­am­ples are Not Bugs, They Are Fea­tures by Ilyas et.al. (makes clear some­thing I sus­pected for a while about that topic)

  World Models by Ha and Sch­mid­hu­ber

  (Those two pa­pers are far from the most in­fluen­tial ML pa­pers in the last decade! But I usu­ally learn ML from video lec­tures/​blog posts/​talk­ing to peo­ple rather than pa­pers)

  (Prob­a­bly also var­i­ous AI Safety stuff, though no spe­cific pa­per comes to mind).

  De­sign­ing Data-In­ten­sive Ap­pli­ca­tions quoted a ton of pa­pers (that I did not read).

  In Eco­nomics:

  The aca­demic text­book Com­pas­sion by the Pound.

  Poor Eco­nomics (which won the 2019 No­bel Prize!)

  Meta*:

  Com­ment on ‘The aes­ti­va­tion hy­poth­e­sis for re­solv­ing Fermi’s para­dox’

  Does suffer­ing dom­i­nate en­joy­ment in the an­i­mal king­dom?

  *(the re­search/​ar­gu­ments weren’t di­rectly de­ci­sion-rele­vant for me, but the fact that they over­turned some­thing a lot of EAs be­lieved to be true were a use­ful meta-up­date)

• Linch 15 Oct 2019 6:01 UTC

  7 points

  in reply to: G Gordon Worley III's comment on: What ac­tions would ob­vi­ously de­crease x-risk?

  This is the only an­swer here I’m mod­er­ately con­fi­dent is cor­rect. A pity the EV is so low!

• Linch 19 Sep 2019 0:28 UTC

  28 points

  on: Linch’s Shortform

  cross-posted from Face­book.

  Some­times I hear peo­ple who cau­tion hu­mil­ity say some­thing like “this ques­tion has stumped the best philoso­phers for cen­turies/​mil­len­nia. How could you pos­si­bly hope to make any progress on it?”. While I con­cur that hu­mil­ity is fre­quently war­ranted and that in many spe­cific cases that in­junc­tion is rea­son­able [1], I think the fram­ing is broadly wrong.
  
  
  In par­tic­u­lar, us­ing ge­ologic time rather than an­thro­polog­i­cal time hides the fact that there prob­a­bly weren’t that many peo­ple ac­tively think­ing about these is­sues, es­pe­cially care­fully, in a sus­tained way, and mak­ing sure to build on the work of the past. For back­ground, 7% of all hu­mans who have ever lived are al­ive to­day, and liv­ing peo­ple com­pose 15% of to­tal hu­man ex­pe­rience [2] so far!!!
  
  
  It will not sur­prise me if there are about as many liv­ing philoso­phers to­day as there were dead philoso­phers in all of writ­ten his­tory.
  
  
  For some spe­cific ques­tions that par­tic­u­larly in­ter­est me (eg. pop­u­la­tion ethics, moral un­cer­tainty), the to­tal re­search work done on these ques­tions is gen­er­ously less than five philoso­pher-life­times. Even for clas­si­cal age-old philo­soph­i­cal dilem­mas/​”grand pro­jects” (like the hard prob­lem of con­scious­ness), to­tal work spent on them is prob­a­bly less than 500 philoso­pher-life­times, and quite pos­si­bly less than 100.
  
  
  There are also solid out­side-view rea­sons to be­lieve that the best philoso­phers to­day are just much more com­pe­tent [3] than the best philoso­phers in his­tory, and have ac­cess to much more re­sources[4].
  
  
  Fi­nally, philos­o­phy can build on progress in nat­u­ral and so­cial sci­ences (eg, com­put­ers, game the­ory).
  
  
  Spec­u­lat­ing fur­ther, it would not sur­prise me, if, say, a par­tic­u­larly thorny and deeply im­por­tant philo­soph­i­cal prob­lem can effec­tively be solved in 100 more philoso­pher-life­times. As­sum­ing 40 years of work and $200,000/​year per philoso­pher, in­clud­ing over­head, this is ~800 mil­lion, or in the same bal­l­park as the cost of de­vel­op­ing a sin­gle drug[5].
  
  
  Is this worth it? Hard to say (es­pe­cially with such made-up num­bers), but the fea­si­bil­ity of solv­ing seem­ingly in­tractable prob­lems no longer seems crazy to me.
  
  
  [1] For ex­am­ple, in­tro philos­o­phy classes will of­ten ask stu­dents to take a strong po­si­tion on ques­tions like de­on­tol­ogy vs. con­se­quen­tial­ism, or de­ter­minism vs. com­pat­i­bil­ism. Ba­sic epistemic hu­mil­ity says it’s un­likely that col­lege un­der­grads can get those ques­tions right in such a short time.
  
  
  [2] https://​​eu­kary­otewrites­blog.com/​​2018/​​10/​​09/​​the-fun­nel-of-hu­man-ex­pe­rience/​​
  
  
  [3] Flynn effect, ed­u­ca­tion, and ed­u­ca­tion of women, among oth­ers. Also, just https://​​en.wikipe­dia.org/​​wiki/​​Athe­nian_democ­racy#Size_and_make-up_of_the_Athe­nian_pop­u­la­tion. (Roughly as many ed­u­cated peo­ple in all of Athens at any given time as a fairly large state uni­ver­sity). Modern peo­ple (or at least peak perform­ers) be­ing more com­pe­tent than past ones is blatantly ob­vi­ous in other fields where pri­or­ity is less im­por­tant (eg, marathon run­ners, chess).
  
  
  [4] Eg, in­ter­net, cheap books, wide­spread liter­acy, and the cur­rent in­tel­lec­tual world is prac­ti­cally mono­lin­gual.
  
  
  [5] https://​​en.wikipe­dia.org/​​wiki/​​Cost_of_drug_development

Linch’s Shortform

• Linch 6 Sep 2019 23:54 UTC

  10 points

  in reply to: Toby_Ord's comment on: Are we liv­ing at the most in­fluen­tial time in his­tory?

  >> And it gets even more so when you run it in terms of per­sons or per­son years (as I be­lieve you should). i.e. mea­sure time with a clock that ticks as each life­time ends, rather than one that ticks each sec­ond. e.g. about 1/​20th of all peo­ple who have ever lived are al­ive now, so the next cen­tury it is not re­ally ¹⁄₂,000th of hu­man his­tory but more like 1/​20th of it.

  And if you use per­son-years, you get some­thing like ¹⁄₇ − 1/​14! [1]

  >> I doubt I can eas­ily con­vince you that the prior I’ve cho­sen is ob­jec­tively best, or even that it is bet­ter than the one you used. Prior-choice is a bit of an art, rather like choice of ax­ioms.

  I’m pretty con­fused about how these dra­mat­i­cally differ­ent pri­ors are formed, and would re­ally ap­pre­ci­ate it if some­body (maybe some­body less busy than Will or Toby?) could give poin­t­ers on how to read up more on form­ing these sort of pri­ors. As you al­lude to, this ques­tion seems to map to an­throp­ics, and I’m cu­ri­ous how much the pri­ors here nec­es­sar­ily maps to your views on an­throp­ics. Eg, am I read­ing the post and your com­ment cor­rectly that Will takes an SIA view and you take an SSA view on an­thropic ques­tions?

  In gen­eral, does any­body have poin­t­ers on how best to rea­son about an­thropic and an­thropic-ad­ja­cent ques­tions?

  [1] https://​​eu­kary­otewrites­blog.com/​​2018/​​10/​​09/​​the-fun­nel-of-hu­man-ex­pe­rience/​​

• Linch 12 Aug 2019 4:52 UTC

  1 point

  in reply to: jpaddison's comment on: What book(s) would you want a gifted teenager to come across?

  One of the most im­pact­ful pur­chases I’ve ever made! :P

• Linch 7 Aug 2019 7:30 UTC

  10 points

  on: What book(s) would you want a gifted teenager to come across?

  En­der’s Game by Or­son Scott Card re­ally spoke to me as a kid, though hope­fully your stu­dents are bet­ter so­cial­ized! :P

• Linch 7 Aug 2019 7:29 UTC

  4 points

  on: What book(s) would you want a gifted teenager to come across?

  The Sig­nal and the Noise by Nate Silver (of 538 fame) is the best and most read­able in­tro­duc­tion to Bayesian statis­tics and Bayesian rea­son­ing that I’m aware of.

• Linch 7 Aug 2019 7:27 UTC

  1 point

  on: What book(s) would you want a gifted teenager to come across?

  Dairy of a Mad­man by Lu Xun was helpful for me in cul­ti­vat­ing a strong sense of dis­satis­fac­tion with the way things are and the im­plicit or ex­plicit rules that gov­ern so­cial re­al­ity.

  I don’t know if there are any good trans­la­tions though.

• Linch 7 Aug 2019 7:08 UTC

  1 point

  in reply to: Aidan O'Gara's comment on: What book(s) would you want a gifted teenager to come across?

  Re Poor Eco­nomics:

  I still re­mem­ber the ex­per­i­ments in (I think) In­dia where they demon­strated that even for peo­ple liv­ing in ex­treme poverty, where most of marginal spend­ing goes to food, in­creased in­come fre­quently re­sulted in peo­ple buy­ing bet­ter-tast­ing calories, not just more calories. A+.

• Linch 7 Aug 2019 7:03 UTC

  2 points

  in reply to: RyanCarey's comment on: What book(s) would you want a gifted teenager to come across?

  I thought Chi­ang was un­usu­ally high in liter­ary merit, but what do you think is the rele­vance to EA?

• Linch 7 Aug 2019 7:02 UTC

  1 point

  in reply to: jpaddison's comment on: What book(s) would you want a gifted teenager to come across?

  Strongly sec­onded. Both had a large effect on me, es­pe­cially Famine, Affluence and Mo­ral­ity when I was a teenager.

• Linch 7 Aug 2019 6:48 UTC

  1 point

  in reply to: Ikaxas's comment on: The Pos­si­bil­ity of an On­go­ing Mo­ral Catas­tro­phe (Sum­mary)

  For #2, Ide­olog­i­cal Tur­ing Tests could be cool too.

• Linch 4 Aug 2019 7:24 UTC

  1 point

  in reply to: Ikaxas's comment on: The Pos­si­bil­ity of an On­go­ing Mo­ral Catas­tro­phe (Sum­mary)

  You may also like our dis­cus­sion sheets for this topic:

  https://​​drive.google.com/​​drive/​​u/​​1/​​fold­ers/​​0B3C8dkkHYGfqeXh1SE5kLVJPdGM

• Linch 4 Aug 2019 3:24 UTC

  1 point

  in reply to: Ikaxas's comment on: The Pos­si­bil­ity of an On­go­ing Mo­ral Catas­tro­phe (Sum­mary)

  Sure! In gen­eral you can as­sume that any­thing I write pub­li­cly is freely available for aca­demic pur­poses. I’d also be in­ter­ested in see­ing the syl­labus if/​when you end up de­sign­ing it.

• Linch 3 Aug 2019 2:05 UTC

  2 points

  in reply to: nonzerosum's comment on: How ur­gent are ex­treme cli­mate change risks?

  Mes­saged. Will share more widely if/​when it’s ready for prime time. :)

• Linch 3 Aug 2019 2:02 UTC

  1 point

  in reply to: aarongertler's comment on: The Pos­si­bil­ity of an On­go­ing Mo­ral Catas­tro­phe (Sum­mary)

  We ab­solutely wel­come sum­maries! Peo­ple get­ting more ways to ac­cess good ma­te­rial is one of the rea­sons the Fo­rum ex­ists.

  Yay!

  did you con­sider copy­ing the sum­mary into a Fo­rum post, rather than link­ing it?

  Yes. I did a lot of non-stan­dard for­mat­ting tricks in Google Docs when I first wrote it (be­cause I wasn’t ex­pect­ing to ever need to port it over to a differ­ent for­mat). So when I first tried to copy it over, the whole thing looked dis­as­trously un­read­able.

  Changed the ti­tle. :)

• Linch 2 Aug 2019 22:28 UTC

  3 points

  on: ‘Longter­mism’

  Great post!

  In gen­eral, if I imag­ine ‘longter­mism’ tak­ing off as a term, I imag­ine it get­ting a lot of sup­port if it des­ig­nates the first con­cept, and a lot of push­back if it des­ig­nates the sec­ond con­cept. It’s also more in line with moral ideas and so­cial philoso­phies that have been suc­cess­ful in the past: en­vi­ron­men­tal­ism claims that pro­tect­ing the en­vi­ron­ment is im­por­tant, not that pro­tect­ing the en­vi­ron­ment is (always) the most im­por­tant thing; fem­i­nism claims that up­hold­ing women’s rights is im­por­tant, not that do­ing so is (always) the most im­por­tant thing. I strug­gle to think of ex­am­ples where the philos­o­phy makes claims about some­thing be­ing the most im­por­tant thing, and in­so­far as I do (to­tal­i­tar­ian marx­ism and fas­cism are ex­am­ples that leap to mind), they aren’t the sort of philoso­phies I want to em­u­late.

  Maybe this is the wrong refer­ence class, but I can think of sev­eral oth­ers: util­i­tar­i­anism, Chris­ti­an­ity, con­se­quen­tial­ism, where the “strong” defi­ni­tion is the most nat­u­ral that comes to mind.

  Ie, a naive in­ter­pre­ta­tion of Chris­tian philos­o­phy is that fol­low­ing the word of God is the most im­por­tant thing (not just one im­por­tant thing among many). Similarly, util­i­tar­i­ans would usu­ally con­sider max­i­miz­ing util­ity to be the most im­por­tant thing, con­se­quen­tial­ists would prob­a­bly con­sider con­se­quences to be more im­por­tant than other moral du­ties, etc.