Jsevillamol

• Jsevillamol 16 Sep 2020 16:52 UTC

  7 points

  in reply to: RyanCarey’s comment on: Some thoughts on EA out­reach to high schoolers

  Without en­ter­ing into too many sen­si­tive de­tails, when I have looked at the out­put of similar pro­grams I have no­ticed that I was ex­cited about the ca­reer path of 1 out of ev­ery 3 par­ti­ci­pants.

  But a) I dont know how much of it was coun­ter­fac­tual, b) when I made the es­ti­ma­tion I had an in­cen­tive to pro­duce an op­ti­mistic an­swer and c) it re­lies on my sub­jec­tive judge­ment, which you may not trust.

  Also worth not­ing that I think that the raw con­ver­sion rate is not the right met­ric to fo­cus on—the out­liers usu­ally ac­count for most the im­pact of these pro­grams.

• Jsevillamol 16 Sep 2020 14:47 UTC

  1 point

  in reply to: NunoSempere’s comment on: Quan­tum com­put­ing timelines

  The cita­tion is a link: (Grace, 2020)

  Just in case: https://​​aiim­pacts.org/​​dis­con­tin­u­ous-progress-in-his­tory-an-up­date/​​

• Jsevillamol 16 Sep 2020 11:16 UTC

  5 points

  in reply to: irving’s comment on: Quan­tum com­put­ing timelines

  It is not in­tended to be a cal­ibrated es­ti­mated, though we were hop­ing that it could help oth­ers make cal­ibrated es­ti­ma­tions.

  The ways that a cal­ibrated es­ti­mate would differ in­clude:

  1. The re­sult is a con­fi­dence in­ter­val, not a cre­dence in­ter­val (most places in the pa­per where it says prob­a­bil­ity is should say con­fi­dence, I apol­o­gize for the over­sight), so your choice of prior can make a big differ­ence to the as­so­ci­ated cre­dence in­ter­val.

  2. The model is as­sum­ing that no dis­con­tin­u­ous progress will hap­pen, but we do not know whether this will hold. (Grace, 2020) es­ti­mates a yearly rate of dis­con­ti­nous break­throughs on any given tech­nol­ogy of 0.1%, so I’d naively ex­pect a 1-(1-0.1%)^20 = 2% chance that there is such a dis­con­tin­u­ous break­through for quan­tum com­put­ing in the next 20 years.

  3. The model makes op­ti­mistic as­sump­tions of progress—namely that a) the rate of ex­po­nen­tial progress will hold for both the phys­i­cal qubit count and the gate er­ror rate, b) there is no cor­re­la­tion be­tween the met­rics in a sys­tem (which we show it is prob­a­bly an op­ti­mistic as­sump­tion, since it is eas­ier to op­ti­mize only one of the met­rics than both) and c) we ig­nore the is­sue of qubit con­nec­tivity due to lack of data and mod­el­ling difficulty.

  If I was pressed to put a cre­dence bound on it, I’d as­sign about 95% chance that EITHER the model is ba­si­cally cor­rect OR that the timelines are slower than ex­pected (most likely if the ex­po­nen­tial trend of progress on gate er­ror rate does not hold in the next 20 years), for an up­per bound on the prob­a­bil­ity that we will have RSA 2048 quan­tum at­tacks by 2040 of <5% + 95% 5% ~= 10%.

  Either case, I think that the model should make us puz­zle over the ex­pert timelines, and in­quire whether they are tak­ing into ac­count any ex­tra in­for­ma­tion or be­ing too op­ti­mistic.

  
  

  EDIT: I made an ar­tih­metic mis­take, now cor­rected (thanks to Eric Martin for point­ing it out)

Quan­tum com­put­ing timelines

• Jsevillamol 7 Aug 2020 9:34 UTC

  7 points

  on: What are some low-in­for­ma­tion pri­ors that you find prac­ti­cally use­ful for think­ing about the world?

  In a con­text where mul­ti­ple fore­casts have already been made (by you or other peo­ple), use the ge­o­met­ric mean of the odds as a blind ag­gre­gate:

  $$\hat{p}=\frac{{(\prod \frac{p_i}{1-p_i})}^{\frac{1}{n}}}{1+{(\prod \frac{p_i}{1-p_i})}^{\frac{1}{n}}}$$

  If you want to get fancy, use an ex­trem­ized ver­sion of this pool­ing method, by scal­ing the log odds by a fac­tor $a$ :

  $$\hat{p}=\frac{{(\prod \frac{p_i}{1-p_i})}^{\frac{a}{n}}}{1+{(\prod \frac{p_i}{1-p_i})}^{\frac{a}{n}}}$$

  Satopaa et al have found that in prac­tice $a\in [1.161,3.921]$ gives the best re­sults.

• Jsevillamol 24 Jul 2020 10:37 UTC

  1 point

  in reply to: PabloAMC’s comment on: Assess­ing the im­pact of quan­tum cryptanalysis

  I think we broadly agree.

  I be­lieve that chem­istry and ma­te­rial sci­ence are two ap­pli­ca­tions where quan­tum com­put­ing might be a use­ful tool, since simu­lat­ing very sim­ple phys­i­cal sys­tems is some­thing where a quan­tum com­puter ex­cels at but ar­guably sig­nifi­cantly slower to do in a clas­si­cal com­puter.

  On the other hand, peo­ple more versed on ma­te­rial sci­ence and chem­istry I talked to seemed to be­lieve that (1) clas­si­cal ap­prox­i­ma­tions will be good enough to ap­proach prob­lems in these ar­eas and (2) in silico de­sign is not a huge bot­tle­neck any­way.

  So I am open to a quan­tum com­put­ing rev­olu­tion in chem­istry and ma­te­rial sci­ence, but mod­er­ately skep­ti­cal.

  
  

  Sum­ma­riz­ing my cur­rent be­liefs about how im­por­tant quan­tum com­put­ing will be for fu­ture ap­pli­ca­tions:

  • Cryp­to­anal­y­sis ⇒ very im­por­tant for solv­ing a hand­ful of prob­lems rele­vant for mod­ern se­cu­rity, with no plau­si­ble alternative

  • Chem­istry and ma­te­rial sci­ence ⇒ Plau­si­bly use­ful, not rev­olu­tion­ary.

  • AI and op­ti­miza­tion ⇒ un­likely to be use­ful, huge con­straints to overcome

  • Biol­ogy and medicine ⇒ not use­ful, sys­tems too com­plex to model

• Jsevillamol 23 Jul 2020 11:34 UTC

  4 points

  in reply to: irving’s comment on: Assess­ing the im­pact of quan­tum cryptanalysis

  Thank you so much for your kind words and juicy feed­back!

  Google has already de­ployed post-quan­tum schemes as a test

  I did not know about this, and this ac­tu­ally up­dates me on how much over­head will be needed for post quan­tum crypto (the NIST ex­pert I in­ter­viewed gave me an im­pres­sion that it was large and es­sen­tially would need spe­cial­ized hard­ware to meet perfor­mance ex­pec­ta­tions, but this seems to speak to the con­trary (?))

  here may be sig­nifi­cant eco­nomic costs due to pub­lic key schemes de­ployed “at rest”

  To make sure I un­der­stand your point, let me try to para­phase. You are point­ing out that:

  1) past com­mu­ni­ca­tions that are recorded will be ren­dered in­se­cure by quan­tum computing

  2) there are some tran­si­tion costs as­so­ci­ated with post quan­tum crypto—which are re­lated to for ex­am­ple the cost of re­build­ing PGP cer­tifi­cate net­works.

  If so, I agree that this is a rele­vant con­sid­er­a­tion but does not change the bot­tom line.

  Thank you again for read­ing my pa­per!

• Jsevillamol 22 Jul 2020 15:05 UTC

  13 points

  in reply to: David_Kristoffersson’s comment on: Assess­ing the im­pact of quan­tum cryptanalysis

  Note that we be­lieve that quan­tum supremacy has already been achieved.

  As in, the quan­tum com­puter Sy­camore from Google is ca­pa­ble of solv­ing a (toy) prob­lem that we cur­rently be­lieve un­fea­si­ble in a clas­si­cal com­puter.

  Of course, there is a more in­ter­est­ing ques­tion of when will we be able to solve prac­ti­cal prob­lems us­ing quan­tum com­put­ing. Ex­perts be­lieve that the me­dian for a prac­ti­cal at­tack on mod­ern crypto is ~2035.

  I re­gard­less be­lieve that out­side (and ar­guably within) quan­tum crypt­anal­y­sis the ap­pli­ca­tions will be fairly limited.

  The pa­per in my post goes in more de­tail about this.

Assess­ing the im­pact of quan­tum cryptanalysis

My ex­pe­rience as a CLR grantee and vis­it­ing re­searcher at CSER

• Jsevillamol 11 Feb 2020 10:21 UTC

  7 points

  on: Up­date on civ­i­liza­tional col­lapse research

  I’m cur­rently work­ing on ex­pli­cat­ing some of these fac­tors, but some ex­am­ples would be dras­tic cli­mate change, long-lived ra­dionu­clides, in­crease in per­sis­tent pathogens.

  Can you ex­plain the bit about long-lived ra­dionu­clides?

  How would they be pro­duced? How would they af­fect “tech­nolog­i­cal car­ry­ing ca­pac­ity”?

Model­ling Van­tage Points

Quan­tum Com­put­ing : A pre­limi­nary re­search anal­y­sis report

• Jsevillamol 9 Oct 2019 9:46 UTC

  15 points

  on: [WIP] Sum­mary Re­view of ITN Critiques

  Thank you for writ­ing this up—always good to see crit­i­cism of key ideas.

  I want to con­test point 4.

  The fact that we can de­com­pose “Good done /​ ex­tra per­son or $” into three fac­tors that can be roughly in­ter­preted as Scale, Tractabil­ity and Ne­glect­ed­ness is not a prob­lem, but a de­sir­able prop­erty.

  In ul­ti­mate in­stance, we want to eval­u­ate marginal cost effec­tive­ness ie “Good done /​ ex­tra per­son or $”. How­ever this is difficult, so we want to split it up in sim­pler terms.

  The math­e­mat­i­cal equa­tion that de­com­poses the cost serves as a guaran­tee that by es­ti­mat­ing all three fac­tors we will not be leav­ing any­thing im­por­tant be­hind.

My ex­pe­rience on a sum­mer re­search programme

• Jsevillamol 6 Sep 2019 17:05 UTC

  1 point

  in reply to: Flodorner’s comment on: Im­pli­ca­tions of Quan­tum Com­put­ing for Ar­tifi­cial In­tel­li­gence al­ign­ment re­search (ABRIDGED)

  I do agree with your as­s­es­ment, and I would be medium ex­cited about some­body in­for­mally re­search­ing what al­gorithms can be quan­tized to see if there is low hang­ing fruit in terms of sim­plify­ing as­sump­tions that could be made in a world where ad­vanced AI is quan­tum-pow­ered.

  How­ever my cur­rent in­tu­ition is there is no much sense in dig­ging in this un­less we were sort of con­fi­dent that 1) we will have ac­cess to QC be­fore TAI and that 2) QC will be a core com­po­nent of AI.

  To give a bit more con­text to the ar­ti­cle, Pablo and me origi­nally wrote it be­cause we dis­agreed on whether cur­rent re­search in AI Align­ment would still be use­ful if quan­tum com­put­ing was a core com­po­nent of ad­vanced AI sys­tems.

  Had we con­cluded that quan­tum ofus­ca­tion threat­ened to in­val­i­date some as­sump­tions made by cur­rent re­search, we would have been more em­phatic about the ne­ces­sity of hav­ing quan­tum com­put­ing ex­perts work­ing on “safe­guard­ing our re­search” on AI Align­ment.

Im­pli­ca­tions of Quan­tum Com­put­ing for Ar­tifi­cial In­tel­li­gence al­ign­ment re­search (ABRIDGED)

A sum­mary of Ni­cholas Beck­stead’s writ­ing on Bayesian Ethics

• Jsevillamol 2 Sep 2019 10:49 UTC

  12 points

  on: Cause X Guide

  I like this post a lot; it is suc­cint and pro­vides a great ac­tion­able for EAs to act on.

  Stylis­ti­cally I would pre­fer if the Or­ga­ni­za­tion sec­tion was bro­ken down into a para­graphs per sec­tion to make it eas­ier to read.

  I like that you pre­com­mited to a trans­par­ent way of se­lect­ing the new causes you pre­sent to the read­ers and limited the scope to 15. I would per­son­ally have liked to see them bro­ken up in sec­tions de­pend­ing on what method they were cho­sen by.

  For other read­ers who are ea­ger for more, here there are other two that satisfy the crite­ria but I sup­pose did not make it to the list:

  Atom­i­cally Pre­cise Man­u­fac­tur­ing (cause area en­dorse by two ma­jor or­ga­ni­za­tions—OPP and Eric Drexler from FHI)

  Align­ing Recom­mender Sys­tems (cause pro­file with more than 50 up­votes in the EA fo­rum)

• Jsevillamol 7 Aug 2019 18:37 UTC

  5 points

  on: How to gen­er­ate re­search proposals

  As fur­ther read­ing I re­cently came across Re­search as a Stochas­tic De­ci­sion Pro­cess, which dis­cusses an­other sys­tem­atic ap­proach to re­search.

  Sum­mary copy pasted from the ar­ti­cle:

  
  

  Summary

  Many of our de­fault in­tu­itions about how to pur­sue un­cer­tain ideas are coun­ter­pro­duc­tive:

  • We of­ten try eas­ier tasks first, when in­stead we should try the most in­for­ma­tive tasks first.

  • We of­ten con­flate a high-level ap­proach with a low-level in­stan­ti­a­tion of the ap­proach.

  • We are of­ten too slow to try to dis­prove our own ideas.

  Build­ing frame­works that reify the re­search pro­cess as a con­crete search prob­lem can help un­earth these in­cor­rect in­tu­itions and re­place them with sys­tem­atic rea­son­ing.