matthew.vandermerwe

• matthew.vandermerweAug 1, 2022, 6:09 PM

  7 points

  0 ∶ 0

  on: On the Vuln­er­a­ble World Hypothesis

  Thanks for writing this, I like the forensic approach. I’ve long wished there was more discussion of the VWH paper, so it’s been great to see yours and Maxwell Tabarrok’s post in recent weeks.

  Not an objection to your argument, but minor quibble with your reconstructed Bostrom argument:

  P4: Ubiquitous real-time worldwide surveillance is the best way to decrease the risk of global catastrophes

  I think it’s worth noting that the paper’s conclusion is that both ubiquitous surveillance and effective global governance are required for avoiding existential catastrophe,^[1] even if only discussing one of these.

  [Disclaimer: I work for Nick Bostrom, these are my personal views]

  1. ^

     from conclusion: “We traced the root cause of our civilizational exposure to two structural properties of the contemporary world order: on the one hand, the lack of preventive policing capacity to block, with extremely high reliability, individuals or small groups from carrying out actions that are highly illegal; and, on the other hand, the lack of global governance capacity to reliably solve the gravest international coordination problems even when vital national interests by default incentivize states to defect. General stabilization against potential civilizational vulnerabilities [...] would require that both of these governance gaps be eliminated.”

• matthew.vandermerweJul 5, 2022, 1:43 PM

  21 points

  0 ∶ 0

  in reply to: Zach Stein-Perlman’s comment on: Fu­ture Mat­ters #3: digi­tal sen­tience, AGI ruin, and fore­cast­ing track records

  Hi Zach, thank you for your comment. I’ll field this one, as I wrote both of the summaries.

  This strongly suggests that Bostrom is commenting on LaMDA, but he’s discussing “the ethics and political status of digital minds” in general.

  I’m comfortable with this suggestion. Bostrom’s comment was made (i.e. uploaded to nickbostrom.com) the day after the Lemoine story broke. (source: I manage the website).

  “[Yudkowsky] recently announced that MIRI had pretty much given up on solving AI alignment”

  I chose this phrasing on the basis of the second sentence of the post: “MIRI didn’t solve AGI alignment and at least knows that it didn’t.” Thanks for pointing me to Bensinger’s comment, which I hadn’t seen. I remain confused by how much of the post should be interpreted literally vs tongue-in-cheek. I will add the following note into the summary:

  (Edit: Rob Bensinger clarifies in the comments that “MIRI has [not] decided to give up on reducing existential risk from AI.”)

  Thanks!

Fu­ture Mat­ters #3: digi­tal sen­tience, AGI ruin, and fore­cast­ing track records

Fu­ture Mat­ters #2: Clue­less skep­ti­cism, ‘longter­mist’ as an iden­tity, and nan­otech­nol­ogy strat­egy research

• matthew.vandermerweMay 21, 2022, 7:36 PM

  4 points

  0 ∶ 0

  on: Cool Offices ?

  Cool Offices ?

  Good/​reliable AC and ventilation are very important IMO.

• matthew.vandermerweMay 21, 2022, 7:27 PM

  9 points

  0 ∶ 0

  on: Si­mu­la­tion ar­gu­ment?

  I’m trying to understand the simulation argument.

  You might enjoy Joe Carlsmith’s essay, Simulation Arguments (LW).

Fu­ture Mat­ters #1: AI take­off, longter­mism vs. ex­is­ten­tial risk, and prob­a­bil­ity discounting

Fu­ture Mat­ters #0: Space gov­er­nance, fu­ture-proof ethics, and the launch of the Fu­ture Fund

• matthew.vandermerweJan 18, 2022, 1:41 PM

  3 points

  0 ∶ 0

  on: How many lives has the U.S. Pres­i­dent’s Emer­gency Plan for AIDS Relief (PEPFAR) saved?

  This Vox article by Dylan Matthews cites these two studies, which try to get at this question:

  • Bendavid & Bhattacharya (2008) — finds 1.2m deaths averted in twelve countries from ~2004–7

  • Bendavid et al. (2012) “found that PEPFAR reduced all-cause adult mortality in affected African countries by 16 to 20 percent.”

  EDIT to add: here’s a more recent analysis, looking at mortality impact up to 2018 — Kates et al. (2021)

• matthew.vandermerweSep 15, 2021, 7:30 PM

  3 points

  0 ∶ 0

  in reply to: matthew.vandermerwe’s comment on: Ethics of ex­is­ten­tial risk

  btw — there’s a short section on this in my old Existential Risk wikipedia draft. maybe some useful stuff to incorporate into this.

• matthew.vandermerweSep 15, 2021, 7:23 PM

  1 point

  0 ∶ 0

  in reply to: MichaelA🔸’s comment on: Ethics of ex­is­ten­tial risk

  weak disagree. FWIW lots of good cites in endnotes to chapter 2 of The Precipice pp.305–12; and Moynihan’s X-Risk.

• matthew.vandermerweJul 18, 2021, 12:02 PM

  3 points

  0 ∶ 0

  in reply to: annaleptikon’s comment on: What are things ev­ery­one here should (maybe) read?

  I considered writing a post about the same biography you mentioned for the forum.

  I would love to read such a post!

  It’s very humbling to see how much he already thought of, which we now call EA.

  Agreed — I think the Ramsey/​Keynes-era Apostles would make an interesting case study of a ‘proto-EA’ community.

• matthew.vandermerweJun 22, 2021, 10:59 AM

  13 points

  0 ∶ 0

  on: Should EA Buy Distri­bu­tion Rights for Foun­da­tional Books?

  Another historical precedent

  In 1820, James Mill seeks permission for a plan to print and circulate 1,000 copies of his Essay on Government, originally published as a Supplement to Napier’s Encyclopaedia Britannica:

  I have yet to speak to you about an application which has been made to me as to the article on Government, from certain persons, who think it calculated to disseminate very useful notions, and wish to give a stimulus to the circulation of them. Their proposal is, to print (not for sale, but gratis distribution) a thousand copies. I have refused my consent till I should learn from you, whether this would be considered an impropriety with respect to the Supplement. To me it appears the reverse, as the distribution would in some degree operate as an advertisement.

  Ernest Barker suggests it was quite successful:

  Mill’s article was thus given a wider circulation that the Supplement to the Encyclopedia would have afforded by itself … By 1824 … there had appeared what was possibly a second edition … Mill, in a letter of August 1825, speaks of the second reprint ‘being all gone, and great demand remaining.’ (He also mentions … that his essays ‘are the text-books of the young men of the Union at Cambridge’.

• matthew.vandermerweJun 4, 2021, 2:29 PM

  14 points

  0 ∶ 0

  in reply to: Stefan_Schubert’s comment on: Ethics of ex­is­ten­tial risk

  FWIW, and setting aside stylistic considerations for the Wiki, I dislike ‘x-risk’ as a term and avoid using it myself even in informal discussions.

  • it’s ambiguous between ‘extinction’ and ‘existential’, which is already a common confusion

  • it seems unserious and somewhat flippant (vaguely comic book/​sci-fi vibes)

  • the ‘x’ prefix can denote the edgy, or sexual (e.g. X Games; x-rated; Generation X?)

  • ‘x’ also often denotes an unknown value (e.g. in ‘Cause X’ — another abbreviation I dislike; or indeed Stefan’s comment earlier in this thread)

• matthew.vandermerweJun 4, 2021, 2:06 PM

  3 points

  0 ∶ 0

  in reply to: Pablo’s comment on: Ethics of ex­is­ten­tial risk

  I prefer this option to all others mentioned here.

• matthew.vandermerweMay 20, 2021, 10:34 AM

  5 points

  0 ∶ 0

  in reply to: Max_Daniel’s comment on: What are things ev­ery­one here should (maybe) read?

  I also kind of think everyone should read at least one biography, in particular of people who have become scientifically, intellectually, culturally, or politically influential.

  Some biographies I’ve enjoyed in this vein:

  • Frank Ramsey: A Sheer Excess of Powers

  • The Price of Peace: Money, Democracy, and the Life of John Maynard Keynes

  • Karl Marx: a Nineteenth-Century Life

• matthew.vandermerweOct 8, 2020, 5:34 PM

  5 points

  0 ∶ 0

  in reply to: jacobpfau’s comment on: Some learn­ings I had from fore­cast­ing in 2020

  With regards to the AGI timeline, it’s important to note that Metaculus’ resolution criteria are quite different from a ‘standard’ interpretation of what would constitute AGI^[1], (or human-level AI^[2], superintelligence^[3], transformative AI, etc.). It’s also unclear what proportion of forecasters have read this fine print (interested to hear others’ views on this), which further complicates interpretation.

  For these purposes we will thus define “an artificial general intelligence” as a single unified software system that can satisfy the following criteria, all easily completable by a typical college-educated human.

  • Able to reliably pass a Turing test of the type that would win the Loebner Silver Prize.

  • Able to score 90% or more on a robust version of the Winograd Schema Challenge, e.g. the “Winogrande” challenge or comparable data set for which human performance is at 90+%

  • Be able to score 75th percentile (as compared to the corresponding year’s human students; this was a score of 600 in 2016) on all the full mathematics section of a circa-2015-2020 standard SAT exam, using just images of the exam pages and having less than ten SAT exams as part of the training data. (Training on other corpuses of math problems is fair game as long as they are arguably distinct from SAT exams.)

  • Be able to learn the classic Atari game “Montezuma’s revenge” (based on just visual inputs and standard controls) and explore all 24 rooms based on the equivalent of less than 100 hours of real-time play (see closely-related question.)

  By “unified” we mean that the system is integrated enough that it can, for example, explain its reasoning on an SAT problem or Winograd schema question, or verbally report its progress and identify objects during videogame play. (This is not really meant to be an additional capability of “introspection” so much as a provision that the system not simply be cobbled together as a set of sub-systems specialized to tasks like the above, but rather a single system applicable to many problems.)

  ---

  1. ↩︎

     OpenAI Charter

  2. ↩︎

     expert survey

  3. ↩︎

     Bostrom

• matthew.vandermerweOct 3, 2020, 2:03 PM

  52 points

  0 ∶ 0

  on: Has any­one gone into the ‘High-Im­pact PA’ path?

  I work at FHI, as RA and project manager for Toby Ord/​The Precipice (2018–20), and more recently as RA to Nick Bostrom (2020–). Prior to this, I spent 2 years in finance, where my role was effectively that of an RA (researching cement companies, rather than existential risk). All of the below is in reference to my time working with Toby.

  Let me know if a longer post on being an RA would be useful, as this might motivate me to write it.

  Impact

  I think a lot of the impact can be captured in terms of being a multiplier^[1] on their time, as discussed by Caroline and Tanya. This can be sub-divided into two (fuzzy, non-exhaustive) pathways:

  • Decision-making — helping them make better decisions, ranging from small (e.g. should they appear on podcast X) to big (e.g. should they write a book)

  • Execution — helping them better execute their plans

  When I joined Toby on The Precipice, a large proportion of his impact was ‘locked in’ insofar he was definitely writing the book. There were some important decisions, but I expect more of my impact was via execution, which influenced (1) the quality of the book itself; (2) the likelihood of it’s being published on schedule; (3) the promotion of the book and its ideas; (4) the proportion of Toby’s productive time it took up, i.e. by freeing up time for him to do non-book things. Over the course of my role, I think I (very roughly) added 5–25% to the book’s impact, and freed up 10–33% of Toby’s time.

  Career decisions

  Before joining Toby, I was planning to join the first cohort of FHI’s Research Scholars Program and pursued my own independent projects for 2 years. At the time, the most compelling reason for choosing the RA role was:

  • Toby’s book will have large impact X, and I can expect to multiply this by ~10%, for impact of ~0.1X

  • If I ‘do my own thing’, it would take me much longer than 2 years to find and execute a project with at least 0.1X impact (relative to the canonical book on existential risk…)

  One thing I didn’t foresee is how valuable the role would be for my development as a researcher. While I’ve had less opportunity to choose my own research projects; publish papers; etc., I think this has been substantially outweighed by the learning benefits of working so closely with a top tier researcher on important projects. Overall, I expect that working with Toby ‘sped up’ my development by a few years relative to doing independent research of some sort.

  One noteworthy feature of being a ‘high-impact RA/​PA/​etc’ is that while these jobs are relatively highly regarded in EA circles, they can sound a bit baffling to anyone else. As such, I think I’ve built up some pretty EA-specific career capital.

  Some other names

  Here’s an incomplete list of people who have done (or are doing) this line of work, other than Caroline and myself:

  Nick Bostrom — Kyle Scott, Tanya Singh, Andrew Snyder-Beattie

  Toby Ord — Andrew Snyder-Beattie, Joe Carlsmith

  Will MacAskill – Pablo Stafforini, Laura Pomarius, Luisa Rodriguez, Frankie Andersen-Wood, Aron Vallinder

  ---

  1. ↩︎

     some RA trivia — Richard Kahn, the economist normally credited with the idea of a (fiscal) multiplier, was a long-time RA to John Maynard Keynes, of whom Keynes’ wrote “He is a marvelous critic and suggester and improver … There never was anyone in the history of the world to whom it was so helpful to submit one’s stuff.”

  What links here?

  • Notes on EA-re­lated re­search, writ­ing, test­ing fit, learn­ing, and the Forum by MichaelA🔸 (Mar 27, 2021, 9:52 AM; 98 points)
  • In­ter­ven­tion op­tions for im­prov­ing the EA-al­igned re­search pipeline by MichaelA🔸 (May 28, 2021, 2:26 PM; 49 points)
  • MichaelA🔸's comment on MichaelA’s Quick takes by MichaelA🔸 (May 16, 2021, 2:26 PM; 40 points)
  • Notes on effec­tive-al­tru­ism-re­lated re­search, writ­ing, test­ing fit, learn­ing, and the EA Forum by MichaelA (LessWrong; Mar 28, 2021, 11:43 PM; 14 points)
  • MichaelA🔸's comment on MichaelA’s Quick takes by MichaelA🔸 (Feb 22, 2021, 1:20 AM; 8 points)
  • MichaelA🔸's comment on MichaelA’s Quick takes by MichaelA🔸 (May 22, 2021, 7:36 AM; 5 points)
  • MichaelA🔸's comment on Pos­si­ble gaps in the EA community by Michelle_Hutchinson (Feb 22, 2021, 1:26 AM; 3 points)

• matthew.vandermerweSep 16, 2020, 12:07 PM

  62 points

  1 ∶ 0

  on: Some thoughts on EA out­reach to high schoolers

  If there were more orgs doing this, there’d be the risk of abuse working with minors if in-person.

  I think this deserves more than a brief mention. One of the two high school programs mentioned (ESPR) failed to safeguard students from someone later credibly accused of serious abuse, as detailed in CFAR’s write-up:

  Of the interactions CFAR had with Brent, we consider the decision to let him assist at ESPR—a program we helped run for high school students—to have been particularly unwise … We do not believe any students were harmed. However, Brent did invite a student (a minor) to leave camp early to join him at Burning Man. Beforehand, Brent had persuaded a CFAR staff member to ask the camp director for permission for Brent to invite the student. Multiple other staff members stepped in to prevent this, by which time the student had decided against attending anyway.

  This is a terrible track record for this sort of outreach effort. I think it provides a strong reason against pursuing it further without a high degree of assurance that the appropriate lessons have been learned — something which doesn’t seem to have been addressed in the post or comments.

  What links here?

  • RyanCarey's comment on RyanCarey’s Quick takes by RyanCarey (Mar 27, 2021, 3:30 PM; 25 points)

• matthew.vandermerweAug 7, 2020, 8:48 AM

  3 points

  0 ∶ 0

  in reply to: Max_Daniel’s comment on: Max_Daniel’s Shortform

  Nice post. I’m reminded of this Bertrand Russell passage:

  “all the labours of the ages, all the devotion, all the inspiration, all the noonday brightness of human genius, are destined to extinction in the vast death of the solar system, and that the whole temple of Man’s achievement must inevitably be buried beneath the debris of a universe in ruins … Only within the scaffolding of these truths, only on the firm foundation of unyielding despair, can the soul’s habitation henceforth be safely built.” —A Free Man’s Worship, 1903

  I take Russell as arguing that the inevitability (as he saw it) of extinction undermines the possibility of enduring achievement, and that we must therefore either ground life’s meaning in something else, or accept nihilism.

  At a stretch, maybe you could run your argument together with Russell’s — if we ground life’s meaning in achievement, then avoiding nihilism requires that humanity neither go extinct nor achieve total existential security.