Just a bundle of subroutines sans free will, aka flawed post-monkey #~100,000,000,000. Peace and love xo
(All my posts, except those where I state otherwise, reflect my views and not those of my employer(s).)
Will Aldred
Karma: 1,656
The Last Temptation of Christ (Alexander, 2012)
The FTX Future Fund launched in Feb 2022, and Open Phil were hiring for a program officer for their new Global Health and Wellbeing program in Feb 2022 (see here).
For context, all FTX funds go/went to longtermist causes; Open Phil currently has two grantmaking programs (see here): one in Longtermism and the other one being the Global Health and Wellbeing program that launched—I assume—around Feb ’22.
So my guess, though I’m not certain, is that the launches of FTX Future Fund and Open Phil’s Global Health and Wellbeing program were linked, and that Open Phil did increase its neartermist:longtermist funding ratio when FTX funds became available.
(It’s interesting to note that, at present, my above comment is on −1 agreement karma after 50 votes. This suggests that the question of rebalancing the neartermist:longtermist funding ratio is genuinely controversial, as opposed to there being a community consensus either way.)
Thanks for the post, I appreciate the clarity it brings.
Given FTX Foundation’s focus on existential risk and longtermism, the most direct impacts are on our longtermist work. We don’t anticipate any immediate changes to our Global Health and Wellbeing work as a result of the recent news.
Would it not make sense for Open Phil to shift some of its neartermist/global health funds to longtermist causes?
Although any neartermist:longtermist funds ratio is, in my opinion, fairly arbitrary, this ratio has increased significantly following the FTX event. Thus, seems to me that Open Phil should maybe consider acting to rebalance it.
(I’d be curious to hear a solid counterargument.)
See also Michael Aird’s comments on this Tarsney (2020) paper. His main points are:
‘Tarsney’s model updates me towards thinking reducing non-extinction existential risks should be a little less of a priority than I previously thought.’ (link to full comment)
‘Tarsney seems to me to understate the likelihood that accounting for non-human animals would substantially affect the case for longtermism.’ (link)
‘The paper ignores 2 factors that could strengthen the case for longtermism—namely, possible increases in how efficiently resources are used and in what extremes of experiences can be reached.’ (link)
‘Tarsney writes “resources committed at earlier time should have greater impact, all else being equal”. I think that this is misleading and an oversimplification. See Crucial questions about optimal timing of work and donations and other posts tagged Timing of Philanthropy.’ (link)
’I think it’d be interesting to run a sensitivity analysis on Tarsney’s model(s), and to think about the value of information we’d get from further investigation of:
how likely the future is to resemble Tarsney’s cubic growth model vs his steady model
whether there are other models that are substantially likely, whether the model structures should be changed
what the most reasonable distribution for each parameter is.′ (link)
CERI Research Symposium Presentations (incl. Youtube links)
Roodman’s proposed “restriction that the various frameworks agree” makes no sense.
I’m with you. I think Roodman must disagree with the idea of giving probabilies to different—and necessarily conflicting—models of the world, but to me this seems like an odd position/disagreement. I might also be missing something.
Belated congrats on completing your PhD! I’m looking forward to reading the sections you’ve highlighted.
We discuss [...] how to develop your own inside views about AI Alignment.
See also Neel Nanda’s (2022):
You might want to check out ‘Research Summary: The Subjective Experience of Time’ (Schukraft, 2020).
I notice a two-karma system has been implemented in at least one EA Forum post before, see the comments section to this “Fanatical EAs should support very weird projects” post.
Love the post. For more examples—some of them EA-oriented—of Fermi estimates / back of the envelope calculations (BOTECs), see Botec Horseman’s Tweets.
(Note: Botec Horseman is neither myself nor Nuño.)
One data point here (I’m unsure how much weight to give it, probably not very much) is the 1983 movie The Day After, which is about the aftermath of nuclear war.
He [President Reagan] wrote in his diary that the film [The Day After] was “very effective and left me greatly depressed”, and that it changed his mind on the prevailing policy on a “nuclear war”. The film was also screened for the Joint Chiefs of Staff. A government advisor who attended the screening, a friend of Meyer’s, told him: “If you wanted to draw blood, you did it. Those guys sat there like they were turned to stone.” In 1987, Reagan and Soviet leader Mikhail Gorbachev signed the Intermediate-Range Nuclear Forces Treaty, which resulted in the banning and reducing of their nuclear arsenal. In Reagan’s memoirs, he drew a direct line from the film to the signing.
(Wikipedia; emphasis added)
Thanks for writing this post, have added it to my collection “AI timelines via bioanchors: the debate in one place”.
Feedback I’ve been giving to junior x-risk researchers
Tell us which papers should feature in the “Introduction to Nick Bostrom” series, and in what order. We’re especially keen for suggestions on what the first three papers should be.
If the series is to be 3 papers long, I’d suggest:
If the series is to be a dozen papers long, I’d suggest:
(DM’ed you)
Widespread access to atomically precise manufacturing could lead to widespread ability to unilaterally produce things as destructive as nuclear weapons.
See below an excerpt from the atomically precise manufacturing (APM) section of my and Michael Aird’s shallow review of tech developments that could increase risks from nuclear weapons (link):
The concern related to nuclear risk is that APM could potentially make proliferation and/or huge nuclear arsenal sizes much more likely. Specifically:
APM could make it much harder to monitor/control who has nuclear weapons, including even non-state groups or perhaps individuals.
APM could make it much more likely that non-state groups or individuals would not only attain one or a few but rather many nuclear weapons.
This could land us in the very risky-seeming “easy nukes” scenario described in Bostrom’s (2019) The Vulnerable World Hypothesis paper.
APM could make it so that nuclear-armed states can more cheaply, easily, and quickly build hundreds, thousands, tens of thousands, or even millions of nuclear weapons.
The enhanced ease and speed of doing this could undermine arms race stability.
This might also mean that even just the perception of APM being on the horizon could undermine strategic stability, even before APM has arrived.
And nuclear conflicts involving huge numbers of warheads are much more likely to cause an existential catastrophe, or otherwise increase existential risk, than nuclear conflicts involving the tens, hundreds, or thousands of nuclear weapons each nuclear-armed state currently has.
A counterpoint is that at least some nuclear-armed states already have the physical capacity to make many more nuclear warheads than they do (given enough time to build up nuclear manufacturing infrastructure), but having many more warheads doesn’t appear to be what these states are aiming for. This is some evidence that nuclear weapon production being easier to make might not result in huge arsenal sizes. But:
This is only some evidence.
And some nuclear-armed states (e.g., North Korea, Pakistan, and maybe India) may be developing nuclear weapons about as fast as they reasonably can, given their economies. As such, perhaps having access to APM would be likely to substantially affect at least their arsenal sizes.
(Note that we haven’t spent much time thinking about these points, nor seen much prior discussion of them, so this should all be taken as tentative and speculative.)
See also “A Fable of Science and Politics” (Yudkowsky, 2006).