Don’t panic.
jai
Karma: 881
Meetup : Secular Solstice (Seattle)
The Copenhagen Interpretation of Ethics
I’ve been asked to post “500 Million But Not a Single One More” on the EA Forum so that it’s easier to include in a sequence of EA-related content. I need 5 karma to post—and the most straightforward way to get that seems like straight-up begging.
Just so this comment isn’t entirely devoid of content: This Week In Polio is a great way to track humanity’s (hopefully) final battles against Polio and one of my go-to pages when I’m looking to feel good about my species.
Hello and welcome Sammy! Excited about Future Perfect and looking forward to what Vox does with it. It looks like your comment may have gotten cut off, to the detriment of anyone who wishes to stay informed.
500 Million, But Not A Single One More
You can click on your username in the upper-right corner. Under “My Drafts”, there will be an option for “New Post”.
You can also hover over your username, and the drop-down menu that appears should include an option to create a new post.
Given the massive decline in expected EA liquidity since the purchase and the fact that the purchase was largely justified on the grounds that as a durable asset it could be converted back into liquid funds with minimal loss, why not sell it now?.
Nit: I was very explicitly asking why not sell, not suggesting a commitment to sell; I don’t appreciate the rhetorical pivot to argue against a point I was not making.
Yes, they should check that it makes financial sense. Though break-even shouldn’t be an anchor or decision point—what matters is whether the money from a sale could be better spent elsewhere, which doesn’t really have anything to do with money already spent.
If it doesn’t make sense to sell, then they shouldn’t sell—but they should also be ready to admit that they made an error in their initial analysis, because it would mean that the bulk of the cost is effectively gone and the true price in opportunity cost was significantly higher than the marginal costs implied.
To a first approximation, it’s either a mostly-recoverable expense that didn’t really cost that much when you consider it maintains its value as a durable good, or it’s not financially feasible to sell and all of those funds are sunk. It can’t be both.
I labeled this a nit in the hopes of avoiding this kind of deep spiral, but—“commit” is a significant intensifier that specifically implies overriding or ignoring opposing reasons beyond what would normally be expected.
“Why not sell the abbey?” is seeking countervailing reasons while implicitly framing “sell” as the default option to overcome.
”Should they commit to selling the abbey?” is seeking to ignore all possible countervailing reasons and implicitly frames “keep” as the default option which can only be overcome with an overwhelmingly strong argument.
Replacing the former with the latter is choosing a weaker opponent for the “keep” position to overcome.
This is a minor rhetorical thing that does not warrant as many words as this, but it is a real and meaningful thing.
Reply here—https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=tZP9seDyxvS7ot4t7
tldr: this is way more ink than should be spilt over a minor rhetorical point, but I believe it’s a real and meaningful (though minor) strawmanning that selects a weaker opponent for the “keep” position to face.
But that’s not a function of whether “a lot of the fixed costs (like transaction costs) have been paid”. That is very specifically referring to break-even relative to costs already expended, not to counterfactual spending. It’s okay if you’ve changed your mind, but I don’t think that what you said originally is consistent with this comment.
Interesting perspective, although I’m not sure how much we actually disagree. “Complicated and open”, to me reads as “difficult”
Is there a rephrasing of the initial statement you would endorse that makes this clearer? I’d suggest “If you apply a security mindset (Murphy’s Law) to the problem of AI alignment, it should quickly become apparent that we do not currently possess the means to ensure that any given AI is safe.”
It doesn’t seem to me that titotal is assuming MI is solved; having direct access to the brain doesn’t give you full insight into someone’s thoughts either, because neuroscience is basically a pile of unsolved problems with growing-but-still-very-incomplete-picture of low-level and high-level details. We don’t even have a consensus on how memory is physically implemented.
Nonetheless, if you had a bunch of invasive probes feeding you gigabytes/sec of live data from the brain of the genius general of the opposing army, it would be extremely likely to be useful information.
A really interesting thing is that, at the moment, this appears in practice to be a very-asymmetrical advantage. The high-level reasoning processes that GPT-4 implements don’t seem to be able to introspect about fine-grained details, like “how many tokens are in a given string”. The information is obviously and straightforwardly part of the model, but absent external help the model doesn’t seem to bridge the gap between low-level implementation details and high-level reasoning abilities—like us.
Here’s a quick attempt at a subset of conjunctive assumptions in Nate’s framing:
- The functional ceiling for AGI is sufficiently above the current level of human civilization to eliminate it
- There is a sharp cutoff between non-AGI AI and AGI, such that early kind-of-AGI doesn’t send up enough warning signals to cause a drastic change in trajectory.
- Early AGIs don’t result in a multi-polar world where superhuman-but-not-godlike agents can’t actually quickly and recursively self-improve, in part because none of them wants any of the others to take over—and without being able to grow stronger, humanity remains a viable player.
I’d like to apologize for “What Almost Was” being a dead link now—I’ve been trying to retire that blog for a while now while preserving the content elsewhere. “What Almost Was” can now be found here.
The current form appears to only allow uploading image files; I can upload a PNG, but not an SVG. This is probably just as well in my case, as the SVG only makes it more painfully obvious that I have no idea how to use Inkscape, but it seems like unintended behavior you might want to change.
The Rational Animations team has animated this, with narration by Robert Miles. It’s great, as is the rest of their growing body of work.
Most Leading AI Experts Believe That Advanced AI Could Be Extremely Dangerous to Humanity
Today, May 7 2023, I am planning to make some updates to this piece for historical accuracy. As part of this update, I’m including some citations and commentary in this comment.
I feel torn writing about some of these, because it’s attempting to summarize and quantify an incomprehensible quantity of suffering and death. The numbers are important, to recognize what was lost, to honor the fallen as best we can, and to emphasize the importance of killing Smallpox. But quantifying that loss, for me at least, requires temporarily embracing a detached attitude, especially when the clues are many and uncertainty is abundant. So that is how I will be describing citations and numbers for the remainder of this comment—and then, when I’ve collected as many citations and clarified as many estimations as I can, I will grapple with them as a human.
Much of the narrative is just from Wikipedia’s page on the History of Smallpox. This has changed a bit since 2014.
The figure “500 Million” comes from Donald Henderson by way of Our World in Data.
”Five million in India” comes from this article by Eugene D. Robin, Robert F. McCauley in the Indian Journal of Medical Ethics, 1997.“[A] thousand Europeans every day in the 18th century” is from the Wikipedia page. Specifically, “400,000“ annual deaths.
[Update] The text originally said that Smallpox killed “more than fifty million Native Americans”. I cannot find any source I might have originally used for this claim. When attempting to research this more recently, I came across the 2019 paper “Earth system impacts of the European arrival and Great Dying in the Americas after 1492”, which estimates 55 million all-cause deaths from European contact across the Americas between 1492 and 1600. “Counting the Dead: Estimating the Loss of Life in the Indigenous Holocaust, 1492-Present” is part of a body of literature exploring the interplay between disease and intentional violence, which further complicates things. Lots of things can make people vulnerable to disease. If a family is forced to relocate, goes hungry, and subsequently are weaker when they fall ill, it seems incorrect to say that the illness alone killed them. All told, disease is widely agreed to be the most significant contributor to the great dying of the Indigenous Peoples of the Americas, and of the diseases Smallpox was probably the most significant killer. Given the complexities and uncertainties here, when I finish writing this comment, I intend to update the text to “over ten million Native Americans” as a lower bound that I feel 80% confident about. I think I could refine this further with more research—this is still extremely ad hoc—but this is as much time as I can afford at the moment.
“more soldiers and civilians than any weapon, any soldier, any army”: See Wikipedia’s “List of anthropogenic disasters by death toll”
[Update] “In China, in the 10th century, humanity began to fight back.” I believe I originally got this figure from Our World In Data, but now believe it to be inaccurate. There are “references” that say the tradition dates back to the tenth century, but the evidence is sparse. Going by Wikipedia’s article on Variolation, I’m updating this to “the 15th century”.
Edward Jenner, 1796: Wikipedia. I’m choosing the day of the first inoculation (James Phipps) on May 14, 1796 rather than the publication date of the resulting paper, Inquiry into the Variolae vaccinae known as the Cow Pox, in 1798.
From here on most of the narrative can be traced through the Eradication section of Wikipedia’s article on Smallpox:
- “On August 26, 1807, Bavaria became the first country in the world to introduce compulsory vaccinations.”
—“In Northern Europe a number of countries had eliminated smallpox by 1900”.
- “In 1958 Professor Viktor Zhdanov, Deputy Minister of Health for the USSR, called on the World Health Assembly to undertake a global initiative to eradicate smallpox.”
—“In 1966 an international team, the Smallpox Eradication Unit, was formed under the leadership of an American, Donald Henderson.[127] In 1967, the World Health Organization intensified the global smallpox eradication by contributing $2.4 million annually to the effort, and adopted the new disease surveillance method promoted by Czech epidemiologist Karel Raška.”
—“To eradicate smallpox, each outbreak had to be stopped from spreading, by isolation of cases and vaccination of everyone who lived close by.[129] This process is known as “ring vaccination”.”
—“The WHO established a network of consultants who assisted countries in setting up surveillance and containment activities.”
—“donations of vaccine were provided primarily by the Soviet Union and the United States”
—“The last naturally occurring case of indigenous smallpox (variola minor) was diagnosed in Ali Maow Maalin, a hospital cook in Merca, Somalia, on 26 October 1977”
—“The global eradication of smallpox was certified, based on intense verification activities, by a commission of eminent scientists on 9 December 1979 and subsequently endorsed by the World Health Assembly on 8 May 1980.”
You’re definitely not alone—I’ve felt a lot of these things. Thank you for writing this. I think a lot of people are going to get a lot of good out of it.