>Bostrom says that if everyone could make nuclear weapons in their own home, civilization would be destroyed by default because terrorists, malcontent people and “folk who just want to see what would happen” would blow up most cities.
Yes, and what would the world be like to change this?
It’s terrible to think that the reason we are safe is because others are powerless. If EA seeks to maximize human potential, I think it’s really insightful that we are confident many people would destroy just because they can. And I think focusing on real well being of people is a way we can confront this.
Let’s do the thought experiment: What would the world look like where anyone had power to destroy the world at any time—and choose not to? Where no one made that choice?
What kind of care and support systems would exist? How would we respect each other? How would society be organized?
I think this is a good line of thinking because it helps understand how the world is vulnerable, and how we can make it less so.
It feels weird to me to hear that something is terrible to think. It might be terrible that we’re only alive because everyone doesn’t have the option to kill everyone else instantly, but it’s also true. Thinking true thoughts isn’t terrible.
If everyone has a button that could destroy all life on the planet, I feel like it’s unrealistic to expect that button to remain unpressed for more than a few hours. The most misanthropic person on Earth is very, very misanthropic. I’m not confident that many people would press the button, but the whole thing is that it only takes one.
Given that currently people don’t have such a button, it seems easier to think how we can prevent that button from existing, rather than how we could make everyone agree not to press the button. The button is a power no one should have.
>Bostrom says that if everyone could make nuclear weapons in their own home, civilization would be destroyed by default because terrorists, malcontent people and “folk who just want to see what would happen” would blow up most cities.
Yes, and what would the world be like to change this?
It’s terrible to think that the reason we are safe is because others are powerless. If EA seeks to maximize human potential, I think it’s really insightful that we are confident many people would destroy just because they can. And I think focusing on real well being of people is a way we can confront this.
Let’s do the thought experiment: What would the world look like where anyone had power to destroy the world at any time—and choose not to? Where no one made that choice?
What kind of care and support systems would exist? How would we respect each other? How would society be organized?
I think this is a good line of thinking because it helps understand how the world is vulnerable, and how we can make it less so.
-Kristopher
It feels weird to me to hear that something is terrible to think. It might be terrible that we’re only alive because everyone doesn’t have the option to kill everyone else instantly, but it’s also true. Thinking true thoughts isn’t terrible.
If everyone has a button that could destroy all life on the planet, I feel like it’s unrealistic to expect that button to remain unpressed for more than a few hours. The most misanthropic person on Earth is very, very misanthropic. I’m not confident that many people would press the button, but the whole thing is that it only takes one.
Given that currently people don’t have such a button, it seems easier to think how we can prevent that button from existing, rather than how we could make everyone agree not to press the button. The button is a power no one should have.