I am not making a case for who should be referred to as “the AI Safety community”, and if you draw a larger circle then you get a lot more people with lower p(doom)s. You still get mostly x-risk people as opposed to other risks and people thinking that it’s only x-risk that justifies intervening, implicitly if not explicitly.
> To address another claim: “The Dial of Progress” by Zvi, a core LessWrong contributor, makes the case that technology is not always good (similar to the “Technology Bucket Error” post) and the comments overwhelmingly seem to agree.
This post is a great example of my point. If you pay attention to the end, Zvi says that the dial thinking is mostly right and is morally sympathetic to it, just that AGI is an exception. I wanted it to mean what you thought it meant :(
Again, just giving my impressions from interacting with AI safety people: it doesn’t seem to me like I get this impression by drawing a larger circle—I don’t recall hearing the types of arguments you allude to even from people I consider “core” to AI safety. I think it would help me understand if you were able to provide some examples? (Although like I said, I found examples either way hard to search for, so I understand if you don’t have any available.)
I still disagree about the Dial post: at the end Zvi says
Seeing highly intelligent thinkers who are otherwise natural partners and allies making a variety of obvious nonsense arguments, in ways that seem immune to correction, in ways that seem designed to prevent humanity from taking action to prevent its own extinction, is extremely frustrating. Even more frustrating is not knowing why it is happening, and responding in unproductive ways.
So my read is that he wants to explain and understand the position as well as possible, so that he can cooperate as effectively as possible with people who take the Dial position. He also agrees on lots of object-level points with the people he’s arguing against. But ultimately actually using the Dial as an argument is “obvious nonsense,” for the same reason the Technology Bucket Error is an error.
I am not making a case for who should be referred to as “the AI Safety community”, and if you draw a larger circle then you get a lot more people with lower p(doom)s. You still get mostly x-risk people as opposed to other risks and people thinking that it’s only x-risk that justifies intervening, implicitly if not explicitly.
> To address another claim: “The Dial of Progress” by Zvi, a core LessWrong contributor, makes the case that technology is not always good (similar to the “Technology Bucket Error” post) and the comments overwhelmingly seem to agree.
This post is a great example of my point. If you pay attention to the end, Zvi says that the dial thinking is mostly right and is morally sympathetic to it, just that AGI is an exception. I wanted it to mean what you thought it meant :(
Again, just giving my impressions from interacting with AI safety people: it doesn’t seem to me like I get this impression by drawing a larger circle—I don’t recall hearing the types of arguments you allude to even from people I consider “core” to AI safety. I think it would help me understand if you were able to provide some examples? (Although like I said, I found examples either way hard to search for, so I understand if you don’t have any available.)
I still disagree about the Dial post: at the end Zvi says
So my read is that he wants to explain and understand the position as well as possible, so that he can cooperate as effectively as possible with people who take the Dial position. He also agrees on lots of object-level points with the people he’s arguing against. But ultimately actually using the Dial as an argument is “obvious nonsense,” for the same reason the Technology Bucket Error is an error.
I was going on my memory of that post and I don’t have the spoons to go through it again, so I’ll take your word for it.