Yeah, I was going to post that tweet. I’d also like to mention my related thread that if you have a history of crying wolf, then when wolves do start to appear, you’ll likely be turned to as a wolf expert.
There’s an additional problem that people who sound the alarms will likely be accused by some of “crying wolf” regardless of the outcome:
World A) Group X cries wolf. AI was not actually dangerous, nothing bad happens. Group X (rightly) gets accused of crying wolf and loses credibility, even if AI gets dangerous at some future point.
World B) Group X cries wolf. AI is actually dangerous, but because they cried wolf, we manage the risk and there is no catastrophe. Seeing the absence of a catastrophe, some people will accuse group X of crying wolf and they lose credibility.
I gave an argument for why I don’t think the cry wolf-effects would be as large as one might think in World A. Afaict your comment doesn’t engage with my argument.
I’m not sure what you’re trying to say with your comment about World B. If we manage to permanently solve the risks relating to AI, then we’ve solved the problem. Whether some people will then be accused of having cried wolf seems far less important relative to that.
You’re right—my comment is addressing an additional problem. (So I maybe should’ve made it a standalone comment)
As far as your second point is concerned—that’s true, unless we will face risk (again, and possibly more) at a later point. I agree with you that “crying wolf-effects” matter less or not at all under conditions where a problem is solved once and for all (unless it affects the credibility of a community which simultaneously works on other problems which remain unsolved, as is probably true of the EA community).
I also guess cry wolf-effects won’t be as large as one might think—e.g. I think people will look more at how strong AI systems appear at a given point than at whether people have previously warned about AI risk.
Yeah, I was going to post that tweet. I’d also like to mention my related thread that if you have a history of crying wolf, then when wolves do start to appear, you’ll likely be turned to as a wolf expert.
There’s an additional problem that people who sound the alarms will likely be accused by some of “crying wolf” regardless of the outcome:
World A) Group X cries wolf. AI was not actually dangerous, nothing bad happens. Group X (rightly) gets accused of crying wolf and loses credibility, even if AI gets dangerous at some future point.
World B) Group X cries wolf. AI is actually dangerous, but because they cried wolf, we manage the risk and there is no catastrophe. Seeing the absence of a catastrophe, some people will accuse group X of crying wolf and they lose credibility.
I gave an argument for why I don’t think the cry wolf-effects would be as large as one might think in World A. Afaict your comment doesn’t engage with my argument.
I’m not sure what you’re trying to say with your comment about World B. If we manage to permanently solve the risks relating to AI, then we’ve solved the problem. Whether some people will then be accused of having cried wolf seems far less important relative to that.
You’re right—my comment is addressing an additional problem. (So I maybe should’ve made it a standalone comment)
As far as your second point is concerned—that’s true, unless we will face risk (again, and possibly more) at a later point. I agree with you that “crying wolf-effects” matter less or not at all under conditions where a problem is solved once and for all (unless it affects the credibility of a community which simultaneously works on other problems which remain unsolved, as is probably true of the EA community).