the better strategy of focusing on the easier wins
I feel that you are not really appreciating the point that such “easier wins” aren’t in fact wins at all, in terms of keeping us all alive. They might make some people feel better, but they are very unlikely to reduce AI takeover risk to, say, a comfortable 0.1% (In fact I don’t think they will reduce it to below 50%).
I think I’m particularly triggered by all this because of a conversation I had last year with someone who takes AI takeover risk very seriously and could double AI safety philanthropy if they wanted to. I was arguing they should start funding AI safety, but the conversation was a total misfire because they conflated “AI safety” with “stop AI development”: their view was that that will never happen, and they were actively annoyed that they were hearing what they considered to be such a dumb idea. My guess was that EY’s TIME article was a big factor there.
Well hearing this, I am triggered that someone “who takes AI takeover risk very seriously” would think that stopping AI development was “such a dumb idea”! I’d question whether they do actually take AI takeover risk seriously at all. Whether or not a Pause is “realistic” or “will never happen”, we have to try! It really is our only shot if we actually care about staying alive for more than another few years. More people need to realise this. And I still don’t understand how people can think that the default outcome of AGI/ASI is survival for humanity, or an OK outcome.
...the question is how one can be so confident that any work we do now (including with ~AGI assistance, including if we’ve bought extra time via control measures and/or deals with misaligned ~AGIs) is insufficient, such that the only thing that makes a meaningful difference to x-risk, even in expectation, is a global moratorium. And I’m still not seeing the case for that.
I’d flip this completely, and say: the question is why we should be so confident that any work we do now (including with AI assistance, including if we’ve bought extra time via control measures and/or deals with misaligned AIs) is sufficient to solve alignment, such that the only thing that makes a meaningful difference to x-risk, even in expectation, a global moratorium, is unnecessary. I’m still not seeing the case for that.
I feel that you are not really appreciating the point that such “easier wins” aren’t in fact wins at all, in terms of keeping us all alive. They might make some people feel better, but they are very unlikely to reduce AI takeover risk to, say, a comfortable 0.1% (In fact I don’t think they will reduce it to below 50%).
Well hearing this, I am triggered that someone “who takes AI takeover risk very seriously” would think that stopping AI development was “such a dumb idea”! I’d question whether they do actually take AI takeover risk seriously at all. Whether or not a Pause is “realistic” or “will never happen”, we have to try! It really is our only shot if we actually care about staying alive for more than another few years. More people need to realise this. And I still don’t understand how people can think that the default outcome of AGI/ASI is survival for humanity, or an OK outcome.
I’d flip this completely, and say: the question is why we should be so confident that any work we do now (including with AI assistance, including if we’ve bought extra time via control measures and/or deals with misaligned AIs) is sufficient to solve alignment, such that the only thing that makes a meaningful difference to x-risk, even in expectation, a global moratorium, is unnecessary. I’m still not seeing the case for that.