I understand that people undermining alignment work will attempt to justify it with the concern that it may advance capabilities and may not advance safety. I just think that’s a terribly mistaken drunk mormon hypothesis.
It’s proceeding from the false premise that somehow alignment and capabilities are two equal concerns that we trade off against each other, and if you can imagine a way that it ends up advancing capabilities and not advancing alignment, you shouldn’t do it. But the tradeoff is not anywhere close to equal, the more accurate perspective would be that if you can imagine it advancing alignment, you should do it.
Yes, I was arguing against Pause AI’s position (both summarized by you in the piece and as expressed by their leadership elsewhere), I’m aware that you hadn’t expressed a strong pro- or con- stance on it either way.
My comments were a little overly combative, which wasn’t meant to be directed at you in particular—my frustration is that I think a lot of people consider “We cautiously support some technical safety work” to be a moderate position while I believe it is insane.
I agree that it’s theoretically possible for there to be bad alignment-capabilities tradeoffs, just like it’s theoretically possible for there to be bad tradeoffs of anything vs anything.
But in the actual world we live in today, there is widespread skepticism of the existence of AI risk, including among AI researchers. Many people are happy to work full steam ahead on developing AGI with no thought to potential existential risks, and declare this publicly. The issue has low salience among the general public and even less so abroad. And the difficulty of building AGI falls automatically year by year due to the advance of Moore’s law.
With all of these factors in play, I think AGI is happening. More to the point, if it doesn’t happen, it won’t be because a bunch of individual people who were worried about AI risk refrained from doing things that might advance capabilities; it would be because we somehow got an international agreement. Individual people withdrawing from the field doesn’t have anything to do with an AI pause; all it can do is slow things down. And there is a hard limit on how much individual withdrawal can slow things down, because there is a large core of people who believe AI poses no risk who want to advance it as much as possible who will not be dissuaded, and the more you slow down the more they’re aided by Moore’s law.
In this environment, it is really important to try to figure out alignment. That is the thing to focus on. The problem with all the examples of bad alignment research you gave above is not that they advance capabilities, it is that they are bad alignment research.