Robust alignment requires alignment-relevant intervention during pretraining
Maybe the line between pretraining and posttraining will blur with future models?. Also I think taking alignment into account when pretraining will pretty obviously improve outcomes, the question is more about how much is “robust”. Or what is alignment in this sense. Like “refuse to do bioweapons” or actual “deep” AGI alignment?
emanuelr comments on Community Polls on Alignment Controversies