I asked Alex “no chance you can comment on whether you think assistance games are mostly irrelevant to modern deep learning?”
His response was “i think it’s mostly irrelevant, yeah, with moderate confidence”. He then told me he’d lost his EA forum credentials and said I should feel free to cross-post his message here.
(For what it’s worth, as people may have guessed, I disagree with him—I think you can totally do CIRL-type stuff with modern deep learning, to the extent you can do anything with modern deep learning.)
FYI, current cutting-edge large language models are trained on a massive amount of text on the internet (in the case of GPT-4, likely approximately all the text OpenAI could get their hands on). So they certainly have tons of information about stuff other than the task at hand.