My p(doom) went down slightly (From around 30% to around 25%) mainly as a result of how GPT-4 caused governments to begin taking AI seriously in a way I didnât predict. My timelines havenât changedâthe only capability increase of GPT-4 that really surprised me was its multimodal nature. (Thus, governments waking up to this was a double surprise, because it clearly surprised them in a way that it didnât surprise me!)
Iâm also less worried about misalignment and more worried about misuse when it comes to the next five years, due to how LLMâs appear to behave. It seems that LLMâs arenât particularly agentic by default, but can certainly be induced to perform agent-like behaviourâGPT-4â˛s inability to do this well seems to be a capability issue that I expect to be resolved in a generation or two. Thus, Iâm less worried about the training of GPT-N but still worried about the deployment of GPT-N. It makes me put more credence in the slow takeoff scenario.
This also makes me much more uncertain about the merits of pausing in the short-term, like the next year or two. I expect that if our options were âPause nowâ or âPause after another year or twoâ, the latter is better. In practice, I know the world doesnât work that way and slowing down AI now likely slows down the whole timeline, which complicates things. I still think that government efforts like the UKâs AISI are net-positive (Iâm joining them for a reason, after all) but I think a lot of the benefit to reducing x-risk here is building a mature field around AI policy and evaluations before we need itâif we wait until I think the threat of misaligned AI is imminent, that may be too late.
My p(doom) went down slightly (From around 30% to around 25%) mainly as a result of how GPT-4 caused governments to begin taking AI seriously in a way I didnât predict. My timelines havenât changedâthe only capability increase of GPT-4 that really surprised me was its multimodal nature. (Thus, governments waking up to this was a double surprise, because it clearly surprised them in a way that it didnât surprise me!)
Iâm also less worried about misalignment and more worried about misuse when it comes to the next five years, due to how LLMâs appear to behave. It seems that LLMâs arenât particularly agentic by default, but can certainly be induced to perform agent-like behaviourâGPT-4â˛s inability to do this well seems to be a capability issue that I expect to be resolved in a generation or two. Thus, Iâm less worried about the training of GPT-N but still worried about the deployment of GPT-N. It makes me put more credence in the slow takeoff scenario.
This also makes me much more uncertain about the merits of pausing in the short-term, like the next year or two. I expect that if our options were âPause nowâ or âPause after another year or twoâ, the latter is better. In practice, I know the world doesnât work that way and slowing down AI now likely slows down the whole timeline, which complicates things. I still think that government efforts like the UKâs AISI are net-positive (Iâm joining them for a reason, after all) but I think a lot of the benefit to reducing x-risk here is building a mature field around AI policy and evaluations before we need itâif we wait until I think the threat of misaligned AI is imminent, that may be too late.