My p(doom) went down slightly (From around 30% to around 25%) mainly as a result of how GPT-4 caused governments to begin taking AI seriously in a way I didn’t predict. My timelines haven’t changed—the only capability increase of GPT-4 that really surprised me was its multimodal nature. (Thus, governments waking up to this was a double surprise, because it clearly surprised them in a way that it didn’t surprise me!)
I’m also less worried about misalignment and more worried about misuse when it comes to the next five years, due to how LLM”s appear to behave. It seems that LLM’s aren’t particularly agentic by default, but can certainly be induced to perform agent-like behaviour—GPT-4′s inability to do this well seems to be a capability issue that I expect to be resolved in a generation or two. Thus, I’m less worried about the training of GPT-N but still worried about the deployment of GPT-N. It makes me put more credence in the slow takeoff scenario.
This also makes me much more uncertain about the merits of pausing in the short-term, like the next year or two. I expect that if our options were “Pause now” or “Pause after another year or two”, the latter is better. In practice, I know the world doesn’t work that way and slowing down AI now likely slows down the whole timeline, which complicates things. I still think that government efforts like the UK’s AISI are net-positive (I’m joining them for a reason, after all) but I think a lot of the benefit to reducing x-risk here is building a mature field around AI policy and evaluations before we need it—if we wait until I think the threat of misaligned AI is imminent, that may be too late.
My p(doom) went down slightly (From around 30% to around 25%) mainly as a result of how GPT-4 caused governments to begin taking AI seriously in a way I didn’t predict. My timelines haven’t changed—the only capability increase of GPT-4 that really surprised me was its multimodal nature. (Thus, governments waking up to this was a double surprise, because it clearly surprised them in a way that it didn’t surprise me!)
I’m also less worried about misalignment and more worried about misuse when it comes to the next five years, due to how LLM”s appear to behave. It seems that LLM’s aren’t particularly agentic by default, but can certainly be induced to perform agent-like behaviour—GPT-4′s inability to do this well seems to be a capability issue that I expect to be resolved in a generation or two. Thus, I’m less worried about the training of GPT-N but still worried about the deployment of GPT-N. It makes me put more credence in the slow takeoff scenario.
This also makes me much more uncertain about the merits of pausing in the short-term, like the next year or two. I expect that if our options were “Pause now” or “Pause after another year or two”, the latter is better. In practice, I know the world doesn’t work that way and slowing down AI now likely slows down the whole timeline, which complicates things. I still think that government efforts like the UK’s AISI are net-positive (I’m joining them for a reason, after all) but I think a lot of the benefit to reducing x-risk here is building a mature field around AI policy and evaluations before we need it—if we wait until I think the threat of misaligned AI is imminent, that may be too late.