I think for this purpose we should distinguish between AI misuse and misalignment (with the caveat that this is not a perfect distinction and there might be many more issues, etc.)
AI misuse:
A group of people wants to kill all of humanity (or have other hugely catastrophic goals), and they use AI tools.
I think this is definitely a concern for Bio risk, and some of the suggestions you make sound very reasonable. (Also, one should be careful how to talk about these possibilities, in order not to give anyone ideas, eg make sure that certain aspects of your research into AI assisted pandemics don’t go viral).
So there is an overlap, but one could also argue that this type of scenario belongs into the field of Bio risk. Of course, collaboration with AI policy people could be hugely important (eg to make certain kinds of open source AI illegal without drawing too much attention to it).
In my personal guess, misalignment is a bigger part of catastrophic risk,
and also receives a lot more attention within AI safety work.
AI misalignment:
An AI that is smarter than humans has goals that conflict with those of humans.
Humans go extinct sooner or later, because the AI wants to take control or because it is mostly indifferent about human existence (but not because some programmed-in hate for humans).
Here, Bio could be a concrete attack vector. But there probably many attack vectors, some of which require more or less capable AIs to pull of. My guess here is that Bio is at most a small part here, and focusing on concrete attack vectors might not be that valuable by comparison (although not be of zero value).
So, I would say that these types of misalignment worries should be considered “pure” AI catastrophic risks.
Against focusing too much on concrete attack vectors:
Building an AI that is smarter than all of humanity combined and wants to exterminate humanity is just a bad idea.
If we try to predict how exactly the AI will kill us, we will probably be wrong because the AI is smarter than us.
If we try to enumerate possible attack vectors one by one and defend against them, this might slow down the AI by some years (or weeks), but the AI can probably come up with attack vectors we haven’t thought of, or invent new technologies. Also, the AI might need multiple attack vectors and technologies, eg to ensure its own energy supply.
If that kind of AI has acquired enough resources/compute, we will probably lose against that AI, and defending against concrete scenarios might have some benefits, but it seems preferable to focus on not building that kind of AI in the first place.
An analogy I find helpful: If you play chess against Magnus Carlsen, then you don’t know in advance which concrete moves he will play, but you know that you will lose.
I think for this purpose we should distinguish between AI misuse and misalignment (with the caveat that this is not a perfect distinction and there might be many more issues, etc.)
AI misuse:
A group of people wants to kill all of humanity (or have other hugely catastrophic goals), and they use AI tools.
I think this is definitely a concern for Bio risk, and some of the suggestions you make sound very reasonable. (Also, one should be careful how to talk about these possibilities, in order not to give anyone ideas, eg make sure that certain aspects of your research into AI assisted pandemics don’t go viral). So there is an overlap, but one could also argue that this type of scenario belongs into the field of Bio risk. Of course, collaboration with AI policy people could be hugely important (eg to make certain kinds of open source AI illegal without drawing too much attention to it).
In my personal guess, misalignment is a bigger part of catastrophic risk, and also receives a lot more attention within AI safety work.
AI misalignment:
An AI that is smarter than humans has goals that conflict with those of humans. Humans go extinct sooner or later, because the AI wants to take control or because it is mostly indifferent about human existence (but not because some programmed-in hate for humans).
Here, Bio could be a concrete attack vector. But there probably many attack vectors, some of which require more or less capable AIs to pull of. My guess here is that Bio is at most a small part here, and focusing on concrete attack vectors might not be that valuable by comparison (although not be of zero value).
So, I would say that these types of misalignment worries should be considered “pure” AI catastrophic risks.
Against focusing too much on concrete attack vectors:
Building an AI that is smarter than all of humanity combined and wants to exterminate humanity is just a bad idea. If we try to predict how exactly the AI will kill us, we will probably be wrong because the AI is smarter than us. If we try to enumerate possible attack vectors one by one and defend against them, this might slow down the AI by some years (or weeks), but the AI can probably come up with attack vectors we haven’t thought of, or invent new technologies. Also, the AI might need multiple attack vectors and technologies, eg to ensure its own energy supply. If that kind of AI has acquired enough resources/compute, we will probably lose against that AI, and defending against concrete scenarios might have some benefits, but it seems preferable to focus on not building that kind of AI in the first place.
An analogy I find helpful: If you play chess against Magnus Carlsen, then you don’t know in advance which concrete moves he will play, but you know that you will lose.