I think the “tech will be more dangerous later” argument is underappreciated, and it’s partly because the ability of AI to speed up research is overestimated. From your passage here:
It’s unclear how much marginal impacts matter here: if the malicious AI has to run 100 biological experiments in the span of a month or only 50, these may be similarly easy.
This seems like a drastic underestimate of how many experiments would be required to make an x-risk deadly virus (as opposed to just a really really bad one). It’s easy to make a virus that is reliably deadly, and one that reliably spreads, but the two tradeoff against each other in a way that makes reliable world-murder ridiculously difficult. I don’t think a mere 100 experiments is enough to overcome this.
Yep, seems reasonable, I don’t really have any clue here. One consideration is that this AI is probably way better than all the human scientists and can design particularly high-value experiments, also biological simulations will likely be much better in the future. Maybe the bio-security community gets a bunch of useful stuff done by then which makes the AI’s job even harder.
Interesting post!
I think the “tech will be more dangerous later” argument is underappreciated, and it’s partly because the ability of AI to speed up research is overestimated. From your passage here:
This seems like a drastic underestimate of how many experiments would be required to make an x-risk deadly virus (as opposed to just a really really bad one). It’s easy to make a virus that is reliably deadly, and one that reliably spreads, but the two tradeoff against each other in a way that makes reliable world-murder ridiculously difficult. I don’t think a mere 100 experiments is enough to overcome this.
Yep, seems reasonable, I don’t really have any clue here. One consideration is that this AI is probably way better than all the human scientists and can design particularly high-value experiments, also biological simulations will likely be much better in the future. Maybe the bio-security community gets a bunch of useful stuff done by then which makes the AI’s job even harder.