This project would be valuable if the costs outweighed the benefits.
It could be relatively expensive (in person-hours) to run (there might be a tonne of publications to vet!) and relies on us being good (low false positive, high recall) at identifying biohazards (my prior is that this is actually pretty hard and those biohazardous publications would happen anyway). We’d also need to worry about incentivising people to make it harder to tell that their work is dangerous.
Biohazards are bad but preventing biohazards might have low marginal returns when some already exist. It’s not that any new biohazard is fine; it’s that marginal biohazard might be pretty rare (like something that advances the possibilities) relative to “acceptable” sort of non-marginal biohazards (i.e., another bad genome for something as bad as what’s already public knowledge). Other work might advance what’s possible without being a biohazard persay (i.e., AlphaFold).
I think a way to verify if this is a good project might be to talk to the Spiez lab. They run a biosecurity conference every year and invite anyone doing work that could be dangerous to attend.
I’d like to see a more comprehensive model for what biosecurity risk looks like that can motivate a comparison of project ideas. In the absence of that, it’s really hard to say where we get the most benefit.
This project would be valuable if the costs outweighed the benefits.
It could be relatively expensive (in person-hours) to run (there might be a tonne of publications to vet!) and relies on us being good (low false positive, high recall) at identifying biohazards (my prior is that this is actually pretty hard and those biohazardous publications would happen anyway). We’d also need to worry about incentivising people to make it harder to tell that their work is dangerous.
Biohazards are bad but preventing biohazards might have low marginal returns when some already exist. It’s not that any new biohazard is fine; it’s that marginal biohazard might be pretty rare (like something that advances the possibilities) relative to “acceptable” sort of non-marginal biohazards (i.e., another bad genome for something as bad as what’s already public knowledge). Other work might advance what’s possible without being a biohazard persay (i.e., AlphaFold).
I think a way to verify if this is a good project might be to talk to the Spiez lab. They run a biosecurity conference every year and invite anyone doing work that could be dangerous to attend.
I’m happy to chat more about it.
Regarding the problem being expensive (because there are many publications to vet or something like that):
I think I could help optimize that part, it sounds like an engineering problem, which is a thing I’m ok with
I’d like to see a more comprehensive model for what biosecurity risk looks like that can motivate a comparison of project ideas. In the absence of that, it’s really hard to say where we get the most benefit.
We don’t have to drop all our plans in favor of the one top plan
Or as Dumbledore said: