This is so cool, I meant to alert you to the excellent work of Dr. Esvelt on DNA sequencing… I was thinking, can you use cryptography to collate knowledge but prevent the risk and dynamics issues with some persons knowing too much about risky sequences and check sequences automatically?
How this, according to my limited understanding, can work:
1) Biosecurity researchers from around the world submit risky sequences to your database, which secures it by cryptography so that no one can read it (even the coders—each works on a part of the code and there are too many too different (e. g. nationality) with high understanding of the risk)
2) Sequences from any research are checked against the database, without human participation. The output does not identify the sequence, only the level of risk and further checking and other measures requirements or the recommendation to discontinue the research.
There has to be trust in the decisionmaking of the AI (cost-benefit analysis: better not to vaccinate against this sickness but not cause a high-limitations pandemic) and international agreement on following the recommendations.
3) Human biosecurity researchers continue to check research identified by the database as with various levels of risk (both identifying potential risk based on context understanding and reviewing research with the sequences that the security researcher submitted). The latter coordinates risky sequence thought development so biosecurity researchers should be vetted, specialize in reading research but have limited abilities to translate it into actions, and observed for contacts with risky persons, such as the military or extra-military groups. Also, it should be checked if a risky sequence proliferates among some researchers, especially in a short time. If there is a possibility of researchers’ risky engagement, the researchers should be publicly identified (for national and international security institutions) and offered alternative engagement with much lower risk. Risky sequences can be added to the database and the risk level of some research possibly increased by humans.
4) Pathogen researchers can appeal to the AI decisions, if they think that the risk-benefit analysis was conducted erroneously. In this case human experts check the outputs that the system explains without identifying the sequences and conduct context analysis. Biosecurity researchers can reduce (or increase) the risk level.
5) For the context analysis, human observants should be gathering information about laboratories and individuals and forming perspectives on their risk based on complex mental models based on systems understanding. These can be digitized as notes for other humans (can be reduced to numbers but the context understanding should be retained). These insights should be discussed to improve collective understanding.
6) It should be popular among the biosecurity researchers to develop and run measures that reduce research risk, from implementing infrastructure and training staff members, to running international cooperation programs on biosecurity preparedness which normalize the shamefulness of proliferation and point out the limited effectiveness of risky programs due to preparedness, the perhaps much higher risk of those who would be planning an attack, the ineffectiveness of targeting individuals due to institutional set up (you have to make agreements with institutions, not target an individual and think your problems will be solved), and retaliation by various economic, political, and military means. Certifications can be issued.
7) Network analysis with individuals, institutions, and research particulars can be conducted. This can improve context analysis.
-
I can imagine that if you raise awareness among ‘garage labs’ that anyone can cause a pandemic, no one is prepared to deal with this, and it has a great potential as a treat, then people are going to research threats in their garages, interested in taking over the world. How are you going to limit the popularization of risky research while ensuring that sound standards are followed universally?
What are you going to do about terrorist groups who may not be interested in participating in your program? If nothing, how will you contribute to others’ efforts in improving the situations to limit terrorism (including by reducing or not raising interest due to threat potential)?
What about groups that already research pandemic potential pathogens, such as militaries, who are not interested in participating in your program, but could be interested in learning more? Are you going to attract them, perhaps narrating a possibility of benefit from being an insider, while hoping that by participating with you they become interested in biosecurity or defense/preparedness rather than offensive measures development? Or, how are you going to approach these or support the work of others, if anyhow?
Are you also going to cover RNA, or other sequences, which would constitute the entirety of biological risk? (What if someone researches mushrooms, for example, or I mean something not covered by your project—if it is actually impossible to cover all, would it be better to just not popularize it—the more people think the more they come up with innovative solutions—but is it that currently, many nations could destroy large parts of the world by various means but are not doing it because it seems suboptimal for them, so even if people think about possibilities, they are not going to undertake harmful actions?)
This is so cool, I meant to alert you to the excellent work of Dr. Esvelt on DNA sequencing… I was thinking, can you use cryptography to collate knowledge but prevent the risk and dynamics issues with some persons knowing too much about risky sequences and check sequences automatically?
How this, according to my limited understanding, can work:
1) Biosecurity researchers from around the world submit risky sequences to your database, which secures it by cryptography so that no one can read it (even the coders—each works on a part of the code and there are too many too different (e. g. nationality) with high understanding of the risk)
2) Sequences from any research are checked against the database, without human participation. The output does not identify the sequence, only the level of risk and further checking and other measures requirements or the recommendation to discontinue the research.
There has to be trust in the decisionmaking of the AI (cost-benefit analysis: better not to vaccinate against this sickness but not cause a high-limitations pandemic) and international agreement on following the recommendations.
3) Human biosecurity researchers continue to check research identified by the database as with various levels of risk (both identifying potential risk based on context understanding and reviewing research with the sequences that the security researcher submitted). The latter coordinates risky sequence thought development so biosecurity researchers should be vetted, specialize in reading research but have limited abilities to translate it into actions, and observed for contacts with risky persons, such as the military or extra-military groups. Also, it should be checked if a risky sequence proliferates among some researchers, especially in a short time. If there is a possibility of researchers’ risky engagement, the researchers should be publicly identified (for national and international security institutions) and offered alternative engagement with much lower risk. Risky sequences can be added to the database and the risk level of some research possibly increased by humans.
4) Pathogen researchers can appeal to the AI decisions, if they think that the risk-benefit analysis was conducted erroneously. In this case human experts check the outputs that the system explains without identifying the sequences and conduct context analysis. Biosecurity researchers can reduce (or increase) the risk level.
5) For the context analysis, human observants should be gathering information about laboratories and individuals and forming perspectives on their risk based on complex mental models based on systems understanding. These can be digitized as notes for other humans (can be reduced to numbers but the context understanding should be retained). These insights should be discussed to improve collective understanding.
6) It should be popular among the biosecurity researchers to develop and run measures that reduce research risk, from implementing infrastructure and training staff members, to running international cooperation programs on biosecurity preparedness which normalize the shamefulness of proliferation and point out the limited effectiveness of risky programs due to preparedness, the perhaps much higher risk of those who would be planning an attack, the ineffectiveness of targeting individuals due to institutional set up (you have to make agreements with institutions, not target an individual and think your problems will be solved), and retaliation by various economic, political, and military means. Certifications can be issued.
7) Network analysis with individuals, institutions, and research particulars can be conducted. This can improve context analysis.
-
I can imagine that if you raise awareness among ‘garage labs’ that anyone can cause a pandemic, no one is prepared to deal with this, and it has a great potential as a treat, then people are going to research threats in their garages, interested in taking over the world. How are you going to limit the popularization of risky research while ensuring that sound standards are followed universally?
What are you going to do about terrorist groups who may not be interested in participating in your program? If nothing, how will you contribute to others’ efforts in improving the situations to limit terrorism (including by reducing or not raising interest due to threat potential)?
What about groups that already research pandemic potential pathogens, such as militaries, who are not interested in participating in your program, but could be interested in learning more? Are you going to attract them, perhaps narrating a possibility of benefit from being an insider, while hoping that by participating with you they become interested in biosecurity or defense/preparedness rather than offensive measures development? Or, how are you going to approach these or support the work of others, if anyhow?
Are you also going to cover RNA, or other sequences, which would constitute the entirety of biological risk? (What if someone researches mushrooms, for example, or I mean something not covered by your project—if it is actually impossible to cover all, would it be better to just not popularize it—the more people think the more they come up with innovative solutions—but is it that currently, many nations could destroy large parts of the world by various means but are not doing it because it seems suboptimal for them, so even if people think about possibilities, they are not going to undertake harmful actions?)