Ok, that can be a better interpretation: adding the audience’s capacity to commit harm into info hazards considerations.
That makes sense that the information about the existence of potentially harmful info can be shared also with people who can hold decisionmakers accountable to use their knowledge positively.
Whether this will succeed can depend on the attitude of the public toward the topic, which can depend on the ‘spirit’ of those who share the info. Using your examples, it an info comes from a resource such as the EA Forum, where the norm is to focus on impact and prevent harm, then even public who would normatively influence decisionmakers can have a similarly safe preferences regarding the topic.
However, one can also imagine that the public will seek to present that the info can be used for selfish gain or harm (since people may want to ‘side’ with a harmful entity due to fear, seek to gain standing or attention on social media due to posting about a threat, or aim to gain privilege for their group by harming others). Since the general public is not trained in double-thinking the possible impacts of their actions and since risk memes can spread faster than safety ones, publicly sharing the existence of risky topics, in good faith, can normalize and expedite harmful advancement of these subjects.
Crowd wisdom can apply when solutions are not already developed, only decisionmakers need to implement them, and when the public has the skills to come up with these solutions. For example, if only a treaty needs to be signed and budget spent on lab safety, then a few individuals can complete it. Or, people untrained in universal values research can have a limited ability to contribute to it.
Cybersecurity is an example of a field that requires cooperation of many experts who are not more likely to engage in a risky use of the info. Bomb recipes info, on the other hand, does not extensively help safety experts (who may specialize in legislation and regulations to prevent harm due to explosives) and could motivate otherwise uninterested actors to research this topic further. In this, cybersecurity can be analogous to AI safety and explosives info to biosecurity.
Spirit hazard can also make (empower or inspire) bad actors. The lower the cost of involvement (e. g. due to consequences, financial and other resources cost), the riskier it can be to share the info (and not necessarily more likely that (potentially) bad actors could have it already). So, risky info with low cost of negative involvement should not be shared.
Risky info should be shared if i) the cost of involvement is high, ii) it is highly unlikely that the group would use it to increase the riskiness of norms, iii) it is likely that not sharing this security info with the group would make decisionmakers advance risk, and iv) this topic is not subject to the unilateralist’s curse (e. g. if one person tries to make an explosive many others would prevent them from doing so).
Ok, that can be a better interpretation: adding the audience’s capacity to commit harm into info hazards considerations.
That makes sense that the information about the existence of potentially harmful info can be shared also with people who can hold decisionmakers accountable to use their knowledge positively.
Whether this will succeed can depend on the attitude of the public toward the topic, which can depend on the ‘spirit’ of those who share the info. Using your examples, it an info comes from a resource such as the EA Forum, where the norm is to focus on impact and prevent harm, then even public who would normatively influence decisionmakers can have a similarly safe preferences regarding the topic.
However, one can also imagine that the public will seek to present that the info can be used for selfish gain or harm (since people may want to ‘side’ with a harmful entity due to fear, seek to gain standing or attention on social media due to posting about a threat, or aim to gain privilege for their group by harming others). Since the general public is not trained in double-thinking the possible impacts of their actions and since risk memes can spread faster than safety ones, publicly sharing the existence of risky topics, in good faith, can normalize and expedite harmful advancement of these subjects.
Crowd wisdom can apply when solutions are not already developed, only decisionmakers need to implement them, and when the public has the skills to come up with these solutions. For example, if only a treaty needs to be signed and budget spent on lab safety, then a few individuals can complete it. Or, people untrained in universal values research can have a limited ability to contribute to it.
Cybersecurity is an example of a field that requires cooperation of many experts who are not more likely to engage in a risky use of the info. Bomb recipes info, on the other hand, does not extensively help safety experts (who may specialize in legislation and regulations to prevent harm due to explosives) and could motivate otherwise uninterested actors to research this topic further. In this, cybersecurity can be analogous to AI safety and explosives info to biosecurity.
Spirit hazard can also make (empower or inspire) bad actors. The lower the cost of involvement (e. g. due to consequences, financial and other resources cost), the riskier it can be to share the info (and not necessarily more likely that (potentially) bad actors could have it already). So, risky info with low cost of negative involvement should not be shared.
Risky info should be shared if i) the cost of involvement is high, ii) it is highly unlikely that the group would use it to increase the riskiness of norms, iii) it is likely that not sharing this security info with the group would make decisionmakers advance risk, and iv) this topic is not subject to the unilateralist’s curse (e. g. if one person tries to make an explosive many others would prevent them from doing so).