In essence, is this is an alteration of information hazard that accounts for a group’s propensity to commit harm? That seems like a useful concept. It seems small, but I think it’s important to differentiate between the groups that information is being spread to (e.g. spreading information on this forum probably has lower risk than spreading the same information on 4chan).
In regards to your question about whether EA should consider spirit hazards, I haven’t read deeply enough into the concept of information hazards to know if this is well-trodden ground (I just read the top forum posts from your link) but it seems like spirit hazards are only one half of the cost-benefit equation in sharing information. You suggest that risky information should be shared only with decisionmakers, but I can think of two scenarios in which that would not be ideal.
1. Accountability—when information is spread more broadly, decisionmakers can be held accountable for their actions (e.g. an AI company could be called out for unsafe practices if it’s broadly known what practices are unsafe and why). 2. Wisdom of crowds—spreading information about a risk allows many people to independently try to develop solutions to that risk, which can be more successful than a lone decisionmaker trying to develop solutions.
This whole balance is analogous to cybersecurity, where security researchers actively spread information on vulnerabilities, because that allows other security researchers to fix them + learn from them, to stay ahead of hackers. Of course, not every scenario calls for the same information spread—explosives experts would definitely not want to spread bomb recipes on internet forums in the hope of helping out other explosives experts.
Perhaps the difference between cybersecurity and explosives that makes information spread more beneficial in the former case is that hacking is much more accessible to malicious actors than building a bomb, so dangerous information will inevitably be discovered by someone. So spirit hazard depends on the likelihood of information spreading to bad actors anyway. In domains which have a higher cost of involvement (and thus are more centralized) this is less likely to be the case, so maybe spirit hazard is still high in cases that we generally care about.
Ok, that can be a better interpretation: adding the audience’s capacity to commit harm into info hazards considerations.
That makes sense that the information about the existence of potentially harmful info can be shared also with people who can hold decisionmakers accountable to use their knowledge positively.
Whether this will succeed can depend on the attitude of the public toward the topic, which can depend on the ‘spirit’ of those who share the info. Using your examples, it an info comes from a resource such as the EA Forum, where the norm is to focus on impact and prevent harm, then even public who would normatively influence decisionmakers can have a similarly safe preferences regarding the topic.
However, one can also imagine that the public will seek to present that the info can be used for selfish gain or harm (since people may want to ‘side’ with a harmful entity due to fear, seek to gain standing or attention on social media due to posting about a threat, or aim to gain privilege for their group by harming others). Since the general public is not trained in double-thinking the possible impacts of their actions and since risk memes can spread faster than safety ones, publicly sharing the existence of risky topics, in good faith, can normalize and expedite harmful advancement of these subjects.
Crowd wisdom can apply when solutions are not already developed, only decisionmakers need to implement them, and when the public has the skills to come up with these solutions. For example, if only a treaty needs to be signed and budget spent on lab safety, then a few individuals can complete it. Or, people untrained in universal values research can have a limited ability to contribute to it.
Cybersecurity is an example of a field that requires cooperation of many experts who are not more likely to engage in a risky use of the info. Bomb recipes info, on the other hand, does not extensively help safety experts (who may specialize in legislation and regulations to prevent harm due to explosives) and could motivate otherwise uninterested actors to research this topic further. In this, cybersecurity can be analogous to AI safety and explosives info to biosecurity.
Spirit hazard can also make (empower or inspire) bad actors. The lower the cost of involvement (e. g. due to consequences, financial and other resources cost), the riskier it can be to share the info (and not necessarily more likely that (potentially) bad actors could have it already). So, risky info with low cost of negative involvement should not be shared.
Risky info should be shared if i) the cost of involvement is high, ii) it is highly unlikely that the group would use it to increase the riskiness of norms, iii) it is likely that not sharing this security info with the group would make decisionmakers advance risk, and iv) this topic is not subject to the unilateralist’s curse (e. g. if one person tries to make an explosive many others would prevent them from doing so).
In essence, is this is an alteration of information hazard that accounts for a group’s propensity to commit harm? That seems like a useful concept. It seems small, but I think it’s important to differentiate between the groups that information is being spread to (e.g. spreading information on this forum probably has lower risk than spreading the same information on 4chan).
In regards to your question about whether EA should consider spirit hazards, I haven’t read deeply enough into the concept of information hazards to know if this is well-trodden ground (I just read the top forum posts from your link) but it seems like spirit hazards are only one half of the cost-benefit equation in sharing information. You suggest that risky information should be shared only with decisionmakers, but I can think of two scenarios in which that would not be ideal.
1. Accountability—when information is spread more broadly, decisionmakers can be held accountable for their actions (e.g. an AI company could be called out for unsafe practices if it’s broadly known what practices are unsafe and why).
2. Wisdom of crowds—spreading information about a risk allows many people to independently try to develop solutions to that risk, which can be more successful than a lone decisionmaker trying to develop solutions.
This whole balance is analogous to cybersecurity, where security researchers actively spread information on vulnerabilities, because that allows other security researchers to fix them + learn from them, to stay ahead of hackers. Of course, not every scenario calls for the same information spread—explosives experts would definitely not want to spread bomb recipes on internet forums in the hope of helping out other explosives experts.
Perhaps the difference between cybersecurity and explosives that makes information spread more beneficial in the former case is that hacking is much more accessible to malicious actors than building a bomb, so dangerous information will inevitably be discovered by someone. So spirit hazard depends on the likelihood of information spreading to bad actors anyway. In domains which have a higher cost of involvement (and thus are more centralized) this is less likely to be the case, so maybe spirit hazard is still high in cases that we generally care about.
Ok, that can be a better interpretation: adding the audience’s capacity to commit harm into info hazards considerations.
That makes sense that the information about the existence of potentially harmful info can be shared also with people who can hold decisionmakers accountable to use their knowledge positively.
Whether this will succeed can depend on the attitude of the public toward the topic, which can depend on the ‘spirit’ of those who share the info. Using your examples, it an info comes from a resource such as the EA Forum, where the norm is to focus on impact and prevent harm, then even public who would normatively influence decisionmakers can have a similarly safe preferences regarding the topic.
However, one can also imagine that the public will seek to present that the info can be used for selfish gain or harm (since people may want to ‘side’ with a harmful entity due to fear, seek to gain standing or attention on social media due to posting about a threat, or aim to gain privilege for their group by harming others). Since the general public is not trained in double-thinking the possible impacts of their actions and since risk memes can spread faster than safety ones, publicly sharing the existence of risky topics, in good faith, can normalize and expedite harmful advancement of these subjects.
Crowd wisdom can apply when solutions are not already developed, only decisionmakers need to implement them, and when the public has the skills to come up with these solutions. For example, if only a treaty needs to be signed and budget spent on lab safety, then a few individuals can complete it. Or, people untrained in universal values research can have a limited ability to contribute to it.
Cybersecurity is an example of a field that requires cooperation of many experts who are not more likely to engage in a risky use of the info. Bomb recipes info, on the other hand, does not extensively help safety experts (who may specialize in legislation and regulations to prevent harm due to explosives) and could motivate otherwise uninterested actors to research this topic further. In this, cybersecurity can be analogous to AI safety and explosives info to biosecurity.
Spirit hazard can also make (empower or inspire) bad actors. The lower the cost of involvement (e. g. due to consequences, financial and other resources cost), the riskier it can be to share the info (and not necessarily more likely that (potentially) bad actors could have it already). So, risky info with low cost of negative involvement should not be shared.
Risky info should be shared if i) the cost of involvement is high, ii) it is highly unlikely that the group would use it to increase the riskiness of norms, iii) it is likely that not sharing this security info with the group would make decisionmakers advance risk, and iv) this topic is not subject to the unilateralist’s curse (e. g. if one person tries to make an explosive many others would prevent them from doing so).