If we really take infohazards seriously, we shouldn’t just be imagining EAs casually reading draft essays, sharing them, and the ideas gradually percolating out to potential bad actors.
Instead, we should take a fully adversarial, red-team mind-set, and ask, if a large, highly capable geopolitical power wanted to mine EA insights for potential applications of AI technology that could give them an advantage (even at some risk to humanity in general), how would we keep that from happening?
We would be naive to think that intelligence agencies of various major countries that are interested in AI don’t have at least a few intelligence analysts reading EA Forum, LessWrong, & Alignment Forum, looking for tips that might be useful—but that we might consider infohazards.
Strongly endorsed this comment.
If we really take infohazards seriously, we shouldn’t just be imagining EAs casually reading draft essays, sharing them, and the ideas gradually percolating out to potential bad actors.
Instead, we should take a fully adversarial, red-team mind-set, and ask, if a large, highly capable geopolitical power wanted to mine EA insights for potential applications of AI technology that could give them an advantage (even at some risk to humanity in general), how would we keep that from happening?
We would be naive to think that intelligence agencies of various major countries that are interested in AI don’t have at least a few intelligence analysts reading EA Forum, LessWrong, & Alignment Forum, looking for tips that might be useful—but that we might consider infohazards.