These are really important points, thanks for starting a discussion on this topic. It seems like the infohazard manual by Chris Bakerlee and Tessa Alexanian is an excellent way forward. What do you think it is lacking? I am not an expert here; I’m not trying to imply their framework is complete, I genuinely want to learn more about what’s needed and it’s hard to convey tone via text.
You also mention Chris and Tessa’s manual “doesn’t echo a consensus among senior community members”. This surprises me, because I consider Chris to be a key senior community member in the biosecurity space. He is quite literally Open Philanthropy Project’s senior program associate in biosecurity and pandemic preparedness. Tessa also seems to be a leader in the biosecurity space; she has run all the biosecurity information sessions I’ve attended, has been the featured guest on the most popular biosecurity podcasts I recommend people listen to, and she has written extensively on possible new projects in the biosecurity space. At the bottom of the manual, Chris and Tessa thank a bunch of other people for contributing. This list of people encompasses a big percentage of who I consider to be senior members of the EA biosecurity community.
You clearly didn’t write this in a vacuum—indeed you seem to have written this post with feedback from Tessa—so I am again asking with genuine curiosity, what do other senior biosecurity community members think we should do about infohazards? And are these EA aligned people with different frameworks? Or are you referencing people outside the EA community who do work in government bio weapons programs or academic synthetic biology researchers?
Thanks for this comment, and thanks to Nadia for writing the post, I’m really happy to see it up on the forum!
Chris and I wrote the guidance for reading groups and early entrants to the field; this was partly because we felt that new folks are most likely to feel stuck/intimidated/forced-into-deference/etc. and because it’s where we most often found ourselves repeating the same advice over and over.
I think there are people whose opinions I respect who would disagree with the guidance in a few ways:
We recommend a few kinds of interpersonal interventions, and some people think this is a poor way to manage information hazards, and the community should aim to have much more explicit / regimented policies
We recommend quite a bit of caution about information hazards, which more conservative people might consider an attention hazard in and of itself (drawing attention to the fact that information that would enable harm could be generated)
We recommend quite a bit of caution about information hazards, which less conservative people might consider too encouraging of deference or secrecy (e.g. people who have run into more trouble doing successful advocacy or recruiting/fostering talent, people who have different models of infohazard dyanmics, people who are worried that a lack of transparency worsens the community’s prioritization)
We don’t cover a lot of common scenarios, as Nadia noted in her comment
(Side note: it’s always both flattering and confusing to be considered a “senior member” of this community. I suppose it’s true, because EA is very young, but I have many collaborators and colleagues who have decade(s) of experience working full-time on biorisk reduction, which I most certainly do not.)
(Side note: it’s always both flattering and confusing to be considered a “senior member” of this community. I suppose it’s true, because EA is very young, but I have many collaborators and colleagues who have decade(s) of experience working full-time on biorisk reduction, which I most certainly do not.)
I think part of this is that you are quite active on the forum, give talks at conferences, etc., making you much more visible to newcomers in the field. Others in biosecurity have decades of experience but are less visible to newcomers. Thus, it is understandable to infer that you are a “senior member.”
On what is lacking: It was written for reading groups, which is already a softly gatekept space. It doesn’t provide guidance on other communication channels: what people could write blogs or tweets about, what is safe to talk to LLMs about, what about google docs, etc. Indeed, I was concerned about infinitely abstract galaxy-brain infohazard potential from this very post.
On dissent:
I wanted to double down on the message in the document itself that is preliminary and not the be-all-end-all.
I have reached out to one person I have in mind within EA biosecurity who pushed back on the infohazard guidance document to give them the option to share their disagreement, potentially anonymously.
These are really important points, thanks for starting a discussion on this topic. It seems like the infohazard manual by Chris Bakerlee and Tessa Alexanian is an excellent way forward. What do you think it is lacking? I am not an expert here; I’m not trying to imply their framework is complete, I genuinely want to learn more about what’s needed and it’s hard to convey tone via text.
You also mention Chris and Tessa’s manual “doesn’t echo a consensus among senior community members”. This surprises me, because I consider Chris to be a key senior community member in the biosecurity space. He is quite literally Open Philanthropy Project’s senior program associate in biosecurity and pandemic preparedness. Tessa also seems to be a leader in the biosecurity space; she has run all the biosecurity information sessions I’ve attended, has been the featured guest on the most popular biosecurity podcasts I recommend people listen to, and she has written extensively on possible new projects in the biosecurity space. At the bottom of the manual, Chris and Tessa thank a bunch of other people for contributing. This list of people encompasses a big percentage of who I consider to be senior members of the EA biosecurity community.
You clearly didn’t write this in a vacuum—indeed you seem to have written this post with feedback from Tessa—so I am again asking with genuine curiosity, what do other senior biosecurity community members think we should do about infohazards? And are these EA aligned people with different frameworks? Or are you referencing people outside the EA community who do work in government bio weapons programs or academic synthetic biology researchers?
Thanks very much for your time!
Thanks for this comment, and thanks to Nadia for writing the post, I’m really happy to see it up on the forum!
Chris and I wrote the guidance for reading groups and early entrants to the field; this was partly because we felt that new folks are most likely to feel stuck/intimidated/forced-into-deference/etc. and because it’s where we most often found ourselves repeating the same advice over and over.
I think there are people whose opinions I respect who would disagree with the guidance in a few ways:
We recommend a few kinds of interpersonal interventions, and some people think this is a poor way to manage information hazards, and the community should aim to have much more explicit / regimented policies
We recommend quite a bit of caution about information hazards, which more conservative people might consider an attention hazard in and of itself (drawing attention to the fact that information that would enable harm could be generated)
We recommend quite a bit of caution about information hazards, which less conservative people might consider too encouraging of deference or secrecy (e.g. people who have run into more trouble doing successful advocacy or recruiting/fostering talent, people who have different models of infohazard dyanmics, people who are worried that a lack of transparency worsens the community’s prioritization)
We don’t cover a lot of common scenarios, as Nadia noted in her comment
(Side note: it’s always both flattering and confusing to be considered a “senior member” of this community. I suppose it’s true, because EA is very young, but I have many collaborators and colleagues who have decade(s) of experience working full-time on biorisk reduction, which I most certainly do not.)
I think part of this is that you are quite active on the forum, give talks at conferences, etc., making you much more visible to newcomers in the field. Others in biosecurity have decades of experience but are less visible to newcomers. Thus, it is understandable to infer that you are a “senior member.”
Thanks, really helpful context!
Looking around and realizing you’re the grown up now can be startling. When did I sign up for this responsibility????
Thanks for your comment!
On what is lacking: It was written for reading groups, which is already a softly gatekept space. It doesn’t provide guidance on other communication channels: what people could write blogs or tweets about, what is safe to talk to LLMs about, what about google docs, etc. Indeed, I was concerned about infinitely abstract galaxy-brain infohazard potential from this very post.
On dissent:
I wanted to double down on the message in the document itself that is preliminary and not the be-all-end-all.
I have reached out to one person I have in mind within EA biosecurity who pushed back on the infohazard guidance document to give them the option to share their disagreement, potentially anonymously.
Thank you, super helpful contect!