I wonder if you have some addendums to the point of secrecy and the AI safety and EA community’s thoughts about info hazards. Are we building a community that automatically believes in both the risks and the competition being higher because organizations (e.g. MIRI) shout wolf while keeping why they shout wolf relatively secret (i.e. their experiments in making aligned AI). I don’t know what my own opinion on this is, but would you argue for a more open policy, given these insights?
Thanks for this. I’m more counselling “be careful about secrecy” rather than “don’t be secret”. Especially be careful about secret sprints, being told you’re in a race but can’t see the secret information why, and careful about “you have to take part in this secret project”.
On the capability side, the shift in AI/ML publication and release norms towards staged release (not releasing full model immediately but carefully checking for misuse potential first), structured access (through APIs) and so on has been positive, I think.
On the risks/analysis side, MIRI have their own “nondisclosed-by-default” policy on publication. CSER and other academic research groups tend towards more of a “disclosed-by-default” policy.
I wonder if you have some addendums to the point of secrecy and the AI safety and EA community’s thoughts about info hazards. Are we building a community that automatically believes in both the risks and the competition being higher because organizations (e.g. MIRI) shout wolf while keeping why they shout wolf relatively secret (i.e. their experiments in making aligned AI). I don’t know what my own opinion on this is, but would you argue for a more open policy, given these insights?
Thanks for this. I’m more counselling “be careful about secrecy” rather than “don’t be secret”. Especially be careful about secret sprints, being told you’re in a race but can’t see the secret information why, and careful about “you have to take part in this secret project”.
On the capability side, the shift in AI/ML publication and release norms towards staged release (not releasing full model immediately but carefully checking for misuse potential first), structured access (through APIs) and so on has been positive, I think.
On the risks/analysis side, MIRI have their own “nondisclosed-by-default” policy on publication. CSER and other academic research groups tend towards more of a “disclosed-by-default” policy.