I think this post and Yudkowski’s Twitter thread that started it are probably harmful to the cause of AI safety.
OpenAI is one of the top AI labs worldwide, and the difference between their cooperation and antagonism to the AI safety community means a lot for the overall project. Elon Musk might be one of the top private funders of AI research, so his cooperation is also important.
I think that both this post and the Twitter thread reduce the likelihood of cooperation without accomplishing enough in return. I think that the potential to do harm to potential cooperation is about the same for a well-researched, well-considered comment as for an off-the-cuff comment, but the potential to do good is much higher for comments of the first type than the second. So, for comments that might cause offense, the standard for research and consideration should be higher than usual.
This post: it’s extremely hard to understand what exactly OpenAI is being accused of doing wrong. Your sentence “The small marginal impact of having anything to do with popularizing AI Safety dominates any good these movements many have produced.” reads to me as an argument that Yudkowsky is wrong, and the fact that the launch lead indirectly to more AI safety discourse means that it was a positive. However, this doesn’t match the valence of your post.
Your second argument, that most of their safety researchers left, is indeed some cause for concern (edit: although seemingly quite independent from your first point). However, surely it is perfectly possible to ask the departed safety researchers whether they themselves think that their departures should be taken as a signal of no confidence in OpenAI’s commitment to safety before advocating actions to be taken against them. To clarify: you may or may not get a helpful response, but I think that this is an easy thing to do, is clearly a reasonable step if you are wondering what these departures mean, and I think you should take such easy & reasonable steps before advocating a position like this.
If OpenAI is pursuing extremely risky research without proper regard to safety, then the argument set out here ought to be far stronger. If not, then it is inappropriate to advocate doing harm to OpenAI researchers.
The Twitter thread: To an outsider, it seems like this concerns regarding the language employed at OpenAI’s launch were resolved quickly in a manner that addressed the concerns of safety advocates. If the resolution did not address their concerns, and safety advocates think that this should be widely known, then that should be explained clearly, and this thread did no such thing.
It looked to me like Yudkowsky was arguing, as he often likes to, that contributions to AI risk are cardinally greater than contributions to anything else when assessing someone’s impact on something. It is not obvious to me that he intended this particular episode to have more impact than his many other statements to this effect. Nonetheless, it seems to have done so (at least, I’m seeing it pop up in several different venues), and I at least would appreciate if he could clarify if there is an ongong issue here and what it is, or not.
Thanks for the recommendation. I spent about an hour looking for contact info, but was only able to find 5 public addresses of ex-OpenAI employees involved in the recent exodus. I emailed them all, and provided an anonymous Google Form as well. I’ll provide an update if I do hear back from anyone.
OpenAI is one of the top AI labs worldwide, and the difference between their cooperation and antagonism to the AI safety community means a lot for the overall project. Elon Musk might be one of the top private funders of AI research, so his cooperation is also important.
Ah okay, so it is about not antagonizing our new overlords, got it.
I think this post and Yudkowski’s Twitter thread that started it are probably harmful to the cause of AI safety.
OpenAI is one of the top AI labs worldwide, and the difference between their cooperation and antagonism to the AI safety community means a lot for the overall project. Elon Musk might be one of the top private funders of AI research, so his cooperation is also important.
I think that both this post and the Twitter thread reduce the likelihood of cooperation without accomplishing enough in return. I think that the potential to do harm to potential cooperation is about the same for a well-researched, well-considered comment as for an off-the-cuff comment, but the potential to do good is much higher for comments of the first type than the second. So, for comments that might cause offense, the standard for research and consideration should be higher than usual.
This post: it’s extremely hard to understand what exactly OpenAI is being accused of doing wrong. Your sentence “The small marginal impact of having anything to do with popularizing AI Safety dominates any good these movements many have produced.” reads to me as an argument that Yudkowsky is wrong, and the fact that the launch lead indirectly to more AI safety discourse means that it was a positive. However, this doesn’t match the valence of your post.
Your second argument, that most of their safety researchers left, is indeed some cause for concern (edit: although seemingly quite independent from your first point). However, surely it is perfectly possible to ask the departed safety researchers whether they themselves think that their departures should be taken as a signal of no confidence in OpenAI’s commitment to safety before advocating actions to be taken against them. To clarify: you may or may not get a helpful response, but I think that this is an easy thing to do, is clearly a reasonable step if you are wondering what these departures mean, and I think you should take such easy & reasonable steps before advocating a position like this.
If OpenAI is pursuing extremely risky research without proper regard to safety, then the argument set out here ought to be far stronger. If not, then it is inappropriate to advocate doing harm to OpenAI researchers.
The Twitter thread: To an outsider, it seems like this concerns regarding the language employed at OpenAI’s launch were resolved quickly in a manner that addressed the concerns of safety advocates. If the resolution did not address their concerns, and safety advocates think that this should be widely known, then that should be explained clearly, and this thread did no such thing.
It looked to me like Yudkowsky was arguing, as he often likes to, that contributions to AI risk are cardinally greater than contributions to anything else when assessing someone’s impact on something. It is not obvious to me that he intended this particular episode to have more impact than his many other statements to this effect. Nonetheless, it seems to have done so (at least, I’m seeing it pop up in several different venues), and I at least would appreciate if he could clarify if there is an ongong issue here and what it is, or not.
Unfortunately we may be unlikely to get a statement from a departed safety researcher beyond mine (https://forum.effectivealtruism.org/posts/fmDFytmxwX9qBgcaX/why-aren-t-you-freaking-out-about-openai-at-what-point-would?commentId=WrWycenCHFgs8cak4), at least currently.
Thanks for the recommendation. I spent about an hour looking for contact info, but was only able to find 5 public addresses of ex-OpenAI employees involved in the recent exodus. I emailed them all, and provided an anonymous Google Form as well. I’ll provide an update if I do hear back from anyone.
Ah okay, so it is about not antagonizing our new overlords, got it.