I don’t think Anthropic is Molochian. I imagine Moloch as a situation where everyone knows it’s bad, but nobody can do anything about it, even though everyone collectively could make it stop if they could coordinate. Anthropic isn’t like that because they aren’t being forced to make things worse; they’re making an active and unforced decision to make things worse. The whole “we’re trapped by the incentives!” thing is an extremely lame excuse that we should not let them get away with.
Anthropic itself isn’t Molochian—The molochian-ness (idk if I’m using this term right I don’t read SSC) is that any time there is a disagreement in the community over some issue, and one side aligns more with the real worlds outer loops (money, status, intellectual sexiness), that side will naturally acquire more power within the movement because the movement does not have any way to counteract this other than persuasion which is increasingly difficult as the problems we deal with get more abstract and complex.
Thanks for thinking about all this! You’re right about Anthropic in that sense. I do state that canonical Moloch doesn’t quite fit here. But my concern is a lil downstream from Ant choosing/trapped. I don’t see that as being particularly consequential. My concern is about what happens when their employees’ wealth becomes the dominant funding source for the organizations meant to oversee them. That dynamic operates the same way whether we think Ant is trapped or a willing one, IMO.
The “measurable outcomes” thing is distinct from Moloch IMO, but I do think it’s an important factor in why AI safety work has been less impactful than it could’ve been: people are spending too much effort on Streetlight Effect research (this is especially true of AI-company-funded safety research).
I don’t think Anthropic is Molochian. I imagine Moloch as a situation where everyone knows it’s bad, but nobody can do anything about it, even though everyone collectively could make it stop if they could coordinate. Anthropic isn’t like that because they aren’t being forced to make things worse; they’re making an active and unforced decision to make things worse. The whole “we’re trapped by the incentives!” thing is an extremely lame excuse that we should not let them get away with.
Anthropic itself isn’t Molochian—The molochian-ness (idk if I’m using this term right I don’t read SSC) is that any time there is a disagreement in the community over some issue, and one side aligns more with the real worlds outer loops (money, status, intellectual sexiness), that side will naturally acquire more power within the movement because the movement does not have any way to counteract this other than persuasion which is increasingly difficult as the problems we deal with get more abstract and complex.
Yeah that sounds reasonable to me
Thanks for thinking about all this! You’re right about Anthropic in that sense. I do state that canonical Moloch doesn’t quite fit here. But my concern is a lil downstream from Ant choosing/trapped. I don’t see that as being particularly consequential. My concern is about what happens when their employees’ wealth becomes the dominant funding source for the organizations meant to oversee them. That dynamic operates the same way whether we think Ant is trapped or a willing one, IMO.
The “measurable outcomes” thing is distinct from Moloch IMO, but I do think it’s an important factor in why AI safety work has been less impactful than it could’ve been: people are spending too much effort on Streetlight Effect research (this is especially true of AI-company-funded safety research).