That’s possible, but the responses really aren’t good. For example:
some of the ethics (and decision-theory) can get complicated (see footnote for a bit more discussion[10]
And then there’s a whole lot of moral philosophical-rationalist argument in the footnote. But he completely ignores an obvious option—working to oppose the potentially net-negative organisation. Or in this case: working towards getting an international treaty on AGI/ASI, that can rein in Anthropic and all the others engaged in the suicide race. I think Carlsmith could actually be highly impactful here, if he worked as a lobbyist or diplomat, and a public communicator (perhaps focused on an academic audience).
its primary weakness is underexploring how individual rationalization might systematically lead safety-concerned researchers to converge on similar justifications for joining labs they believe pose existential threats.
That’s possible, but the responses really aren’t good. For example:
And then there’s a whole lot of moral philosophical-rationalist argument in the footnote. But he completely ignores an obvious option—working to oppose the potentially net-negative organisation. Or in this case: working towards getting an international treaty on AGI/ASI, that can rein in Anthropic and all the others engaged in the suicide race. I think Carlsmith could actually be highly impactful here, if he worked as a lobbyist or diplomat, and a public communicator (perhaps focused on an academic audience).
See also (somewhat ironically), the AI roast: