Sure, for the most part, Claude will probably just be used for common requests, but Anthropic have no way of guaranteeing this. You cannot do this by policy, especially if it’s on Amazon hardware that you don’t control and can’t inspect. Ranking agencies by ‘cooperativeness’ should also be taken as lip service until they have a proven mechanism for doing so.
So they are revealing that, to them, AI safety doesn’t mean that they try to prevent AI from doing harm, just that they try to prevent it from doing unintended harm. This is a significant moment for them and I fear what it portends for the whole industry.
Military applications of AI are not an idle concern. AI systems are already being used to increase military capacity by generating and analysing targets faster than humans can (and in this case, seemingly without much oversight). Palantir’s own technology likely also allows police organisations to defer responsibility for racist policing to AI systems.
Sure, for the most part, Claude will probably just be used for common requests, but Anthropic have no way of guaranteeing this. You cannot do this by policy, especially if it’s on Amazon hardware that you don’t control and can’t inspect. Ranking agencies by ‘cooperativeness’ should also be taken as lip service until they have a proven mechanism for doing so.
So they are revealing that, to them, AI safety doesn’t mean that they try to prevent AI from doing harm, just that they try to prevent it from doing unintended harm. This is a significant moment for them and I fear what it portends for the whole industry.