not to be accountable for “pre-harm” enforcement of AI Safety standards (ie. wait for a catastrophe before enforcing any liability).
“if a catastrophic event does occur … the quality of the company’s SSP should be a factor in determining whether the developer exercised ‘reasonable care.’”. (ie. if your safety protocols look good, you can be let off the hook for the consequences of catastrophe).
Also significantly weakening whistleblower protections.
Yeah this seems like a reasonable summary of why the letter is probably bad, but tbc I thought it was questionable before I was able to read the letter (so I don’t want to get credit for doing the homework).
Hypothetically, if companies like Anthropic or OpenAI wanted to create a set of heuristics that lets them acquire power while generating positive (or at least neutral) safety-washing PR among credulous nerds, they can have an modus operandi of:
a) publicly claim to be positive on serious regulations with teeth, whistleblowing, etc, and that the public should not sign a blank check for AI companies inventing among the most dangerous technologies in history, while
b) privately do almost everything in their power to undermine serious regulations or oversight or public accountability.
If we live in that world (which tbc I’m not saying is certain), someone needs to say that the emperor has no clothes. I don’t like being that someone, but here we are.
https://www.lesswrong.com/posts/s58hDHX2GkFDbpGKD/linch-s-shortform?commentId=RfJsudqwEMwTR5S5q
TL;DR
Anthropic are pushing for two key changes
not to be accountable for “pre-harm” enforcement of AI Safety standards (ie. wait for a catastrophe before enforcing any liability).
“if a catastrophic event does occur … the quality of the company’s SSP should be a factor in determining whether the developer exercised ‘reasonable care.’”. (ie. if your safety protocols look good, you can be let off the hook for the consequences of catastrophe).
Also significantly weakening whistleblower protections.
Yeah this seems like a reasonable summary of why the letter is probably bad, but tbc I thought it was questionable before I was able to read the letter (so I don’t want to get credit for doing the homework).
Hypothetically, if companies like Anthropic or OpenAI wanted to create a set of heuristics that lets them acquire power while generating positive (or at least neutral) safety-washing PR among credulous nerds, they can have an modus operandi of:
a) publicly claim to be positive on serious regulations with teeth, whistleblowing, etc, and that the public should not sign a blank check for AI companies inventing among the most dangerous technologies in history, while
b) privately do almost everything in their power to undermine serious regulations or oversight or public accountability.
If we live in that world (which tbc I’m not saying is certain), someone needs to say that the emperor has no clothes. I don’t like being that someone, but here we are.