There’s another problem with the norm of lying for the greater good: One, it is very easy for biased human minds to convince themselves of the lie and become systematically distorted from their path. To put it in Sarah Constantin’s words:
The problem with this reasoning should be obvious. The argument would work just as well if EA did no good at all, and only claimed to do good.
Arbitrary or unreliable claims of moral superiority function like bubbles in economic markets. If you never check the value of a stock against some kind of ground-truth reality, if everyone only looks at its current price and buys or sells based on that, we’ll see prices being inflated based on no reason at all. If you don’t insist on honesty in people’s claims of “for the greater good”, you’ll get hijacked into helping people who aren’t serving the greater good at all.
Another problem is you are much more vulnerable to Goodharting yourself, and eventually you will use it for motivated reasoning, where your pet causes can be lied about, and outsiders can’t tell if the organization is actually doing what it claims. While I think the dentological notion of honesty is too exploitable and naive for the 21st century, I definitely agree with Holden that lying should not be a norm, as well as misleading people should also not be a norm, but a regrettable exception.
There’s another problem with the norm of lying for the greater good: One, it is very easy for biased human minds to convince themselves of the lie and become systematically distorted from their path. To put it in Sarah Constantin’s words:
Another problem is you are much more vulnerable to Goodharting yourself, and eventually you will use it for motivated reasoning, where your pet causes can be lied about, and outsiders can’t tell if the organization is actually doing what it claims. While I think the dentological notion of honesty is too exploitable and naive for the 21st century, I definitely agree with Holden that lying should not be a norm, as well as misleading people should also not be a norm, but a regrettable exception.