I think this post is my favourite for laying out why a really convincing utilitarian argument for something which is common sense very bad shouldn’t move you. From memory Eliezer says something like ~Thinking there’s a really good utilitarian argument doesn’t mean the ends justify the means, it just means your flawed brain with weird motivations feels like there’s a really good utilitarian argument. Your uncertainty in that always dominates and leaves room for common sense arguments, even when you feel really extra super sure. Common sense morality rules like “the ends shouldn’t justify the means” arose because people in practice are very miscallibrated about when the ends actually justify the means so we should take the outside view and assume we are too.~
(By miscallibrated I think I could defend a claim like “90% of the time people think the ends definitely justify the means and this clashes with common sense morality they are wrong”)
I might be butchering the post though so you should definitely read it.
I think this post is my favourite for laying out why a really convincing utilitarian argument for something which is common sense very bad shouldn’t move you. From memory Eliezer says something like ~Thinking there’s a really good utilitarian argument doesn’t mean the ends justify the means, it just means your flawed brain with weird motivations feels like there’s a really good utilitarian argument. Your uncertainty in that always dominates and leaves room for common sense arguments, even when you feel really extra super sure. Common sense morality rules like “the ends shouldn’t justify the means” arose because people in practice are very miscallibrated about when the ends actually justify the means so we should take the outside view and assume we are too.~
(By miscallibrated I think I could defend a claim like “90% of the time people think the ends definitely justify the means and this clashes with common sense morality they are wrong”)
I might be butchering the post though so you should definitely read it.
https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t-justify-means-among-humans