I originally wrote a different response to Wei’s comment, but it wasn’t direct enough. I’m copying the first part here since it may be helpful in explaining what I mean by “moral preferences” vs “personal preferences”:
Each person has a range of preferences, which it’s often convenient to break down into “moral preferences” and “personal preferences”. This isn’t always a clear distinction, but the main differences:
1. Moral preferences are much more universalisable and less person-specific (e.g. “I prefer that people aren’t killed” vs “I prefer that I’m not killed”).
2. Moral preferences are associated with a meta-preference that everyone has the same moral preferences. This is why we feel so strongly that we need to find a shared moral “truth”. Fortunately, most people are in agreement in our societies on the most basic moral questions.
3. Moral preferences are associated with a meta-preference that they are consistent, simple, and actionable. This is why we feel so strongly that we need to find coherent moral theories rather than just following our intuitions.
4. Moral preferences are usually phrased as “X is right/wrong” and “people should do right and not do wrong” rather than “I prefer X”. This often misleads people into thinking that their moral preferences are just pointers to some aspect of reality, the “objective moral truth”, which is what people “objectively should do”.
When we reflect on our moral preferences and try to make them more consistent and actionable, we often end up condensing our initial moral preferences (aka moral intuitions) into moral theories like utilitarianism. Note that we could do this for other preferences as well (e.g. “my theory of food is that I prefer things which have more salt than sugar”) but because I don’t have strong meta-preferences about my food preferences, I don’t bother doing so.
The relationship between moral preferences and personal preferences can be quite complicated. People act on both, but often have a meta-preference to pay more attention to their moral preferences than they currently do. I’d count someone as a utilitarian if they have moral preferences that favour utilitarianism, and these are a non-negligible component of their overall preferences.
I originally wrote a different response to Wei’s comment, but it wasn’t direct enough. I’m copying the first part here since it may be helpful in explaining what I mean by “moral preferences” vs “personal preferences”:
Each person has a range of preferences, which it’s often convenient to break down into “moral preferences” and “personal preferences”. This isn’t always a clear distinction, but the main differences:
1. Moral preferences are much more universalisable and less person-specific (e.g. “I prefer that people aren’t killed” vs “I prefer that I’m not killed”).
2. Moral preferences are associated with a meta-preference that everyone has the same moral preferences. This is why we feel so strongly that we need to find a shared moral “truth”. Fortunately, most people are in agreement in our societies on the most basic moral questions.
3. Moral preferences are associated with a meta-preference that they are consistent, simple, and actionable. This is why we feel so strongly that we need to find coherent moral theories rather than just following our intuitions.
4. Moral preferences are usually phrased as “X is right/wrong” and “people should do right and not do wrong” rather than “I prefer X”. This often misleads people into thinking that their moral preferences are just pointers to some aspect of reality, the “objective moral truth”, which is what people “objectively should do”.
When we reflect on our moral preferences and try to make them more consistent and actionable, we often end up condensing our initial moral preferences (aka moral intuitions) into moral theories like utilitarianism. Note that we could do this for other preferences as well (e.g. “my theory of food is that I prefer things which have more salt than sugar”) but because I don’t have strong meta-preferences about my food preferences, I don’t bother doing so.
The relationship between moral preferences and personal preferences can be quite complicated. People act on both, but often have a meta-preference to pay more attention to their moral preferences than they currently do. I’d count someone as a utilitarian if they have moral preferences that favour utilitarianism, and these are a non-negligible component of their overall preferences.
I found this very helpful.