I think this post is confused on a number of levels.
First, as far as ideal behavior is concerned integrity isn’t a relevant concept. The ideal utilitarian agent will simply always behave in the manner that optimizes expected future utility factoring in the effect that breaking one’s word or other actions will have on the perceptions (and thus future actions) of other people.
Now the post rightly notes that as a limited human agent we aren’t truly able to engage in this kind of analysis. Both because of our computational limitations and our inability to perfectly deceive it is beneficial to adopt heuristics about not lying, stabbing people in the back etc.. (which we may judge to be worth abandoning in exceptional situations).
However, the post gives us no reason to believe it’s particular interpretation of integrity “being straightforward” is the best such heuristic. It merely asserts the author’s belief that this somehow works out to be the best.
This brings us to the second major point, even though the post acknowledges the very reason for considering integrity is that, “I find the ideal of integrity very viscerally compelling, significantly moreso than other abstract beliefs or principles that I often act on.” the post proceeds to act as if it was considering what kind of integrity like notion would be appropriate to design into (or socially construct) in some alternative society of purely rational agents.
Obviously, the way we should act depends hugely on the way in which others will interpret our actions and respond to them. In the actual world WE WILL BE TRUSTED TO THE EXTENT WE RESPECT THE STANDARD SOCIETAL NOTIONS OF INTEGRITY AND TRUST. It doesn’t matter if some other alternate notion of integrity might have been better to have if we don’t show integrity in the traditional manner we will be punished.
In particular, “being straightforward” will often needlessly imperil people’s estimation of our integrity. For example, consider the usual kinds of assurances we give to friends and family that we “will be there for them no matter what” and that “we wouldn’t ever abandon them.” In truth pretty much everyone, if presented with sufficient data showing their friend or family member to be a horrific serial killer with every intention of continuing to torture and kill people, would turn them in even in the face of protestations of innocence. Does that mean that instead of saying “I’ll be there for you whatever happens” we should say “I’ll be there for you as long as the balance of probability doesn’t suggest that supporting you will cost more than 5 QALYs” (quality adjusted life years)?
No, because being straightforward in that sense causes most people to judge us as weird and abnormal and thereby trust us less. Even though everyone understands at some level that these kind of assurances are only true ceterus parabus actually being straightforward about that fact is unusual enough that it causes other people to suspect that they don’t understand our emotions/motivations and thus give us less trust.
In short: yes, the obvious point that we should adopt some kind of heuristic of keeping our word and otherwise modeling integrity is true. However, the suggestion that this nice simple heuristic is somehow the best one is completely unjustified.
The ideal utilitarian agent will simply always behave in the manner that optimizes expected future utility factoring in the effect that breaking one’s word or other actions will have on the perceptions (and thus future actions) of other people
This view is not broadly accepted amongst the EA community. At the very least, this view is self-defeating in the following sense: such an “ideal utilitarian” should not try to convince other people to be an ideal utilitarian, and should attempt to become a non-ideal utilitarian ASAP (see e.g. Parfit’s hitchhiker for the standard counterexample, though obviously there are more realistic cases).
However, the post gives us no reason to believe it’s particular interpretation of integrity “being straightforward” is the best such heuristic. It merely asserts the author’s belief that this somehow works out to be the best.
I argued for my conclusion. You may not buy the arguments, and indeed they aren’t totally tight, but calling it “mere assertion” seems silly.
the very reason for considering integrity is that, “I find the ideal of integrity very viscerally compelling, significantly moreso than other abstract beliefs or principles that I often act on.”
This is neither true, nor what I said.
WE WILL BE TRUSTED TO THE EXTENT WE RESPECT THE STANDARD SOCIETAL NOTIONS OF INTEGRITY AND TRUST
This is what it looks like when something is asserted without argument.
I do agree roughly with this sentiment, but only if it is interpreted sufficiently broadly that it is consistent with my post.
Does that mean that instead of saying “I’ll be there for you whatever happens” we should say “I’ll be there for you as long as the balance of probability doesn’t suggest that supporting you will cost more than 5 QALYs” (quality adjusted life years)?
I tried to spell out pretty explicitly what I recommend in the post, right at the beginning (“when I imagine picking an action, I pretend that picking it causes everyone to know that I am the kind of person who picks that option”), and it clearly doesn’t recommend anything like this.
You seem to use “being straightforward” in a different way than I do. Saying “I’ll be there for you whatever happens” is straightforward if you actually mean the thing that people will understand you as meaning.
Re your first point yup they won’t try to recruit others to that belief but so what? That’s already a bullet any utilitarian has to bite thanks to examples like the aliens who will torture the world if anyone believes utilitarianism is true or ties to act as of it is. There is absolutely nothing self defeating here.
Indeed if we define utilitarianism as simply the belief that ones preference relation on possible worlds is dictated by the total utility in then it follows by definition that the best act an agent can take are just the ones which maximize utility. So maybe the better way to phrase this is as: why care what the agent who pledges to utilitarianism in some way and wants to recruit others might need to do or act that’s a distraction from the simple question of what in fact maximizes utility. If that means convincing everyone not to be utilitarians then so be it.
--
And yes re the rest of your points I guess I just don’t see why it matters what would be good to do if other agents respond in some way you argue would be reasonable. Indeed, what makes consequentialism consequentialism is that you aren’t acting based on what would happen if you imagine interacting with idealized agents like a Kantianesque theory might consider but what actually happens when you actually act.
I agree the caps were aggressive and I apologize for that and I agree I’m not trying to produce evidence which says that in fact how people respond to supposed signals of integrity tends to match what they see as evidence you follow the standard norms. That’s just something people need to consult their own experience and ask themselves if, in their experience, thay tends to be true. Ultimately I think that it’s just not true that a priori analysis of what should make people see you as trustworthy or have any other social reaction is a good guide to what they will do?
But I guess that is just going to return to point 1 and our different conceptions of what is utilitarianism requires.
“WE WILL BE TRUSTED TO THE EXTENT WE RESPECT THE STANDARD SOCIETAL NOTIONS OF INTEGRITY AND TRUST”
I think there is a lot to this, but I feel it can be subsumed into Paul’s rule of thumb:
You should follow a standard societal notion of what is decent behaviour (unless you say ahead of time that you won’t in this case) if you want people to have always thought that you are the kind of person who does that.
Because following standard social rules that everyone assumes to exist is an important part of being able to coordinate with others without very high communication and agreement overheads, you want to at least meet that standard (including following some norms you might have reservations about). Of course this doesn’t preclude you meeting a higher standard if having a reputation for going above and beyond would be useful to you (as Paul argues it often is for most of us).
I think this post is confused on a number of levels.
First, as far as ideal behavior is concerned integrity isn’t a relevant concept. The ideal utilitarian agent will simply always behave in the manner that optimizes expected future utility factoring in the effect that breaking one’s word or other actions will have on the perceptions (and thus future actions) of other people.
Now the post rightly notes that as a limited human agent we aren’t truly able to engage in this kind of analysis. Both because of our computational limitations and our inability to perfectly deceive it is beneficial to adopt heuristics about not lying, stabbing people in the back etc.. (which we may judge to be worth abandoning in exceptional situations).
However, the post gives us no reason to believe it’s particular interpretation of integrity “being straightforward” is the best such heuristic. It merely asserts the author’s belief that this somehow works out to be the best.
This brings us to the second major point, even though the post acknowledges the very reason for considering integrity is that, “I find the ideal of integrity very viscerally compelling, significantly moreso than other abstract beliefs or principles that I often act on.” the post proceeds to act as if it was considering what kind of integrity like notion would be appropriate to design into (or socially construct) in some alternative society of purely rational agents.
Obviously, the way we should act depends hugely on the way in which others will interpret our actions and respond to them. In the actual world WE WILL BE TRUSTED TO THE EXTENT WE RESPECT THE STANDARD SOCIETAL NOTIONS OF INTEGRITY AND TRUST. It doesn’t matter if some other alternate notion of integrity might have been better to have if we don’t show integrity in the traditional manner we will be punished.
In particular, “being straightforward” will often needlessly imperil people’s estimation of our integrity. For example, consider the usual kinds of assurances we give to friends and family that we “will be there for them no matter what” and that “we wouldn’t ever abandon them.” In truth pretty much everyone, if presented with sufficient data showing their friend or family member to be a horrific serial killer with every intention of continuing to torture and kill people, would turn them in even in the face of protestations of innocence. Does that mean that instead of saying “I’ll be there for you whatever happens” we should say “I’ll be there for you as long as the balance of probability doesn’t suggest that supporting you will cost more than 5 QALYs” (quality adjusted life years)?
No, because being straightforward in that sense causes most people to judge us as weird and abnormal and thereby trust us less. Even though everyone understands at some level that these kind of assurances are only true ceterus parabus actually being straightforward about that fact is unusual enough that it causes other people to suspect that they don’t understand our emotions/motivations and thus give us less trust.
In short: yes, the obvious point that we should adopt some kind of heuristic of keeping our word and otherwise modeling integrity is true. However, the suggestion that this nice simple heuristic is somehow the best one is completely unjustified.
I apologize in advance if I’m a bit snarky.
This view is not broadly accepted amongst the EA community. At the very least, this view is self-defeating in the following sense: such an “ideal utilitarian” should not try to convince other people to be an ideal utilitarian, and should attempt to become a non-ideal utilitarian ASAP (see e.g. Parfit’s hitchhiker for the standard counterexample, though obviously there are more realistic cases).
I argued for my conclusion. You may not buy the arguments, and indeed they aren’t totally tight, but calling it “mere assertion” seems silly.
This is neither true, nor what I said.
This is what it looks like when something is asserted without argument.
I do agree roughly with this sentiment, but only if it is interpreted sufficiently broadly that it is consistent with my post.
I tried to spell out pretty explicitly what I recommend in the post, right at the beginning (“when I imagine picking an action, I pretend that picking it causes everyone to know that I am the kind of person who picks that option”), and it clearly doesn’t recommend anything like this.
You seem to use “being straightforward” in a different way than I do. Saying “I’ll be there for you whatever happens” is straightforward if you actually mean the thing that people will understand you as meaning.
Re your first point yup they won’t try to recruit others to that belief but so what? That’s already a bullet any utilitarian has to bite thanks to examples like the aliens who will torture the world if anyone believes utilitarianism is true or ties to act as of it is. There is absolutely nothing self defeating here.
Indeed if we define utilitarianism as simply the belief that ones preference relation on possible worlds is dictated by the total utility in then it follows by definition that the best act an agent can take are just the ones which maximize utility. So maybe the better way to phrase this is as: why care what the agent who pledges to utilitarianism in some way and wants to recruit others might need to do or act that’s a distraction from the simple question of what in fact maximizes utility. If that means convincing everyone not to be utilitarians then so be it.
--
And yes re the rest of your points I guess I just don’t see why it matters what would be good to do if other agents respond in some way you argue would be reasonable. Indeed, what makes consequentialism consequentialism is that you aren’t acting based on what would happen if you imagine interacting with idealized agents like a Kantianesque theory might consider but what actually happens when you actually act.
I agree the caps were aggressive and I apologize for that and I agree I’m not trying to produce evidence which says that in fact how people respond to supposed signals of integrity tends to match what they see as evidence you follow the standard norms. That’s just something people need to consult their own experience and ask themselves if, in their experience, thay tends to be true. Ultimately I think that it’s just not true that a priori analysis of what should make people see you as trustworthy or have any other social reaction is a good guide to what they will do?
But I guess that is just going to return to point 1 and our different conceptions of what is utilitarianism requires.
“WE WILL BE TRUSTED TO THE EXTENT WE RESPECT THE STANDARD SOCIETAL NOTIONS OF INTEGRITY AND TRUST”
I think there is a lot to this, but I feel it can be subsumed into Paul’s rule of thumb:
You should follow a standard societal notion of what is decent behaviour (unless you say ahead of time that you won’t in this case) if you want people to have always thought that you are the kind of person who does that.
Because following standard social rules that everyone assumes to exist is an important part of being able to coordinate with others without very high communication and agreement overheads, you want to at least meet that standard (including following some norms you might have reservations about). Of course this doesn’t preclude you meeting a higher standard if having a reputation for going above and beyond would be useful to you (as Paul argues it often is for most of us).