Have you considered that for some people, the most agenty thing to do would be to change their decision-procedure so it becomes less “agenty”? [...] your idealized self-image/system-2/goal.
Yes, I have, for myself, and I declined, because I didn’t see how it would help. Your analogy is indeed an analogy for what you believe, but it is not evidence. I asked you why you advise reducing one’s agency, and the mere fact that it is theoretically possible for it to be a good idea doesn’t demonstrate that it in fact is.
Note that in the analogy, if the robbers aren’t stupid, they will kill one of your family members because taking the pill is a form of non-compliance, wait for the pill to wear off, and then ask the same question minus one. If the hostage crisis is a good analogy for your internal self, what is to stop “system 1” from breaking its promises or being clever? That’s basically how addiction works: take a pill a day or start withdrawal. Doing that? Good. Now take one more a day or start withdrawal. Replace “pills” with “time/effort not spent on altruism” and you’re doomed.
With EA being too demanding, you don’t even have to change your goals, it suffices to adjust your expectations to yourself.
The utility quota strikes again. Here, your problem is that EA can be “too demanding”—apparently there is some kind of better morality by which you can say that EA is being Wrong, but somehow you don’t decide to use that morality for EA instead.
“Careful reflection” also isn’t enough for humans to converge on an answer for themselves. If it was, tens of thousands of philosophers should have managed to map out morality, and we wouldn’t need the likes of MIRI.
Are you equating “morality” with “figuring out an answer for one’s goals that converges for all humans”? If yes, then I suspect that the reference of “morality” fails because goals probably don’t converge (completely). Why is there so much disagreement in moral philosophy? To a large extent, people seem to be trying to answer different questions. In addition, some people are certainly being irrational at what they’re trying to do, e.g. they fail to distinguish between things that they care about terminally and things they only care about instrumentally, or they might fail to even ask fundamental questions.
No, in this case I’m referring to true morality, whatever it might be. If your explanation was true—if the divergence in theories was because of goals failing to converge or people answering different questions—we would expect philosophers to actually answer their questions. However, what we see is that philosophers do not manage to answer their own questions: every moral theory has holes and unanswered questions, not just differences of opinion between the author and others.
If there were moral consensus, then obviously there would be a single morality, so the lack of a consensus carries some evidential weight, but not much. People are great at creating disagreements over nothing, and ethics is complex enough to be opaque, so we would expect moral disagreement in both worlds with a single coherent morality for humanity and worlds without one.
I agree, see my 2nd footnote in my original post. The point where we disagree is whether you can infer from an existing disagreement about goals that at least one participant is necessary being irrational/wrong about her goals. I’m saying that’s not the case.
And my point is that the inference is obsolete: since neither person has psychological knowledge thirty years ahead of their time, both are necessarily wrong and irrational about their goals.
I’m sufficiently confident that I’m either misunderstanding you or that you’re wrong about your morality [...]
I probably thought about my values more than most EAs and have gone through unusual lengths to lay out my arguments and reasons. If you want to try to find mistakes, inconsistencies or thought experiment that would make me change them, feel free to send me a PM here or on FB.
Why not do this publicly? Why not address the thought experiment I proposed?
In addition, people will disagree about the specifics of even such “straightfoward” things as what “altruism” implies. Is it altruistic to give someone a sleeping pill against their will if they plan to engage in some activity you consider bad for them? Is it altruistic to turn rocks into happy people? People will disagree about what they would choose here, and it’s entirely possible that they are not making any meaningful sort of mistake in the process of disagreeing.
Those things are less straightforward than string theory, in the sense of Kolmogorov complexity. The fact that we can compress those queries into sentences which are simpler to introduce one to than algebra is testimony to how similar humans are.
OK, but even so, I would in such a case at least be right about the theoretical possibility of there being people to whom my advice applies correctly.
Yes, but you couldn’t act on it without the benefit of hindsight. It is also theoretically possible that the moon is made out of cheese and that all information to the contrary has been spread by communist mice.
For what it’s worth, I consider it dangerous that EA will be associated with a lot of “bad press” if people drop out due to it being too stressful.
This should be included in the productivity calculation, naturally. Just like your own mental wellbeing naturally should be part of EA optimisation.
All my experience with pitching EA so far indicates that it’s bad to be too demanding.
And all your experience has had the constant factor of being pitched by you, someone who believes “optimising for EA” being tiring and draining is all part of the plan.
Yes, if “optimising for EA” drains you, you should do less of it, because you aren’t optimising for EA (unless there’s an overlap, which there probably is, in which case you should keep doing the things which optimise for EA).
As a general point, I object to your choice of words: I don’t think my posts ever argued for people to stop trying.
You’re telling people not to try to optimise their full lives to EA right now. If that is what they were trying before, then you are arguing for people to stop trying, QED.
On the topic of choice of words, though, in the original post you write “The same of course also applies to women.”—this implies that the author of the quote did not intend his statement to apply to women, despite using a (or at the time perhaps the) grammatically correct way to refer to an unspecified person of any gender (“he”). Considering you use a gendered pronoun to refer to unspecified people of any gender as well (“she”), I’m confused why you would wrongly ‘correct’ someone out like that.
If the hostage crisis is a good analogy for your internal self, what is to stop “system 1” from breaking its promises or being clever?
There are problems to every approach. Talk about your commitment to others, they will remind you. I’m not saying this whole strategy always works, but I’m quite sure there are many people for whom it is the best idea to try.
Regarding the “utility quota”, what I mean by “personal moral expecations”: Basically, this just makes the point that it is useless to beat yourself up over things you cannot change. And yet we often do this, feel sad about things we probably couldn’t have done differently. (One interesting hypothesis for this reaction is described here.)
People are great at creating disagreements over nothing, and ethics is complex enough to be opaque, so we would expect moral disagreement in both worlds with a single coherent morality for humanity and worlds without one.
Note that if this were true, you still need reasons why you expect there to be just one human morality. I know what EY wrote on the topic, and I find it question-begging and unconvincing. What EY is saying is that human utility-function_1s are complex and similar. What I’m interested in, and what I think you and EY should also be interested in, are utility-function_2s. But that’s another discussion, I’ve been meaning to write up my views on the topic of metaethics and goal-uncertainty, but I expect it’ll take me at least a few months until I get around to it.
This doesn’t really prove my case by itself, but it’s an awesome quote nevertheless, so I’m including it here (David Hume, Enquiry):
“It might reasonably be expected in questions which have been canvassed and disputed with great eagerness, since the first origin of science and philosophy, that the meaning of all the terms, at least, should have been agreed upon among the disputants; and our enquiries, in the course of two thousand years, been able to pass from words to the true and real subject of the controversy. For how easy may it seem to give exact definitions of the terms employed in reasoning, and make these definitions, not the mere sound of words, the object of future scrutiny and examination? But if we consider the matter more narrowly, we shall be apt to draw a quite opposite conclusion. From this circumstance alone, that a controversy has been long kept on foot, and remains still undecided, we may presume that there is some ambiguity in the expression, and that the disputants affix different ideas to the terms employed in the controversy.”
Why not do this publicly? Why not address the thought experiment I proposed?
Lack of time, given that I’ve already written a lot of text on the topic. And because I’m considering to publish some of it at some point in the future, I’m wary of posting long excerpts of it online.
Those things are less straightforward than string theory, in the sense of Kolmogorov complexity. The fact that we can compress those queries into sentences which are simpler to introduce one to than algebra is testimony to how similar humans are.
Isn’t the whole point of string theory that it is pretty simple (in terms of Kolmogorov complexity that is, not in whether I can understand it)? If anything, this would be testimony to how good humans are at natural speech as opposed to math. Although humans aren’t that good at natural speech, because they often don’t notice when they’re being confused or talking past each other. But this is being too metaphorical.
I don’t really understand your point here. Aren’t you presupposing that there is one answer people will converge on with these cases? I’ve talked to very intelligent people about these sorts of questions, and we’ve narrowed down all the factual disagreements we could think of. Certainly it is possible that I and the people I was talking to (who disagreed with my views), were missing something. But it seems more probable that answers just don’t always converge.
And all your experience has had the constant factor of being pitched by you, someone who believes “optimising for EA” being tiring and draining is all part of the plan.
What, I thought you were saying that, at least more so than I’m saying it.
You’re telling people not to try to optimise their full lives to EA right now. If that is what they were trying before, then you are arguing for people to stop trying, QED.
Differentiate between 1) “ways of trying to accomplish a goal, e.g. in terms of decisional algorithms or habits” and 2) “pursuing a goal by whichever means are most effective”. I did not try to discourage anyone from 2), and that’s clearly what is relevant. I’m encouraging people to stop trying a particular variant of 1) because I believe that particular variant of 1) works for some people (to some extent), but not for all of them. It’s a spectrum of course, not just two distinct modes of going about the problem.
Considering you use a gendered pronoun to refer to unspecified people of any gender as well (“she”), I’m confused why you would wrongly ‘correct’ someone out like that.
Let’s not get into this, this discussion is already long enough. I can say that I see the point of your last remark, I did not mean to imply that Williams himself, given the time the text was written, was being sexist.
part 2⁄2
Yes, I have, for myself, and I declined, because I didn’t see how it would help. Your analogy is indeed an analogy for what you believe, but it is not evidence. I asked you why you advise reducing one’s agency, and the mere fact that it is theoretically possible for it to be a good idea doesn’t demonstrate that it in fact is.
Note that in the analogy, if the robbers aren’t stupid, they will kill one of your family members because taking the pill is a form of non-compliance, wait for the pill to wear off, and then ask the same question minus one. If the hostage crisis is a good analogy for your internal self, what is to stop “system 1” from breaking its promises or being clever? That’s basically how addiction works: take a pill a day or start withdrawal. Doing that? Good. Now take one more a day or start withdrawal. Replace “pills” with “time/effort not spent on altruism” and you’re doomed.
The utility quota strikes again. Here, your problem is that EA can be “too demanding”—apparently there is some kind of better morality by which you can say that EA is being Wrong, but somehow you don’t decide to use that morality for EA instead.
No, in this case I’m referring to true morality, whatever it might be. If your explanation was true—if the divergence in theories was because of goals failing to converge or people answering different questions—we would expect philosophers to actually answer their questions. However, what we see is that philosophers do not manage to answer their own questions: every moral theory has holes and unanswered questions, not just differences of opinion between the author and others.
If there were moral consensus, then obviously there would be a single morality, so the lack of a consensus carries some evidential weight, but not much. People are great at creating disagreements over nothing, and ethics is complex enough to be opaque, so we would expect moral disagreement in both worlds with a single coherent morality for humanity and worlds without one.
And my point is that the inference is obsolete: since neither person has psychological knowledge thirty years ahead of their time, both are necessarily wrong and irrational about their goals.
Why not do this publicly? Why not address the thought experiment I proposed?
Those things are less straightforward than string theory, in the sense of Kolmogorov complexity. The fact that we can compress those queries into sentences which are simpler to introduce one to than algebra is testimony to how similar humans are.
Yes, but you couldn’t act on it without the benefit of hindsight. It is also theoretically possible that the moon is made out of cheese and that all information to the contrary has been spread by communist mice.
This should be included in the productivity calculation, naturally. Just like your own mental wellbeing naturally should be part of EA optimisation.
And all your experience has had the constant factor of being pitched by you, someone who believes “optimising for EA” being tiring and draining is all part of the plan.
Yes, if “optimising for EA” drains you, you should do less of it, because you aren’t optimising for EA (unless there’s an overlap, which there probably is, in which case you should keep doing the things which optimise for EA).
You’re telling people not to try to optimise their full lives to EA right now. If that is what they were trying before, then you are arguing for people to stop trying, QED.
On the topic of choice of words, though, in the original post you write “The same of course also applies to women.”—this implies that the author of the quote did not intend his statement to apply to women, despite using a (or at the time perhaps the) grammatically correct way to refer to an unspecified person of any gender (“he”). Considering you use a gendered pronoun to refer to unspecified people of any gender as well (“she”), I’m confused why you would wrongly ‘correct’ someone out like that.
There are problems to every approach. Talk about your commitment to others, they will remind you. I’m not saying this whole strategy always works, but I’m quite sure there are many people for whom it is the best idea to try.
Regarding the “utility quota”, what I mean by “personal moral expecations”: Basically, this just makes the point that it is useless to beat yourself up over things you cannot change. And yet we often do this, feel sad about things we probably couldn’t have done differently. (One interesting hypothesis for this reaction is described here.)
Note that if this were true, you still need reasons why you expect there to be just one human morality. I know what EY wrote on the topic, and I find it question-begging and unconvincing. What EY is saying is that human utility-function_1s are complex and similar. What I’m interested in, and what I think you and EY should also be interested in, are utility-function_2s. But that’s another discussion, I’ve been meaning to write up my views on the topic of metaethics and goal-uncertainty, but I expect it’ll take me at least a few months until I get around to it.
This doesn’t really prove my case by itself, but it’s an awesome quote nevertheless, so I’m including it here (David Hume, Enquiry):
Lack of time, given that I’ve already written a lot of text on the topic. And because I’m considering to publish some of it at some point in the future, I’m wary of posting long excerpts of it online.
Isn’t the whole point of string theory that it is pretty simple (in terms of Kolmogorov complexity that is, not in whether I can understand it)? If anything, this would be testimony to how good humans are at natural speech as opposed to math. Although humans aren’t that good at natural speech, because they often don’t notice when they’re being confused or talking past each other. But this is being too metaphorical.
I don’t really understand your point here. Aren’t you presupposing that there is one answer people will converge on with these cases? I’ve talked to very intelligent people about these sorts of questions, and we’ve narrowed down all the factual disagreements we could think of. Certainly it is possible that I and the people I was talking to (who disagreed with my views), were missing something. But it seems more probable that answers just don’t always converge.
What, I thought you were saying that, at least more so than I’m saying it.
Differentiate between 1) “ways of trying to accomplish a goal, e.g. in terms of decisional algorithms or habits” and 2) “pursuing a goal by whichever means are most effective”. I did not try to discourage anyone from 2), and that’s clearly what is relevant. I’m encouraging people to stop trying a particular variant of 1) because I believe that particular variant of 1) works for some people (to some extent), but not for all of them. It’s a spectrum of course, not just two distinct modes of going about the problem.
Let’s not get into this, this discussion is already long enough. I can say that I see the point of your last remark, I did not mean to imply that Williams himself, given the time the text was written, was being sexist.