Endorsing actions which, in expectation, bring about more intuitively valuable aspects of individual lives (e.g. happiness, preference-satisfaction, etc), or bring about fewer intuitively disvaluable aspects of individual lives
If this is the technical meaning of “in expectation”, this brings in a lot of baggage. I think it implicitly means that you value those things ~linearly in their amount (which makes the second statement superfluous?), and it opens you up to pascal’s mugging.
I think it means that there is something which we value linearly, but that thing might be a complicated function of happiness, preference satisfaction, etc.
As a toy example, say that S(x) is some bounded sigmoid function, and my utility function is to maximize E[S(x)]; it’s always going to be the case that E[S(x1)]≥E[S(x2)]⇔x1≥x2 so I am in some sense scope sensitive, but I don’t think I’m open to Pascal’s mugging. (Correct me if this is wrong though.)
As a toy example, say that S(x) is some bounded sigmoid function, and my utility function is to maximize E[S(x)]; it’s always going to be the case that E[S(x1)]≥E[S(x2)]⇔x1≥x2 so I am in some sense scope sensitive, but I don’t think I’m open to Pascal’s mugging
This seems right to me.
I think it means that there is something which we value linearly, but that thing might be a complicated function of happiness, preference satisfaction, etc.
Yeah, I have no quibbles with this. FWIW, I personally didn’t interpret the passage as saying this, so if that’s what’s meant, I’d recommend reformulating.
(To gesture at where I’m coming from: “in expectation bring about more paperclips” seems much more specific than “in expectation increase some function defined over the number of paperclips”; and I assumed that this statement was similar, except pointing towards the physical structure of “intuitively valuable aspects of individual lives” rather than the physical structure of “paperclips”. In particular, “intuitively valuable aspects of individual lives” seems like a local phenomena rather than something defined over world-histories, and you kind of need to define your utility function over world-histories to represent risk-aversion.)
If this is the technical meaning of “in expectation”, this brings in a lot of baggage. I think it implicitly means that you value those things ~linearly in their amount (which makes the second statement superfluous?), and it opens you up to pascal’s mugging.
I think it means that there is something which we value linearly, but that thing might be a complicated function of happiness, preference satisfaction, etc.
As a toy example, say that S(x) is some bounded sigmoid function, and my utility function is to maximize E[S(x)]; it’s always going to be the case that E[S(x1)]≥E[S(x2)]⇔x1≥x2 so I am in some sense scope sensitive, but I don’t think I’m open to Pascal’s mugging. (Correct me if this is wrong though.)
This seems right to me.
Yeah, I have no quibbles with this. FWIW, I personally didn’t interpret the passage as saying this, so if that’s what’s meant, I’d recommend reformulating.
(To gesture at where I’m coming from: “in expectation bring about more paperclips” seems much more specific than “in expectation increase some function defined over the number of paperclips”; and I assumed that this statement was similar, except pointing towards the physical structure of “intuitively valuable aspects of individual lives” rather than the physical structure of “paperclips”. In particular, “intuitively valuable aspects of individual lives” seems like a local phenomena rather than something defined over world-histories, and you kind of need to define your utility function over world-histories to represent risk-aversion.)
That makes sense; your interpretation does seem reasonable, so perhaps a rephrase a would be helpful.