As a toy example, say that S(x) is some bounded sigmoid function, and my utility function is to maximize E[S(x)]; it’s always going to be the case that E[S(x1)]≥E[S(x2)]⇔x1≥x2 so I am in some sense scope sensitive, but I don’t think I’m open to Pascal’s mugging
This seems right to me.
I think it means that there is something which we value linearly, but that thing might be a complicated function of happiness, preference satisfaction, etc.
Yeah, I have no quibbles with this. FWIW, I personally didn’t interpret the passage as saying this, so if that’s what’s meant, I’d recommend reformulating.
(To gesture at where I’m coming from: “in expectation bring about more paperclips” seems much more specific than “in expectation increase some function defined over the number of paperclips”; and I assumed that this statement was similar, except pointing towards the physical structure of “intuitively valuable aspects of individual lives” rather than the physical structure of “paperclips”. In particular, “intuitively valuable aspects of individual lives” seems like a local phenomena rather than something defined over world-histories, and you kind of need to define your utility function over world-histories to represent risk-aversion.)
This seems right to me.
Yeah, I have no quibbles with this. FWIW, I personally didn’t interpret the passage as saying this, so if that’s what’s meant, I’d recommend reformulating.
(To gesture at where I’m coming from: “in expectation bring about more paperclips” seems much more specific than “in expectation increase some function defined over the number of paperclips”; and I assumed that this statement was similar, except pointing towards the physical structure of “intuitively valuable aspects of individual lives” rather than the physical structure of “paperclips”. In particular, “intuitively valuable aspects of individual lives” seems like a local phenomena rather than something defined over world-histories, and you kind of need to define your utility function over world-histories to represent risk-aversion.)
That makes sense; your interpretation does seem reasonable, so perhaps a rephrase a would be helpful.