The easiest way, in my view, to make a near-optimal future very likely, conditional on non-extinction, is if value is bounded above.
There’s an argument that this is the common sense view. E.g. consider:
Common-sense Eutopia: In the future, there is a very large population with very high well-being; those people are able to do almost anything they want as long as they don’t harm others. They have complete scientific and technological understanding. War and conflict are things of the past. Environmental destruction has been wholly reversed; Earth is now a natural paradise. However, society is limited only to the solar system, and will come to an end once the Sun has exited its red giant phase, in about five billion years.
Does this seem to capture less than one 10^22th of all possible value? (Because there are ~10^22 affectable stars, so civilisation could be over 10^22 times as big). On my common-sense moral intuitions, no.
Making this argument stronger: Normally, quantities of value are defined are in terms of the value of risky gambles. So what it means to say that Common-sense Eutopia is less than one 10^22th of all possible value is that a gamble with a one in 10^22 chance of producing an ideal-society-across-all-the-stars, and a (1 − 1/10^22) chance of near-term extinction, is better than producing Common-sense Eutopia for certain.
But that seems wild. Of all the issues facing classical utilitarianism, this seems the most problematic to me.
The easiest way, in my view, to make a near-optimal future very likely, conditional on non-extinction, is if value is bounded above.
Yes.
Of all the issues facing classical utilitarianism, this seems the most problematic to me.
So you doubt fanaticism, the view that a tiny chance of an astronomically good outcome can be more valuable than a certainty of a decent outcome. What about in the case of certainty? Do you doubt the utilitarian’s objection to Common-sense Eutopia? This kind of aggregation seems important for the case for longtermism. (See the pages of paper dolls in What We Owe the Future.)
Yeah, I think the issue (for me) is not just about fanaticism. Give me Common-sense Eutopia or a gamble with a 90% chance of extinction and a 10% chance of Common-sense Eutopia 20 times the size, and it seems problematic to choose the gamble.
(To be clear—other views, on which value is diminishing, are also really problematic. We’re in impossibility theorem territory, and I see the whole things as a mess; I don’t have a positive view I’m excited about.)
Re WWOTF: You can (and should) think that there’s huge amounts of value at stake in the future, and even think that there’s much more value at stake in the future than there is in the present century, without thinking that value is linear in number of happy people. It diminishes the case a bit, but nowhere near enough for longtermism to not go through.
As you say you can block the obligation to gamble and risk Common-sense Eutopia for something better in different ways/for different reasons.
For me, Common-sense Eutopia sounds pretty appealing because it ensures continuity for existing people. Considering many people don’t have particularly resource-hungry life goals, Common-sense Eutopia would score pretty high on a perspective where it matters what existing people want for the future of themselves and their loved ones.
Even if we say that other considerations besides existing people also matter morally, we may not want those other considerations to just totally swamp/outweigh how good Common-sense Eutopia is from the perspective of existing people.
Re WWOTF: You can (and should) think that there’s huge amounts of value at stake in the future, and even think that there’s much more value at stake in the future than there is in the present century, without thinking that value is linear in number of happy people. It diminishes the case a bit, but nowhere near enough for longtermism to not go through.
Sure you could have a view that it’s great to have 10^12 people, but no more than that, but that seems like a really weird thing to have written in the stars. Or that all that matters is creating the Machine God, so we haven’t attained any value yet. But that doesn’t seem great.
Do you have a gloss on a kind of view that threads the needle nicely without being too crazy, even if it doesn’t ultimately withstand scrutiny?
How much do you think that having lots of mostly or entirely identical future lives is differently valuable than having vastly different positive lives? (Because that would create a reasonable view on which a more limited number of future people can saturate the possible future value.)
Bostrom discusses things like this in Deep Utopia, under the label of ‘interestingness’ (where even if we edit post-humans to never be subjectively bored, maybe they run out of ‘objectively interesting’ things to do and this leads to value not being nearly as high as it could otherwise be). I don’t think he takes a stance on whether or how much interestingness actually matters, but I am only ~half way through the book so far.
This seems almost exactly like the repugnant conclusion. Taken to extremes, intuition disagrees with logic. When that happens, it’s usually the worse for intuition.
I’m not a utilitarian, but I find the repugnant conclusion impossible to reject if you are.
If you want chose what is good for everyone, there’s little argument what that is in those cases.
And if we’re talking about what’s good for everyone, that’s got to be a linear sum of what’s good for each someone. If the sum is nonlinear, who exactly is worth less than the others? This leads to the repugnant conclusion and your conclusion here.
Other definitions of “good for everyone” seem to always mean “what I idiosyncratically prefer for everyone else but me”.
The easiest way, in my view, to make a near-optimal future very likely, conditional on non-extinction, is if value is bounded above.
There’s an argument that this is the common sense view. E.g. consider:
Common-sense Eutopia: In the future, there is a very large population with very high well-being; those people are able to do almost anything they want as long as they don’t harm others. They have complete scientific and technological understanding. War and conflict are things of the past. Environmental destruction has been wholly reversed; Earth is now a natural paradise. However, society is limited only to the solar system, and will come to an end once the Sun has exited its red giant phase, in about five billion years.
Does this seem to capture less than one 10^22th of all possible value? (Because there are ~10^22 affectable stars, so civilisation could be over 10^22 times as big). On my common-sense moral intuitions, no.
Making this argument stronger: Normally, quantities of value are defined are in terms of the value of risky gambles. So what it means to say that Common-sense Eutopia is less than one 10^22th of all possible value is that a gamble with a one in 10^22 chance of producing an ideal-society-across-all-the-stars, and a (1 − 1/10^22) chance of near-term extinction, is better than producing Common-sense Eutopia for certain.
But that seems wild. Of all the issues facing classical utilitarianism, this seems the most problematic to me.
Yes.
So you doubt fanaticism, the view that a tiny chance of an astronomically good outcome can be more valuable than a certainty of a decent outcome. What about in the case of certainty? Do you doubt the utilitarian’s objection to Common-sense Eutopia? This kind of aggregation seems important for the case for longtermism. (See the pages of paper dolls in What We Owe the Future.)
Yeah, I think the issue (for me) is not just about fanaticism. Give me Common-sense Eutopia or a gamble with a 90% chance of extinction and a 10% chance of Common-sense Eutopia 20 times the size, and it seems problematic to choose the gamble.
(To be clear—other views, on which value is diminishing, are also really problematic. We’re in impossibility theorem territory, and I see the whole things as a mess; I don’t have a positive view I’m excited about.)
Re WWOTF: You can (and should) think that there’s huge amounts of value at stake in the future, and even think that there’s much more value at stake in the future than there is in the present century, without thinking that value is linear in number of happy people. It diminishes the case a bit, but nowhere near enough for longtermism to not go through.
As you say you can block the obligation to gamble and risk Common-sense Eutopia for something better in different ways/for different reasons.
For me, Common-sense Eutopia sounds pretty appealing because it ensures continuity for existing people. Considering many people don’t have particularly resource-hungry life goals, Common-sense Eutopia would score pretty high on a perspective where it matters what existing people want for the future of themselves and their loved ones.
Even if we say that other considerations besides existing people also matter morally, we may not want those other considerations to just totally swamp/outweigh how good Common-sense Eutopia is from the perspective of existing people.
Sure you could have a view that it’s great to have 10^12 people, but no more than that, but that seems like a really weird thing to have written in the stars. Or that all that matters is creating the Machine God, so we haven’t attained any value yet. But that doesn’t seem great.
Do you have a gloss on a kind of view that threads the needle nicely without being too crazy, even if it doesn’t ultimately withstand scrutiny?
How much do you think that having lots of mostly or entirely identical future lives is differently valuable than having vastly different positive lives? (Because that would create a reasonable view on which a more limited number of future people can saturate the possible future value.)
Bostrom discusses things like this in Deep Utopia, under the label of ‘interestingness’ (where even if we edit post-humans to never be subjectively bored, maybe they run out of ‘objectively interesting’ things to do and this leads to value not being nearly as high as it could otherwise be). I don’t think he takes a stance on whether or how much interestingness actually matters, but I am only ~half way through the book so far.
This seems almost exactly like the repugnant conclusion. Taken to extremes, intuition disagrees with logic. When that happens, it’s usually the worse for intuition.
I’m not a utilitarian, but I find the repugnant conclusion impossible to reject if you are.
If you want chose what is good for everyone, there’s little argument what that is in those cases.
And if we’re talking about what’s good for everyone, that’s got to be a linear sum of what’s good for each someone. If the sum is nonlinear, who exactly is worth less than the others? This leads to the repugnant conclusion and your conclusion here.
Other definitions of “good for everyone” seem to always mean “what I idiosyncratically prefer for everyone else but me”.