Hi Ajeya, thanks for doing this and for your recent 80K interview! I’m trying to understand what assumptions are needed for the argument you raise in the podcast discussion on fairness agreements that a longtermist worldview should have been willing to trade up all its influence for ever-larger potential universe. There are two points I was wondering if you could comment on if/how these align with your argument.
My intuition says that the argument requires a prior probability distribution on universe size that has an infinite expectation, rather than just a prior with non-zero probability on all possible universe sizes with a finite expectation (like a power-law distribution with k > 2).
But then I figured that even in a universe that was literally infinite but had a non-zero density of value-maximizing civilizations, the amount of influence over that infinite value that any one civilization or organization has might still be finite. So I’m wondering if what is needed to be willing to trade up for influence over ever larger universes is actually something like the expectation E[V/n] being infinite, where V = total potential value in universe and n = number of value-maximizing civilizations.
I agree that your prior would need to have an infinite expectation for the size of the universe for this argument to go through.
I agree with the generalized statement that your prior over “value-I-can-affect” needs to have an infinite expectation, but I don’t think I agree with the operationalization of “value-I-can-affect” as V/n. It seems possible to me that even if there are a high density of value-maximizing civilizations out there, each one could have an infinite impact through e.g. acausal trade. I’m not sure what a crisp operationalization of “value-I-can-affect” would be.
Hi Ajeya, thanks for doing this and for your recent 80K interview! I’m trying to understand what assumptions are needed for the argument you raise in the podcast discussion on fairness agreements that a longtermist worldview should have been willing to trade up all its influence for ever-larger potential universe. There are two points I was wondering if you could comment on if/how these align with your argument.
My intuition says that the argument requires a prior probability distribution on universe size that has an infinite expectation, rather than just a prior with non-zero probability on all possible universe sizes with a finite expectation (like a power-law distribution with k > 2).
But then I figured that even in a universe that was literally infinite but had a non-zero density of value-maximizing civilizations, the amount of influence over that infinite value that any one civilization or organization has might still be finite. So I’m wondering if what is needed to be willing to trade up for influence over ever larger universes is actually something like the expectation E[V/n] being infinite, where V = total potential value in universe and n = number of value-maximizing civilizations.
I agree that your prior would need to have an infinite expectation for the size of the universe for this argument to go through.
I agree with the generalized statement that your prior over “value-I-can-affect” needs to have an infinite expectation, but I don’t think I agree with the operationalization of “value-I-can-affect” as V/n. It seems possible to me that even if there are a high density of value-maximizing civilizations out there, each one could have an infinite impact through e.g. acausal trade. I’m not sure what a crisp operationalization of “value-I-can-affect” would be.
I see, thank you!