I agree with quite a lot of this, however I think one of the most important points for whether or not we get something close to a near-best future is whether or not we have some kind of relatively comprehensive process for determining what humanity does with our future; what I’ve been calling “deep reflection,” which could be something like a long reflection or coherent extrapolated volition.
I think if such a process is used to determine how to use at least some percentage of future resources, then at least that percentage could be close to optimal value; if for some reason there is not a comprehensive process that determines any significant percentage of future resources, then I am quite pessimistic that we would get very far above 0% possible value, as getting high value seems quite difficult because it seems like there are many factors or crucial considerations (e.g. here and here) which could decrease future value, and like getting even a few of these factors wrong may mean we get close to zero value.
I am also quite sympathetic to extremity of the best. My intuition is that it is very likely that optimal use of resources is far, far better than most other likely uses. Would be happy to expand more on this if there is interest.
That said, I think it’s important to recognize the importance of such a comprehensive process as an essential unifying strategic element; conditional on avoiding any existential catastrophes, it seems to me the vast majority of future value[1] will be determined by two factors:
Whether or not we get a comprehensive reflection process at all
What percentage of future resources are used optimally according to the outcome of such a process
I think all of the other factors you mentioned are quite important, but I believe that they gain the vast majority of their value through how they influence the above two factors. Again, this is simply because (i) I believe we need such a process to determine how to get much above zero value with any degree of confidence, and (ii) it seems quite possible there will not be widespread agreement that we should use resources optimally according to the outcome of this process, but hopefully some grand bargain/existential compromise can be reached, or more optimistically perhaps we could try to get on a path where more convergence happens.
Would be curious to hear if you disagree with the previous paragraph, it feels like a pretty big crux for me.
Rather than “the vast majority of future value,” perhaps it would be slightly more accurate to say “the vast majority of positive impact we will have in expectation”; perhaps we are already predestined to achieve very high value or very low value, or more importantly, perhaps there are things we could do that seem like they will have a large impact (positive or negative); but unless our actions are leading to comprehensive strategic clarity (comprehensive reflection), and the likelihood of acting on this clarity (deep reflection), if seems hard to be confident ex ante our actions are highly likely to have highly positive impact in the final analysis.
Thanks for putting this out!
I agree with quite a lot of this, however I think one of the most important points for whether or not we get something close to a near-best future is whether or not we have some kind of relatively comprehensive process for determining what humanity does with our future; what I’ve been calling “deep reflection,” which could be something like a long reflection or coherent extrapolated volition.
I think if such a process is used to determine how to use at least some percentage of future resources, then at least that percentage could be close to optimal value; if for some reason there is not a comprehensive process that determines any significant percentage of future resources, then I am quite pessimistic that we would get very far above 0% possible value, as getting high value seems quite difficult because it seems like there are many factors or crucial considerations (e.g. here and here) which could decrease future value, and like getting even a few of these factors wrong may mean we get close to zero value.
I am also quite sympathetic to extremity of the best. My intuition is that it is very likely that optimal use of resources is far, far better than most other likely uses. Would be happy to expand more on this if there is interest.
That said, I think it’s important to recognize the importance of such a comprehensive process as an essential unifying strategic element; conditional on avoiding any existential catastrophes, it seems to me the vast majority of future value[1] will be determined by two factors:
Whether or not we get a comprehensive reflection process at all
What percentage of future resources are used optimally according to the outcome of such a process
I think all of the other factors you mentioned are quite important, but I believe that they gain the vast majority of their value through how they influence the above two factors. Again, this is simply because (i) I believe we need such a process to determine how to get much above zero value with any degree of confidence, and (ii) it seems quite possible there will not be widespread agreement that we should use resources optimally according to the outcome of this process, but hopefully some grand bargain/existential compromise can be reached, or more optimistically perhaps we could try to get on a path where more convergence happens.
Would be curious to hear if you disagree with the previous paragraph, it feels like a pretty big crux for me.
Rather than “the vast majority of future value,” perhaps it would be slightly more accurate to say “the vast majority of positive impact we will have in expectation”; perhaps we are already predestined to achieve very high value or very low value, or more importantly, perhaps there are things we could do that seem like they will have a large impact (positive or negative); but unless our actions are leading to comprehensive strategic clarity (comprehensive reflection), and the likelihood of acting on this clarity (deep reflection), if seems hard to be confident ex ante our actions are highly likely to have highly positive impact in the final analysis.