In this framework, the value of our future is equal to the area under this curve and the value of altering our trajectory is equal to the area between the original curve and the altered curve.
You mentioned optimal planning in economics, and I’ve wondered whether an optimal control framework might be useful for this sort of analysis. I think the difference between optimal control and the trajectory-altering framework you describe is a bit deeper than the different typical domains. There’s not just one decision to be made, but a nearly-continuous series of decisions extending through the future (a “policy”). Under uncertainty, the present expected value is the presently realized (“instantaneous”) value plus the expectation taken over different futures of the expected value of those futures. Choosing a policy to maximize expected realized value is the control problem.
For the most part, you get the same results with slightly different interpretation. For example, rather than impose some τ, you get an effective lifetime that’s mainly determined by “background” risk but also is allowed to vary based on policy.
One thing that jumped out at me in a toy model is that while the value of reducing existential risk is mathematically the same as an “enhancement” (multiplicative), the time at which we expect to realize that extra value can be very different. In particular, an expected value maximizer may heavily backload the realization of value (even beyond the present expected survival time) if they can neglect present value to expand while reducing existential risk.
I suspect one could learn or make clearer a few other interesting things by following those lines.
I think that is indeed a good analogy. In the chapter, I focus on questions of marginal changes to the curve where there is a small shock to some parameter right now, and then its effects play out. If one is instead asking questions that are less about what we now could do to change the future, but are about how humanity over deep time should act to have a good future, then I think the optimal control techniques could be very useful. And I wouldn’t be surprised if in attempting to understand the dynamics, there were lessons for our present actions too.
Very clear piece!
You mentioned optimal planning in economics, and I’ve wondered whether an optimal control framework might be useful for this sort of analysis. I think the difference between optimal control and the trajectory-altering framework you describe is a bit deeper than the different typical domains. There’s not just one decision to be made, but a nearly-continuous series of decisions extending through the future (a “policy”). Under uncertainty, the present expected value is the presently realized (“instantaneous”) value plus the expectation taken over different futures of the expected value of those futures. Choosing a policy to maximize expected realized value is the control problem.
For the most part, you get the same results with slightly different interpretation. For example, rather than impose some τ, you get an effective lifetime that’s mainly determined by “background” risk but also is allowed to vary based on policy.
One thing that jumped out at me in a toy model is that while the value of reducing existential risk is mathematically the same as an “enhancement” (multiplicative), the time at which we expect to realize that extra value can be very different. In particular, an expected value maximizer may heavily backload the realization of value (even beyond the present expected survival time) if they can neglect present value to expand while reducing existential risk.
I suspect one could learn or make clearer a few other interesting things by following those lines.
I think that is indeed a good analogy. In the chapter, I focus on questions of marginal changes to the curve where there is a small shock to some parameter right now, and then its effects play out. If one is instead asking questions that are less about what we now could do to change the future, but are about how humanity over deep time should act to have a good future, then I think the optimal control techniques could be very useful. And I wouldn’t be surprised if in attempting to understand the dynamics, there were lessons for our present actions too.