Have you come across Toby Ord’s Shaping Humanity’s Longterm Trajectory? I think it might expand on the ways you might do longtermist “accounting” to figure out what matters and compare different kinds of intervention. Saving future lives probably sits under the “Gains” section.
I wasn’t aware of that essay. I think it’s ambitious in exactly the right direction: finding a way to analyze value objectively and quantitatively.
That being said, I think it’s flawed as a physical model of value.
Because it’s a complex topic, I’ll just mention a few quick points (we can expand on them later): 1. The future is not a thing that already exists and merely waits to be predicted. It is an unfolding computation. Because the systems generating it are chaotic, the only way to know the exact future is to run the process itself, step by step. At that point, prediction becomes identical to running through it in reality. 2. So you can’t shift the curve to the left. You can’t just bring an identical version of the future closer. Because by doing that, you’re changing it. In chaotic systems, tiny changes compound into wildly different trajectories. 3. The whole “area under the curve” is problematic. Think about a piece of knowledge that existed in the past and that was lost. That knowledge was valuable when it was saved on at least one substrate. Now that value doesn’t exist. Same with the future. Again… the future doesn’t exist. There is no value under the curve past this moment. 4. I believe there is a much cleaner way to measure objective value. You measure the present moment’s Functional Information (FI), in bits. Another way to think of it is as the structured information that exists now (it’s hard to understand this without the context). The future is a branching consequence of the present structure. 5. Present FI persists by reducing hazard. Reducing hazard over longer horizons requires better models, better coordination, better tools, and better memory. Those successful structures accumulate as new FI.
I’d love to hear some of the counter-arguments against what Paul has wrote here. I have started reading up on long termism and I am having problems with the idea of “We cannot count future people in one scenario while treating people saved today as if their influence ends at the moment of rescue.” that is mentioned as well as the idea that future people need to be repeatably saved to exist. If you avert a existential extinction level disaster in 2030 that allows future people in 2100 to live and flourish, but a second disaster (of the same or a similar type) needs to be averted in 2050, how do you avoid double counting that life saved? I’m sure this problem has been explored but I’d like to read the counter-arguments.
If you avert a existential extinction level disaster in 2030 that allows future people in 2100 to live and flourish, but a second disaster (of the same or a similar type) needs to be averted in 2050, how do you avoid double counting that life saved?
The short answer is that you discount the value of the future you save based on the expected number of lives in it.
I.e. if, in a simple case, you know the Sun will turn into a red giant and kill us all in 2100, then averting extinction this year would says 10s of billions of lives, but no more. This is more complicated when we have X% chance of going extinct by 2100, Y% by 2200 etc…
I’m less sure what to say about Paul’s point that saving a life today = many more lives that exist in the future. I’d guess that demographic projections lower the impact of this on the calculation (i.e. we expect a lower population anyway), but I’m not sure. A more general response is that basically everything except preventing extinction washes out in the long-term, so increasing population over the next 100 years would be no exception.
Hey guys, if you want, give me some hypothetical scenarios (A vs B), and I can work through them from the POV of my framework. Stress test it and show its utility at the same time.
I think longtermism has good intentions, but there is a deep flaw in its accounting.
This is my argument: https://paulcomans.substack.com/p/longtermism-is-counting-wrong
Have you come across Toby Ord’s Shaping Humanity’s Longterm Trajectory? I think it might expand on the ways you might do longtermist “accounting” to figure out what matters and compare different kinds of intervention. Saving future lives probably sits under the “Gains” section.
I wasn’t aware of that essay. I think it’s ambitious in exactly the right direction: finding a way to analyze value objectively and quantitatively.
That being said, I think it’s flawed as a physical model of value.
Because it’s a complex topic, I’ll just mention a few quick points (we can expand on them later):
1. The future is not a thing that already exists and merely waits to be predicted. It is an unfolding computation. Because the systems generating it are chaotic, the only way to know the exact future is to run the process itself, step by step. At that point, prediction becomes identical to running through it in reality.
2. So you can’t shift the curve to the left. You can’t just bring an identical version of the future closer. Because by doing that, you’re changing it. In chaotic systems, tiny changes compound into wildly different trajectories.
3. The whole “area under the curve” is problematic. Think about a piece of knowledge that existed in the past and that was lost. That knowledge was valuable when it was saved on at least one substrate. Now that value doesn’t exist. Same with the future. Again… the future doesn’t exist. There is no value under the curve past this moment.
4. I believe there is a much cleaner way to measure objective value. You measure the present moment’s Functional Information (FI), in bits. Another way to think of it is as the structured information that exists now (it’s hard to understand this without the context). The future is a branching consequence of the present structure.
5. Present FI persists by reducing hazard. Reducing hazard over longer horizons requires better models, better coordination, better tools, and better memory. Those successful structures accumulate as new FI.
Open to discussion. I know it’s a lot :D
I’d love to hear some of the counter-arguments against what Paul has wrote here. I have started reading up on long termism and I am having problems with the idea of “We cannot count future people in one scenario while treating people saved today as if their influence ends at the moment of rescue.” that is mentioned as well as the idea that future people need to be repeatably saved to exist. If you avert a existential extinction level disaster in 2030 that allows future people in 2100 to live and flourish, but a second disaster (of the same or a similar type) needs to be averted in 2050, how do you avoid double counting that life saved? I’m sure this problem has been explored but I’d like to read the counter-arguments.
The short answer is that you discount the value of the future you save based on the expected number of lives in it.
I.e. if, in a simple case, you know the Sun will turn into a red giant and kill us all in 2100, then averting extinction this year would says 10s of billions of lives, but no more. This is more complicated when we have X% chance of going extinct by 2100, Y% by 2200 etc…
I’m less sure what to say about Paul’s point that saving a life today = many more lives that exist in the future. I’d guess that demographic projections lower the impact of this on the calculation (i.e. we expect a lower population anyway), but I’m not sure. A more general response is that basically everything except preventing extinction washes out in the long-term, so increasing population over the next 100 years would be no exception.
Hey guys, if you want, give me some hypothetical scenarios (A vs B), and I can work through them from the POV of my framework. Stress test it and show its utility at the same time.