Executive summary: The author argues that Thorstad’s critique of longtermist “moral mathematics” reduces expected value by only a few orders of magnitude, which is far too small to undermine the case for existential risk reduction, especially given non-trivial chances of extremely large or even unbounded future value.
Key points:
Thorstad claims longtermist models ignore cumulative and background extinction risk, which would sharply reduce the probability of humanity surviving long enough to realize vast future value.
The author responds that we should assign non-trivial credence to reaching a state where extinction risk is near zero, and even a low probability of such stabilization leaves existential risk reduction with extremely large expected value.
The author argues that even if long-run extinction is unavoidable, advanced technology could enable enormous short-term value creation, so longtermist considerations still dominate.
Thorstad claims population models overestimate the likelihood that humanity will maximize future population size, but the author argues that even small probabilities of such futures only reduce expected value by a few orders of magnitude.
The author states that 10^52 possible future people is an underestimate because some scenarios allow astronomically larger or even infinite numbers of happy minds, raising expected value far beyond Thorstad’s assumptions.
The author concludes that Thorstad’s adjustments lower expected value only modestly and cannot overturn the core longtermist argument for prioritizing existential risk reduction.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, andcontact us if you have feedback.
Executive summary: The author argues that Thorstad’s critique of longtermist “moral mathematics” reduces expected value by only a few orders of magnitude, which is far too small to undermine the case for existential risk reduction, especially given non-trivial chances of extremely large or even unbounded future value.
Key points:
Thorstad claims longtermist models ignore cumulative and background extinction risk, which would sharply reduce the probability of humanity surviving long enough to realize vast future value.
The author responds that we should assign non-trivial credence to reaching a state where extinction risk is near zero, and even a low probability of such stabilization leaves existential risk reduction with extremely large expected value.
The author argues that even if long-run extinction is unavoidable, advanced technology could enable enormous short-term value creation, so longtermist considerations still dominate.
Thorstad claims population models overestimate the likelihood that humanity will maximize future population size, but the author argues that even small probabilities of such futures only reduce expected value by a few orders of magnitude.
The author states that 10^52 possible future people is an underestimate because some scenarios allow astronomically larger or even infinite numbers of happy minds, raising expected value far beyond Thorstad’s assumptions.
The author concludes that Thorstad’s adjustments lower expected value only modestly and cannot overturn the core longtermist argument for prioritizing existential risk reduction.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.