It is hard for pure advancements to compete with reducing existential risk as their value turns out not to scale with the duration of humanity’s future.
Relatedly, you say that (emphasis mine):
In that case [“if humanity’s instantaneous value spends a long time at a high plateau”], advancements are better roughly when δt/τ > (γp – 1) — when the percentage of the duration of humanity’s future that we advance is greater than the percentage by which our survival probability is increased. This makes it difficult for advancements to beat existential risk reduction in scenarios where humanity would have a long lifespan if only it could survive the near-term risks. For example, on a million-year lifespan (that of a typical species) a one-year advancement would be roughly as important as a one-in-a-million improvement in nearterm survival probability — but the latter seems much more achievable.
Is the intuition that such advancement could be achieved by doubling the value growth rate for 1 year, which is much harder than decreasing existential risk by 0.0001 %?
The idea is that advancing overall progress by a year means getting a year ahead on social progress, political progress, moral progress, scientific progress, and of course, technological progress. Given that our progress in these is the result of so many people’s work, it seems very hard to me for a small group to lead to changes that improve that by a whole year (even over one’s lifetime). Whereas a small group leading to changes that lead to the neglected area of existential risk reduction of 0.0001% seems a lot more plausible to me — in fact, I’d guess we’ve already achieved that.
Great article, Toby!
Relatedly, you say that (emphasis mine):
Is the intuition that such advancement could be achieved by doubling the value growth rate for 1 year, which is much harder than decreasing existential risk by 0.0001 %?
Thanks!
The idea is that advancing overall progress by a year means getting a year ahead on social progress, political progress, moral progress, scientific progress, and of course, technological progress. Given that our progress in these is the result of so many people’s work, it seems very hard to me for a small group to lead to changes that improve that by a whole year (even over one’s lifetime). Whereas a small group leading to changes that lead to the neglected area of existential risk reduction of 0.0001% seems a lot more plausible to me — in fact, I’d guess we’ve already achieved that.