Thanks! I haven’t read your stuff yet, but it seems like good work; and this has been a reason in my mind for being more in favour of trajectory change than totla extinction reduction for a while. It would only reduce the value of extinction risk reduction by an OOM at most, though?
I’m sympathetic to something in Mediocrity direction (for AI-built civilisations as well as human-built civilisations), but I think it’s very hard to have a full-blooded Mediocrity principle if you also think that you can take actions today to meaningfully increase or decrease the value of Earth-originating civilisation. Suppose that Earth-originating civilisation’s value is V, and if we all worked on it we could increase that to V+ or to V-. If so, then which is the right value for the alien civilisation? Choosing V rather than V+ or V- (or V+++ or V—etc) seems pretty arbitrary.
Rather, we should think about how good our prospects are compared to a random draw civilisation. You might think we’re doing better or worse, but if it’s possible for us to move the value of the future around, then it seems we should be able to reasonably think that we’re quite a bit better (or worse) than the random draw civ.
It would only reduce the value of extinction risk reduction by an OOM at most, though?
Right, at most, one OOM. Higher updates would require us to learn that the universe is more Civ-Saturated than our current best guess. This could be the case if: - humanity’s extinction would not prevent another intelligent civilization from appearing quickly on Earth— OR that intelligent life in the universe is much more frequent (e.g., to learn that intelligent life can appear around red dwarfs whose lifespan is 100B to 1T years).
Suppose that Earth-originating civilisation’s value is V, and if we all worked on it we could increase that to V+ or to V-. If so, then which is the right value for the alien civilisation? Choosing V rather than V+ or V- (or V+++ or V—etc) seems pretty arbitrary.
I guess, as long as V ~ V+++ ~ V--- (like the relative difference is less than 1%), then it is likely not a big issue. However, the relative difference may become large only when we become significantly more certain about the impact of our actions, e.g., if we are the operators choosing the moral values of the first ASI.
Thanks! I haven’t read your stuff yet, but it seems like good work; and this has been a reason in my mind for being more in favour of trajectory change than totla extinction reduction for a while. It would only reduce the value of extinction risk reduction by an OOM at most, though?
I’m sympathetic to something in Mediocrity direction (for AI-built civilisations as well as human-built civilisations), but I think it’s very hard to have a full-blooded Mediocrity principle if you also think that you can take actions today to meaningfully increase or decrease the value of Earth-originating civilisation. Suppose that Earth-originating civilisation’s value is V, and if we all worked on it we could increase that to V+ or to V-. If so, then which is the right value for the alien civilisation? Choosing V rather than V+ or V- (or V+++ or V—etc) seems pretty arbitrary.
Rather, we should think about how good our prospects are compared to a random draw civilisation. You might think we’re doing better or worse, but if it’s possible for us to move the value of the future around, then it seems we should be able to reasonably think that we’re quite a bit better (or worse) than the random draw civ.
Right, at most, one OOM. Higher updates would require us to learn that the universe is more Civ-Saturated than our current best guess. This could be the case if:
- humanity’s extinction would not prevent another intelligent civilization from appearing quickly on Earth—
OR that intelligent life in the universe is much more frequent (e.g., to learn that intelligent life can appear around red dwarfs whose lifespan is 100B to 1T years).
I guess, as long as V ~ V+++ ~ V--- (like the relative difference is less than 1%), then it is likely not a big issue. However, the relative difference may become large only when we become significantly more certain about the impact of our actions, e.g., if we are the operators choosing the moral values of the first ASI.