I think it makes sense to be a bit confused about what claim I’m making and why. I read the paper and made the initial version of these note a few weeks ago, so my memory of what the paper said and how it changed my views is slightly hazy.
But I think the key point is essentially the arguably obvious point that the rate of ENEs can be really important, and that that rate seems likely to be much higher when the target state is something like “a very good system of government or set of values” or “a very bad system of government or set of values” (compared to when the target state is whether an intelligent civilization exists). It does seem much more obvious that extinction or non-extinction are each stronger attractor states that particularly good or particularly bad non-extinction outcomes are.
This is basically something I already knew, but I think Tarsney’s models and analysis made the point a bit more salient, and also made it clearer how important it is (since the rate of ENEs seems like probably one of the most important factors influencing the case for longtermism).
But what I’ve said above kind-of implicitly accepts Tarsney’s focus (for the sake of his working example) on simply whether there is an intelligent civilization around, rather than what it’s doing. In reality, I think that what the civilization is doing is likely also very important.[1] So the above point about particularly good or particularly bad non-extinction outcomes maybe being only weak attractor states might also undermine the significance of keeping an intelligent civilization around.
But here’s one way that might not be true: Maybe we think it’s easier to have a lock-in of—or natural trends that maintain—a good non-extinction outcome than a bad non-extinction outcome. (I think Ord essentially implies this in The Precipice. I might soon post something related to this. It’s also been discussed in some other places, e.g. here.) If so, then the point about the rate of ENEs suggests the case for avoiding unrecoverable dystopias and unrecoverable collapses might be weak, but it wouldn’t as strongly suggest the case for avoiding extinction is weak.
...but this all seems rather complicated, and I’m still not sure my thinking is clear, and even less sure my explanation is clear!
[1] Tarsney does acknowledge roughly this point later in the paper:
Additionally, there are other potential sources of epistemic resistance to longtermism besides Weak Attractors that this paper has not addressed. In particular, these include:
Neutral Attractors To entertain small values of r [the rate of ENEs], we must assume that the state S targeted by a longtermist intervention, and its complement ¬S, are both at least to some extent “attractor” states: Once a system is in state S, or state ¬S, it is unlikely to leave that state any time soon. But to justify significant values of ve and vs, it must also be the case that the attractors we are able to target differ significantly in expected value. And it’s not clear that we can assume this. For instance, perhaps “large interstellar civilization exists in spatial region X” is an attractor state, but “large interstellar civilization exists in region X with healthy norms and institutions that generate a high level of value” is not. If civilizations tend to “wander” unpredictably between high-value and low-value states, it could be that despite their astronomical potential for value, the expected value of large interstellar civilizations is close to zero. In that case, we can have persistent effects on the far future, but not effects that matter (in expectation).
[Writing this comment quickly]
I think it makes sense to be a bit confused about what claim I’m making and why. I read the paper and made the initial version of these note a few weeks ago, so my memory of what the paper said and how it changed my views is slightly hazy.
But I think the key point is essentially the arguably obvious point that the rate of ENEs can be really important, and that that rate seems likely to be much higher when the target state is something like “a very good system of government or set of values” or “a very bad system of government or set of values” (compared to when the target state is whether an intelligent civilization exists). It does seem much more obvious that extinction or non-extinction are each stronger attractor states that particularly good or particularly bad non-extinction outcomes are.
This is basically something I already knew, but I think Tarsney’s models and analysis made the point a bit more salient, and also made it clearer how important it is (since the rate of ENEs seems like probably one of the most important factors influencing the case for longtermism).
But what I’ve said above kind-of implicitly accepts Tarsney’s focus (for the sake of his working example) on simply whether there is an intelligent civilization around, rather than what it’s doing. In reality, I think that what the civilization is doing is likely also very important.[1] So the above point about particularly good or particularly bad non-extinction outcomes maybe being only weak attractor states might also undermine the significance of keeping an intelligent civilization around.
But here’s one way that might not be true: Maybe we think it’s easier to have a lock-in of—or natural trends that maintain—a good non-extinction outcome than a bad non-extinction outcome. (I think Ord essentially implies this in The Precipice. I might soon post something related to this. It’s also been discussed in some other places, e.g. here.) If so, then the point about the rate of ENEs suggests the case for avoiding unrecoverable dystopias and unrecoverable collapses might be weak, but it wouldn’t as strongly suggest the case for avoiding extinction is weak.
...but this all seems rather complicated, and I’m still not sure my thinking is clear, and even less sure my explanation is clear!
[1] Tarsney does acknowledge roughly this point later in the paper:
OK thanks I think that is clearer now.