Surely ālock-inā implies stability and persistence?
Yeah, definitely. I see now that I didnāt clearly explain what I meant. Itās not that I changed my views on how how important the difference between lock-in of a bad governmental system or set of values and a future without such a lock-in is.
Itās more like I somewhat updated my views regarding:
how likely such a lock-in is
and in particular how likely it is that a state that looks like it might be a lock-in would actually be a lock-in
and in particular how much the epistemic challenge to longtermism might undermine a focus on this type of potential lock-in in particular
And as a result, I somewhat updated me views regarding how much we should focus on preventing these outcomes. Analogous to how Iād update my prioritisation of biorisk if I learned the relevant catastrophes were less likely than I thought, even if no less bad.
(Iām still not sure that explanation is 100% clear.)
And yeah, Greaves and MacAskillās ānon-extinction attractor stateā concept is relevant here, and I liked that section of their paper :)
OK thatās clearer, although Iām not immediately sure why the paper would have achieved the following:
I somewhat updated my views regarding:
how likely such a lock-in is
and in particular how likely it is that a state that looks like it might be a lock-in would actually be a lock-in
...
I think Tarsney implies that institutional reform is less likely to be a true lock-in, but he doesnāt really back this up with much argument. He just implies that this point is somewhat obvious. Under this assumption, I can understand why his model would lead to the following update:
...
...
and in particular how much the epistemic challenge to longtermism might undermine a focus on this type of potential lock-in in particular
In other words, if Tarsney had engaged in a discussion about why institutional change isnāt actually likely to be stable/āpersistent, providing object-level reasons for why (which may involve disagreeing with Greaves and MacAskillās points), I think I too would update away from thinking institutional change is that important, but I donāt think he really engages in this discussion.
I should say that I havenāt properly read through the whole paper (I have mainly relied on watching the video and skimming through the paper), so itās possible Iām missing some things.
I think it makes sense to be a bit confused about what claim Iām making and why. I read the paper and made the initial version of these note a few weeks ago, so my memory of what the paper said and how it changed my views is slightly hazy.
But I think the key point is essentially the arguably obvious point that the rate of ENEs can be really important, and that that rate seems likely to be much higher when the target state is something like āa very good system of government or set of valuesā or āa very bad system of government or set of valuesā (compared to when the target state is whether an intelligent civilization exists). It does seem much more obvious that extinction or non-extinction are each stronger attractor states that particularly good or particularly bad non-extinction outcomes are.
This is basically something I already knew, but I think Tarsneyās models and analysis made the point a bit more salient, and also made it clearer how important it is (since the rate of ENEs seems like probably one of the most important factors influencing the case for longtermism).
But what Iāve said above kind-of implicitly accepts Tarsneyās focus (for the sake of his working example) on simply whether there is an intelligent civilization around, rather than what itās doing. In reality, I think that what the civilization is doing is likely also very important.[1] So the above point about particularly good or particularly bad non-extinction outcomes maybe being only weak attractor states might also undermine the significance of keeping an intelligent civilization around.
But hereās one way that might not be true: Maybe we think itās easier to have a lock-in ofāor natural trends that maintaināa good non-extinction outcome than a bad non-extinction outcome. (I think Ord essentially implies this in The Precipice. I might soon post something related to this. Itās also been discussed in some other places, e.g. here.) If so, then the point about the rate of ENEs suggests the case for avoiding unrecoverable dystopias and unrecoverable collapses might be weak, but it wouldnāt as strongly suggest the case for avoiding extinction is weak.
...but this all seems rather complicated, and Iām still not sure my thinking is clear, and even less sure my explanation is clear!
[1] Tarsney does acknowledge roughly this point later in the paper:
Additionally, there are other potential sources of epistemic resistance to longtermism besides Weak Attractors that this paper has not addressed. In particular, these include:
Neutral Attractors To entertain small values of r [the rate of ENEs], we must assume that the state S targeted by a longtermist intervention, and its complement Ā¬S, are both at least to some extent āattractorā states: Once a system is in state S, or state Ā¬S, it is unlikely to leave that state any time soon. But to justify significant values of ve and vs, it must also be the case that the attractors we are able to target differ significantly in expected value. And itās not clear that we can assume this. For instance, perhaps ālarge interstellar civilization exists in spatial region Xā is an attractor state, but ālarge interstellar civilization exists in region X with healthy norms and institutions that generate a high level of valueā is not. If civilizations tend to āwanderā unpredictably between high-value and low-value states, it could be that despite their astronomical potential for value, the expected value of large interstellar civilizations is close to zero. In that case, we can have persistent effects on the far future, but not effects that matter (in expectation).
Yeah, definitely. I see now that I didnāt clearly explain what I meant. Itās not that I changed my views on how how important the difference between lock-in of a bad governmental system or set of values and a future without such a lock-in is.
Itās more like I somewhat updated my views regarding:
how likely such a lock-in is
and in particular how likely it is that a state that looks like it might be a lock-in would actually be a lock-in
and in particular how much the epistemic challenge to longtermism might undermine a focus on this type of potential lock-in in particular
And as a result, I somewhat updated me views regarding how much we should focus on preventing these outcomes. Analogous to how Iād update my prioritisation of biorisk if I learned the relevant catastrophes were less likely than I thought, even if no less bad.
(Iām still not sure that explanation is 100% clear.)
And yeah, Greaves and MacAskillās ānon-extinction attractor stateā concept is relevant here, and I liked that section of their paper :)
OK thatās clearer, although Iām not immediately sure why the paper would have achieved the following:
I think Tarsney implies that institutional reform is less likely to be a true lock-in, but he doesnāt really back this up with much argument. He just implies that this point is somewhat obvious. Under this assumption, I can understand why his model would lead to the following update:
In other words, if Tarsney had engaged in a discussion about why institutional change isnāt actually likely to be stable/āpersistent, providing object-level reasons for why (which may involve disagreeing with Greaves and MacAskillās points), I think I too would update away from thinking institutional change is that important, but I donāt think he really engages in this discussion.
I should say that I havenāt properly read through the whole paper (I have mainly relied on watching the video and skimming through the paper), so itās possible Iām missing some things.
[Writing this comment quickly]
I think it makes sense to be a bit confused about what claim Iām making and why. I read the paper and made the initial version of these note a few weeks ago, so my memory of what the paper said and how it changed my views is slightly hazy.
But I think the key point is essentially the arguably obvious point that the rate of ENEs can be really important, and that that rate seems likely to be much higher when the target state is something like āa very good system of government or set of valuesā or āa very bad system of government or set of valuesā (compared to when the target state is whether an intelligent civilization exists). It does seem much more obvious that extinction or non-extinction are each stronger attractor states that particularly good or particularly bad non-extinction outcomes are.
This is basically something I already knew, but I think Tarsneyās models and analysis made the point a bit more salient, and also made it clearer how important it is (since the rate of ENEs seems like probably one of the most important factors influencing the case for longtermism).
But what Iāve said above kind-of implicitly accepts Tarsneyās focus (for the sake of his working example) on simply whether there is an intelligent civilization around, rather than what itās doing. In reality, I think that what the civilization is doing is likely also very important.[1] So the above point about particularly good or particularly bad non-extinction outcomes maybe being only weak attractor states might also undermine the significance of keeping an intelligent civilization around.
But hereās one way that might not be true: Maybe we think itās easier to have a lock-in ofāor natural trends that maintaināa good non-extinction outcome than a bad non-extinction outcome. (I think Ord essentially implies this in The Precipice. I might soon post something related to this. Itās also been discussed in some other places, e.g. here.) If so, then the point about the rate of ENEs suggests the case for avoiding unrecoverable dystopias and unrecoverable collapses might be weak, but it wouldnāt as strongly suggest the case for avoiding extinction is weak.
...but this all seems rather complicated, and Iām still not sure my thinking is clear, and even less sure my explanation is clear!
[1] Tarsney does acknowledge roughly this point later in the paper:
OK thanks I think that is clearer now.