I think I agree with the core claims Buck is making. But I found the logical structure of this post hard to follow. So here’s my attempt to re-present the core thread of the argument I think Buck is making:
In his original post, Will conditions on long futures being plausible, since these are the worlds that longtermists care about most. Let’s assume from now on that this is the case. Will claims, based on his uniform prior over hinginess, that we should require extraordinary evidence to believe in our century’s hinginess, conditional on long futures being plausible. But there are at least two reasons to think that we shouldn’t use a uniform prior. Firstly, it’s more reasonable to instead have a prior that early times in human history (such as our time) are more likely to be hingey—for example because we should expect humanity to expand over time, and also from considering technological advances.
Secondly: if we condition on long futures being plausible, then xrisk must be near-zero in almost every century (otherwise there’s almost no chance we’d survive for that long). So observing any nonnegligible amount of (preventable) xrisk in our present time becomes very strong evidence of this century being an extreme outlier in terms of xrisk, which implies that it’s also an extreme outlier in terms of hinginess. So using the uniform prior on hinginess means we have to choose between two very implausible options—either current xrisk is in fact incredibly low (far lower than seems plausible, and far lower than Will himself claims to believe it is), or else we’re in a situation that the uniform prior judges as extremely improbable and “fishy”. Instead of biting either of these bullets, it seems preferable to use a prior which isn’t so dogmatic—e.g. a prior which isn’t so surprised by early times in human history being outliers.
Toby gives an example of an alternative (and in my mind better) prior as a reply to Will’s original post.
Note that anyone who conditions on a long future being possible should afterwards doubt the evidence for current xrisk to some degree. But Will is forced to do so to an extreme extent because his uniform prior on hinginess is such a bold one—whereas people with exponentially diminishing priors on hinginess like Toby’s won’t update as much after conditioning. All of this analysis remains roughly the same if you replace Will’s uniformity-of-hinginess prior with a uniformity-of-(preventable)-xrisk prior, and Toby’s exponentially-decreasing-hinginess prior with an exponentially-decreasing-(preventable)-xrisk prior. I add “preventable” here and above because if our current xrisk isn’t preventable, then it’s still possible that we’re in a low-hinginess period.
Lastly, it seems to me that conditioning on long futures being plausible was what caused a lot of the messiness here, and so for pedagogical purposes it’d probably be better to spell out all the different permutations of options more explicitly, and be clearer about when the conditioning is happening.
Firstly, it’s more reasonable to instead have a prior that early times in human history (such as our time) are more likely to be hingey—for example because we should expect humanity to expand over time, and also from considering technological advances.
I don’t really get this. It seems that there are good reasons to believe early times are influential, but also good reasons to believe that later times are influential, and it isn’t clear to me which of these dominates.
For example, Will’s inductive argument against HH pulls towards later times being influential. From Will’s paper:
P1. The influentialness of comparable people in the past has been increasing over time, with increasing knowledge and opportunities being the most important factor.
P2. We should expect our knowledge and opportunities to continue into the future.
C. So we should expect the influentialness of those future people who we can pass resources on to be greater, too.
On the other hand, the observation that we are living on a single planet pulls in the direction of this time being more influential, and Toby had some reasons for thinking that early times should be more influential:
there are also good reasons to suspect that the chance of a century being the most influential should diminish over time. Especially because there are important kinds of significant event (such as the value lock-in or an existential catastrophe) where early occurrence blocks out later occurrence.
Overall it isn’t clear to me from this that we should have a decreasing prior over time? Can anyone help me out here?
I think I agree with the core claims Buck is making. But I found the logical structure of this post hard to follow. So here’s my attempt to re-present the core thread of the argument I think Buck is making:
Toby gives an example of an alternative (and in my mind better) prior as a reply to Will’s original post.
Note that anyone who conditions on a long future being possible should afterwards doubt the evidence for current xrisk to some degree. But Will is forced to do so to an extreme extent because his uniform prior on hinginess is such a bold one—whereas people with exponentially diminishing priors on hinginess like Toby’s won’t update as much after conditioning. All of this analysis remains roughly the same if you replace Will’s uniformity-of-hinginess prior with a uniformity-of-(preventable)-xrisk prior, and Toby’s exponentially-decreasing-hinginess prior with an exponentially-decreasing-(preventable)-xrisk prior. I add “preventable” here and above because if our current xrisk isn’t preventable, then it’s still possible that we’re in a low-hinginess period.
Lastly, it seems to me that conditioning on long futures being plausible was what caused a lot of the messiness here, and so for pedagogical purposes it’d probably be better to spell out all the different permutations of options more explicitly, and be clearer about when the conditioning is happening.
I don’t really get this. It seems that there are good reasons to believe early times are influential, but also good reasons to believe that later times are influential, and it isn’t clear to me which of these dominates.
For example, Will’s inductive argument against HH pulls towards later times being influential. From Will’s paper:
On the other hand, the observation that we are living on a single planet pulls in the direction of this time being more influential, and Toby had some reasons for thinking that early times should be more influential:
Overall it isn’t clear to me from this that we should have a decreasing prior over time? Can anyone help me out here?
I found this rephrasing helpful, thanks Richard.
I agree with all this; thanks for the summary.