So your prior says, unlike Will’s, that there are non-trivial probabilities of very early lock-in. That seems plausible and important. But it seems to me that your analysis not only uses a different prior but also conditions on “we live extremely early” which I think is problematic.
Will argues that it’s very weird we seem to be at an extremely hingy time. So we should discount that possibility. You say that we’re living at an extremely early time and it’s not weird for early times to be hingy. I imagine Will’s response would be “it’s very weird we seem to be living at an extremely early time then” (and it’s doubly weird if it implies we live in an extremely hingy time).
If living at an early time implies something that is extremely unlikely a priori for a random person from the timeline, then there should be an explanation. These 3 explanations seem exhaustive:
1) We’re extremely lucky.
2) We aren’t actually early: E.g. we’re in a simulation or the future is short. (The latter doesn’t necessarily imply that xrisk work doesn’t have much impact because the future might just be short in terms of people in our anthropic reference class).
3) Early people don’t actually have outsized influence: E.g. the hazard/hinge rate in your model is low (perhaps 1/N where N is the length of the future). In a Bayesian graphical model, there should be a strong update in favor of low hinge rates after observing that we live very early (unless another explanation is likely a priori).
Both 2) and 3) seem somewhat plausible a priori so it seems we don’t need to assume that a big coincidence explains how early we live.
I don’t think I’m building in any assumptions about living extremely early—in fact I think it makes as little assumption on that as possible. The prior you get from LLS or from Gott’s doomsday argument says the median number of people to follow us is as many as have lived so far (~100 billion), that we have an equal chance of being in any quantile, and so for example we only have a 1 in a million chance of living in the first millionth. (Though note that since each order of magnitude contributes an equal expected value and there are infinitely many orders of magnitude, the expected number of people is infinite / has no mean.)
If you’re just presenting a prior I agree that you’ve not conditioned on an observation “we’re very early”. But to the extent that your reasoning says there’s a non-trivial probability of [we have extremely high influence over a big future], you do condition on some observation of that kind. In fact, it would seem weird if any Copernican prior could give non-trivial mass to that proposition without an additional observation.
I continue my response here because the rest is more suitable as a higher-level comment.
It’s just an informal way to say that we’re probably typical observers. It’s named after Copernicus because he found that the Earth isn’t as special as people thought.
I don’t know the history of the term or its relationship to Copernicus, but I can say how my forgotten source defined it. Suppose you want to ask, “How long will my car run?” Suppose it’s a weird car that has a different engine and manufacturer than other cars, so those cars aren’t much help. One place you could start is with how long it’s currently be running for. This is based on the prior that you’re observing it on average halfway through its life. If it’s been running for 6 months so far, you would guess 1 year. There surely exists a more rigorous definition than this, but that’s the gist.
So your prior says, unlike Will’s, that there are non-trivial probabilities of very early lock-in. That seems plausible and important. But it seems to me that your analysis not only uses a different prior but also conditions on “we live extremely early” which I think is problematic.
Will argues that it’s very weird we seem to be at an extremely hingy time. So we should discount that possibility. You say that we’re living at an extremely early time and it’s not weird for early times to be hingy. I imagine Will’s response would be “it’s very weird we seem to be living at an extremely early time then” (and it’s doubly weird if it implies we live in an extremely hingy time).
If living at an early time implies something that is extremely unlikely a priori for a random person from the timeline, then there should be an explanation. These 3 explanations seem exhaustive:
1) We’re extremely lucky.
2) We aren’t actually early: E.g. we’re in a simulation or the future is short. (The latter doesn’t necessarily imply that xrisk work doesn’t have much impact because the future might just be short in terms of people in our anthropic reference class).
3) Early people don’t actually have outsized influence: E.g. the hazard/hinge rate in your model is low (perhaps 1/N where N is the length of the future). In a Bayesian graphical model, there should be a strong update in favor of low hinge rates after observing that we live very early (unless another explanation is likely a priori).
Both 2) and 3) seem somewhat plausible a priori so it seems we don’t need to assume that a big coincidence explains how early we live.
I don’t think I’m building in any assumptions about living extremely early—in fact I think it makes as little assumption on that as possible. The prior you get from LLS or from Gott’s doomsday argument says the median number of people to follow us is as many as have lived so far (~100 billion), that we have an equal chance of being in any quantile, and so for example we only have a 1 in a million chance of living in the first millionth. (Though note that since each order of magnitude contributes an equal expected value and there are infinitely many orders of magnitude, the expected number of people is infinite / has no mean.)
If you’re just presenting a prior I agree that you’ve not conditioned on an observation “we’re very early”. But to the extent that your reasoning says there’s a non-trivial probability of [we have extremely high influence over a big future], you do condition on some observation of that kind. In fact, it would seem weird if any Copernican prior could give non-trivial mass to that proposition without an additional observation.
I continue my response here because the rest is more suitable as a higher-level comment.
What is a Copernican prior? I can’t find any google results
It’s just an informal way to say that we’re probably typical observers. It’s named after Copernicus because he found that the Earth isn’t as special as people thought.
I don’t know the history of the term or its relationship to Copernicus, but I can say how my forgotten source defined it. Suppose you want to ask, “How long will my car run?” Suppose it’s a weird car that has a different engine and manufacturer than other cars, so those cars aren’t much help. One place you could start is with how long it’s currently be running for. This is based on the prior that you’re observing it on average halfway through its life. If it’s been running for 6 months so far, you would guess 1 year. There surely exists a more rigorous definition than this, but that’s the gist.
Wikipedia gives the physicist’s version, but EAs (and maybe philosophers?) use it more broadly.
https://en.wikipedia.org/wiki/Copernican_principle
The short summary I use to describe it is that “we” are not that special, for various definitions of the word we.
Some examples on FB.