Well, I agree that this is pretty in the weeds, but personally this has made me view the evolutionary anchor as less forceful.
Like, the argument isn’t “ha, we’re not going to be able to simulate evolution, checkmate AGI doomers”, it’s “the evolutionary anchor was a particularly forceful argument for giving a substantial probability to x-risk this century, even to people who might otherwise be very skeptical. The fact that it doesn’t go through has a variety of small update, e.g., it marginally increases the value of non-x-risk longtermism”
Huh, I guess I didn’t realize how much weight some people put on the evolution anchor. I thought everyone was (like me) treating it as a loose upper bound basically, not something to actually clump lots of probability mass on.
In other words: The people I know who were using the evolutionary anchor (people like myself, Ajeya, etc.) weren’t using it in a way that would be significantly undermined by having to push the anchor up 6 OOMs or so. Like I said, it would be a minor change to the bottom line according to the spreadsheet. Insofar as people were arguing for AGI this century in a way which can be undermined by adding 6 OOMs to the evolutionary anchor then those people are silly & should stop, for multiple reasons, one of which is that maaaybe environmental simulation costs mean that the evolution anchor really is 6 OOMs bigger than Ajeya estimates.
Well, I agree that this is pretty in the weeds, but personally this has made me view the evolutionary anchor as less forceful.
Like, the argument isn’t “ha, we’re not going to be able to simulate evolution, checkmate AGI doomers”, it’s “the evolutionary anchor was a particularly forceful argument for giving a substantial probability to x-risk this century, even to people who might otherwise be very skeptical. The fact that it doesn’t go through has a variety of small update, e.g., it marginally increases the value of non-x-risk longtermism”
Huh, I guess I didn’t realize how much weight some people put on the evolution anchor. I thought everyone was (like me) treating it as a loose upper bound basically, not something to actually clump lots of probability mass on.
In other words: The people I know who were using the evolutionary anchor (people like myself, Ajeya, etc.) weren’t using it in a way that would be significantly undermined by having to push the anchor up 6 OOMs or so. Like I said, it would be a minor change to the bottom line according to the spreadsheet. Insofar as people were arguing for AGI this century in a way which can be undermined by adding 6 OOMs to the evolutionary anchor then those people are silly & should stop, for multiple reasons, one of which is that maaaybe environmental simulation costs mean that the evolution anchor really is 6 OOMs bigger than Ajeya estimates.