I think in this context the natural way to interpret probabilities (or medians) of probabilities is as something like: “what would I think the probability was if I knew a lot more about the situation and got to think about it a lot more, but wasn’t literally omniscient”. Of course that isn’t fully defined (since it’s not specified exactly how much more they get to know), but I think it’s approximately meaningful, and can capture something quite useful/important about credal resilience.
Relatedly, I think the journalist’s presentation is misleading. I think it makes it sound like there’s something like an objective 1-in-230 chance of catastrophe. I think a more accurate presentation would be “expert thinks there’s a 1% chance we live in a world with a high risk”. This changes the appropriate policy implications: if taken at face value they both suggest there’s something important to do, but in the latter case the action is less like “work out what to do about simulation shutdown” and more like “start a serious research programme to work out how likely it is we’re in the fraction of worlds that was offhand labelled as 1%”.
(I also think that you may be right that the cosmologist didn’t really believe their numbers. I think that issue alone would be enough reason to be suspicious of the journalist’s approach. They might reasonably suspect that such off-the-cuff answers might not withstand scrutiny, so it seems bad to report them in a way that could sound like considered expert opinion.)
Yeah I’m arguing that with good reflective governance we should achieve a large fraction of what’s accessible.
It’s quite possible that that means “not quite all”, e.g. maybe there are some trades so that we don’t aestivate in this galaxy, but do in the rest of them; but on the aggregative view that’s almost as good as aestivating everywhere.