As you (and other commenters) note, another aspect of Pascalian probabilities is their subjectivity/ambiguity. Even if you can’t (accurately) generate “what is the probability I get hit by a car if I run across this road now?”, you have “numbers you can stand somewhat near” to gauge the risk—or at least ‘this has happened before’ case studies (cf. asteroids). Although you can motivate more longtermist issues via similar means (e.g. “Well, we’ve seen pandemics at least this bad before”, “What’s the chance folks raising grave concern about an emerging technology prove to be right?”) you typically have less to go on and are reaching further from it.
I think we share similar intuitions: this is a reasonable consideration, but it seems better to account for it quantitatively (e.g. with a sceptical prior or discount for ‘distance from solid epistemic ground’) rather than a qualitative heuristic. E.g. it seems reasonable to discount AI risk estimates (potentially by orders of magnitude) if it all seems very outlandish to you—but then you should treat these ‘all things considered’ estimates at face value.
Thanks for this, Richard.
As you (and other commenters) note, another aspect of Pascalian probabilities is their subjectivity/ambiguity. Even if you can’t (accurately) generate “what is the probability I get hit by a car if I run across this road now?”, you have “numbers you can stand somewhat near” to gauge the risk—or at least ‘this has happened before’ case studies (cf. asteroids). Although you can motivate more longtermist issues via similar means (e.g. “Well, we’ve seen pandemics at least this bad before”, “What’s the chance folks raising grave concern about an emerging technology prove to be right?”) you typically have less to go on and are reaching further from it.
I think we share similar intuitions: this is a reasonable consideration, but it seems better to account for it quantitatively (e.g. with a sceptical prior or discount for ‘distance from solid epistemic ground’) rather than a qualitative heuristic. E.g. it seems reasonable to discount AI risk estimates (potentially by orders of magnitude) if it all seems very outlandish to you—but then you should treat these ‘all things considered’ estimates at face value.