I do understand what you are saying, but my response (albeit as someone who is not steeped in longtermist/X-risk thought) would be “not necessarily (and almost certainly not entirely).”The tl;dr version is “there are lots of claims about X-risks and interventions to reduce x-risks that are reasonably more plausible than their reverse-claim.” e.g., there are decent reasons to believe that certain forms of pandemic preparations reduce x-risk more than they increase x-risk. I can’t (yet) give full, formalistic rules for how I apply the trimming heuristic, but some of the major points are discussed in the blocks below.
One key to using/understanding the trimming heuristic is that it is not meant to directly maximize the accuracy of your beliefs, rather it’s meant to improve the effectiveness of your overall decision-making *in light of constraints on your time/cognitive resources. * If we had infinite time to evaluate everything—even possibilities that seem like red herrings—it would probably (usually) be optimal to do so, but we don’t have infinite time so we have to make decisions as to what to spend our time analyzing and what to accept as “best-guesstimates” for particularly fuzzy questions. Here, intuition (including “when should we rely on various levels of intuition/analysis”) can be far more effective than formalistic rules.
I think another key is to understand the distinction between risk and uncertainty: (to heavily simplify) risk refers to confidently verifiable/specific probabilities (e.g., a 1⁄20 chance of rolling a 1 on a standard 20-sided die) whereas uncertainty refers to when we don’t confidently know the specific degree of risk (e.g., the chance of rolling a 1 on a confusingly-shaped 20-sided die which has never rolled a 1 yet, but perhaps might eventually).
In the end, I think my 3-4-ish conditions or at least factors for using the trimming heuristic are:
There is a high degree of uncertainty associated with the claim (e.g., it is not a well-established fact that there is a +0.01% chance of extinction upon enacting this policy)
The claim seems rather implausible/exaggerated on its face, but would require a non-trivial amount of time to clearly explain why (since it gets increasingly difficult to show why you ought to increase the number of zeros after a decimal point)
You can quickly fight fire with fire (e.g., think of opposite-outcome claims like I described)
There are other, more-realistic arguments to consider and your time is limited.
I do understand what you are saying, but my response (albeit as someone who is not steeped in longtermist/X-risk thought) would be “not necessarily (and almost certainly not entirely).”The tl;dr version is “there are lots of claims about X-risks and interventions to reduce x-risks that are reasonably more plausible than their reverse-claim.” e.g., there are decent reasons to believe that certain forms of pandemic preparations reduce x-risk more than they increase x-risk. I can’t (yet) give full, formalistic rules for how I apply the trimming heuristic, but some of the major points are discussed in the blocks below.
One key to using/understanding the trimming heuristic is that it is not meant to directly maximize the accuracy of your beliefs, rather it’s meant to improve the effectiveness of your overall decision-making *in light of constraints on your time/cognitive resources. * If we had infinite time to evaluate everything—even possibilities that seem like red herrings—it would probably (usually) be optimal to do so, but we don’t have infinite time so we have to make decisions as to what to spend our time analyzing and what to accept as “best-guesstimates” for particularly fuzzy questions. Here, intuition (including “when should we rely on various levels of intuition/analysis”) can be far more effective than formalistic rules.
I think another key is to understand the distinction between risk and uncertainty: (to heavily simplify) risk refers to confidently verifiable/specific probabilities (e.g., a 1⁄20 chance of rolling a 1 on a standard 20-sided die) whereas uncertainty refers to when we don’t confidently know the specific degree of risk (e.g., the chance of rolling a 1 on a confusingly-shaped 20-sided die which has never rolled a 1 yet, but perhaps might eventually).
In the end, I think my 3-4-ish conditions or at least factors for using the trimming heuristic are:
There is a high degree of uncertainty associated with the claim (e.g., it is not a well-established fact that there is a +0.01% chance of extinction upon enacting this policy)
The claim seems rather implausible/exaggerated on its face, but would require a non-trivial amount of time to clearly explain why (since it gets increasingly difficult to show why you ought to increase the number of zeros after a decimal point)
You can quickly fight fire with fire (e.g., think of opposite-outcome claims like I described)
There are other, more-realistic arguments to consider and your time is limited.