Minor note: the “Pascal’s Mugging” isn’t about the chance of x-risk itself, but rather the delta you can achieve through any particular program/action (vs. the cost of that choice).
By that token most particular scientific experiments or contributions to political efforts may be such: e.g. if there is a referendum to pass a pro-innovation regulatory reform and science funding package, a given donation or staffer in support of it is very unlikely to counterfactually tip it into passing, although the expected value and average returns could be high, and the collective effort has a large chance of success.
Sure, but the delta you can achieve with anything is small, depending on how you delineate an action. True, x-risk reduction is on the more extreme end of this spectrum, but I think the question should be “can these small deltas/changes add up to a big delta/change? (vs. the cost of that choice)” and the answer to that seems to be “yes.”
Is your issue more along the following?
Humans are bad at estimating very small percentages accurately, and can be orders of magnitudes off (and the same goes for astronomical values in the long-term future)
Arguments for the cost-effectiveness of x-risk reduction rely on estimating very small percentages (and the same goes for astronomical values in the long-term future)
(Conlusion) Arguments for the cost-effectiveness of x-risk reduction cannot be trusted.
If so, I would reject 2, because I believe we shouldn’t try to quantify things at those levels of precision. This does get us to your question “How does XR weigh costs and benefits?”, which I think is a good question to which I don’t have a great answer to. It would be something along the lines of “there’s a grey area where I don’t know how to make those tradeoffs, but most things do not fall into the grey area so I’m not worrying too much about this. If I wouldn’t fund something that supposedly reduces x-risk, it’s either that I think it might increase x-risk, or because I think there are better options available for me to fund”. Do you believe that many more choices fall into that grey area?
Thanks JP!
Minor note: the “Pascal’s Mugging” isn’t about the chance of x-risk itself, but rather the delta you can achieve through any particular program/action (vs. the cost of that choice).
By that token most particular scientific experiments or contributions to political efforts may be such: e.g. if there is a referendum to pass a pro-innovation regulatory reform and science funding package, a given donation or staffer in support of it is very unlikely to counterfactually tip it into passing, although the expected value and average returns could be high, and the collective effort has a large chance of success.
Sure, but the delta you can achieve with anything is small, depending on how you delineate an action. True, x-risk reduction is on the more extreme end of this spectrum, but I think the question should be “can these small deltas/changes add up to a big delta/change? (vs. the cost of that choice)” and the answer to that seems to be “yes.”
Is your issue more along the following?
Humans are bad at estimating very small percentages accurately, and can be orders of magnitudes off (and the same goes for astronomical values in the long-term future)
Arguments for the cost-effectiveness of x-risk reduction rely on estimating very small percentages (and the same goes for astronomical values in the long-term future)
(Conlusion) Arguments for the cost-effectiveness of x-risk reduction cannot be trusted.
If so, I would reject 2, because I believe we shouldn’t try to quantify things at those levels of precision. This does get us to your question “How does XR weigh costs and benefits?”, which I think is a good question to which I don’t have a great answer to. It would be something along the lines of “there’s a grey area where I don’t know how to make those tradeoffs, but most things do not fall into the grey area so I’m not worrying too much about this. If I wouldn’t fund something that supposedly reduces x-risk, it’s either that I think it might increase x-risk, or because I think there are better options available for me to fund”. Do you believe that many more choices fall into that grey area?