Hey Joshua, appreciate you sharing your thoughts (strong upvoted)! I think we actually agree about the effects of sharing numerical credences more than you might think, but disagree about the solution.
But it also causes people to anchor on what may ultimately be an extremely shaky and speculative guess, hindering further independent analysis and leading to long citation trails. For example, I think the “1-in-6” estimate from The Precipice may have led to premature anchoring on that figure, and likely is relied upon too much relative to how speculative it necessarily is.
I agree that this is a substantial downside of sharing numerical credences. I saw it first-hand with people taking the numbers in my previous post more seriously than I had intended (as you also mentioned!)
However, I think there are large benefits to sharing numerical credences, such that the solution isn’t to share credences less but instead to improve the culture around them.
I think we should shift EA’s culture be more favorable of sharing numerical credences even (especially!) when everyone involved knows they’re tentative, brittle, etc. And we should be able to have discussions involving credences and worry less that others will take them too seriously.
I’ve been hopefully contributing to this some, e.g. by describing my motivation for including confidence numbers as: “I decided it was worth it to propose a definition and go ahead and use as many made-up numbers as possible for transparency.” And I’ve attempted to push back when I’ve perceived others as having taken credences/BOTECs I’ve given too seriously in the past.
Some more ideas for shifting the culture around numerical credences:
Use resilience to demonstrate how brittle your beliefs are.
Highlight how much other reasonable people disagree with your credences.
Openly change your mind and publicly shift your credences when new evidence comes in, or someone presents a good counter-argument.
Explicitly encourage others not to cite your numbers, if you believe they are too brittle (you mention this in your other comment).
I’d love to get others’ ideas for shifting the culture here!
Hey Joshua, appreciate you sharing your thoughts (strong upvoted)! I think we actually agree about the effects of sharing numerical credences more than you might think, but disagree about the solution.
I agree that this is a substantial downside of sharing numerical credences. I saw it first-hand with people taking the numbers in my previous post more seriously than I had intended (as you also mentioned!)
However, I think there are large benefits to sharing numerical credences, such that the solution isn’t to share credences less but instead to improve the culture around them.
I think we should shift EA’s culture be more favorable of sharing numerical credences even (especially!) when everyone involved knows they’re tentative, brittle, etc. And we should be able to have discussions involving credences and worry less that others will take them too seriously.
I’ve been hopefully contributing to this some, e.g. by describing my motivation for including confidence numbers as: “I decided it was worth it to propose a definition and go ahead and use as many made-up numbers as possible for transparency.” And I’ve attempted to push back when I’ve perceived others as having taken credences/BOTECs I’ve given too seriously in the past.
Some more ideas for shifting the culture around numerical credences:
Use resilience to demonstrate how brittle your beliefs are.
Highlight how much other reasonable people disagree with your credences.
Openly change your mind and publicly shift your credences when new evidence comes in, or someone presents a good counter-argument.
Explicitly encourage others not to cite your numbers, if you believe they are too brittle (you mention this in your other comment).
I’d love to get others’ ideas for shifting the culture here!