Interesting work! One of my main annoyances with pop-Bayesianism as practiced casually by EA is the focus on giving single number estimates with no uncertainty ranges.
In contrast, a primer on Bayesian statistics will switch to distributions pretty much immediately. If you Wikipedia “prior”, you immediately get “prior probability distribution”. I blame the sequences for this, which seemingly decided to just stay in Bayes 101 territory and barely mention Bayes 102.
I’m curious about your methodology: Is it based on existing Bayesian statistical methods, or a competing theory? Do you think it could be approximated in a way that would help a casual user make approximate predictions? How would you apply it to a real world problem, like the “P(doom)” issue that everyone is always going on about?
Yes, I agree distributions are better than single numbers. I think part of the problem for podcasts/conversations is that it’s easier to quickly say a number than a probability distribution, though that excuse works slightly less well for the written medium.
I didn’t base it off an existing method. While @Jobst tells me I have good “math instincts” that has yet to translate itself into actually being good at math, so this mostly comes from me reading the philosophical literature and trying to come up with solutions to some of the proposed problems. Maybe something similar already exists in math, though Jobst and some people he talked to didn’t know of it and they’re professional mathematicians.
As for casual users, I would urge them to take ‘agnosticism’ (or at least large ranges) seriously. Sometimes we really do not know something and the EA culture of urging people to put a number on something can give us a false sense of confidence that we understand it. Especially in scenarios of interactions between agents where mind games can cause estimations to behave irregularly. I mentioned how this can go wrong with a prediction market, but a version of that can happen with any group project. Regular human beings do on occasion foreact e.g. If we know we like each-other, and I think you’ll expect me to do something on a special day, the chance that I will do it is higher than if I didn’t think so. All of this doesn’t even mention a problem with Bayesianism I wasn’t able to solve, the absent-minded driver problem. Once we add fallible memories to foreaction the math goes well beyond me.
I don’t know what “P(doom)” means. Even beyond the whole problem of modeling billions of interacting irrational agents some of whom will change their behavior based on what I predict, I just don’t think the question is clear. Like, if I do something that decreases the chance of a sharp global welfare regression, increases the chance of an s-risk, and has no effect on x-risk, what happens to P(doom)? Are we all talking about the same thing? Shouldn’t this at the very least be two variables, one for probability and one for how “doom-y” it is? Wouldn’t that variable be different depending on your normative framework? What about inequality-aversion, if there is one super duper über happy utility monster and all other life is wiped out, is that a ‘doom’ scenario? What about time discounting, does the heat death of the universe mean that P(doom) is 1? I don’t know, P(doom) confuses me.
Interesting work! One of my main annoyances with pop-Bayesianism as practiced casually by EA is the focus on giving single number estimates with no uncertainty ranges.
In contrast, a primer on Bayesian statistics will switch to distributions pretty much immediately. If you Wikipedia “prior”, you immediately get “prior probability distribution”. I blame the sequences for this, which seemingly decided to just stay in Bayes 101 territory and barely mention Bayes 102.
I’m curious about your methodology: Is it based on existing Bayesian statistical methods, or a competing theory? Do you think it could be approximated in a way that would help a casual user make approximate predictions? How would you apply it to a real world problem, like the “P(doom)” issue that everyone is always going on about?
Thank you!
Yes, I agree distributions are better than single numbers. I think part of the problem for podcasts/conversations is that it’s easier to quickly say a number than a probability distribution, though that excuse works slightly less well for the written medium.
I didn’t base it off an existing method. While @Jobst tells me I have good “math instincts” that has yet to translate itself into actually being good at math, so this mostly comes from me reading the philosophical literature and trying to come up with solutions to some of the proposed problems. Maybe something similar already exists in math, though Jobst and some people he talked to didn’t know of it and they’re professional mathematicians.
As for casual users, I would urge them to take ‘agnosticism’ (or at least large ranges) seriously. Sometimes we really do not know something and the EA culture of urging people to put a number on something can give us a false sense of confidence that we understand it. Especially in scenarios of interactions between agents where mind games can cause estimations to behave irregularly. I mentioned how this can go wrong with a prediction market, but a version of that can happen with any group project. Regular human beings do on occasion foreact e.g. If we know we like each-other, and I think you’ll expect me to do something on a special day, the chance that I will do it is higher than if I didn’t think so.
All of this doesn’t even mention a problem with Bayesianism I wasn’t able to solve, the absent-minded driver problem. Once we add fallible memories to foreaction the math goes well beyond me.
I don’t know what “P(doom)” means. Even beyond the whole problem of modeling billions of interacting irrational agents some of whom will change their behavior based on what I predict, I just don’t think the question is clear. Like, if I do something that decreases the chance of a sharp global welfare regression, increases the chance of an s-risk, and has no effect on x-risk, what happens to P(doom)? Are we all talking about the same thing? Shouldn’t this at the very least be two variables, one for probability and one for how “doom-y” it is? Wouldn’t that variable be different depending on your normative framework? What about inequality-aversion, if there is one super duper über happy utility monster and all other life is wiped out, is that a ‘doom’ scenario? What about time discounting, does the heat death of the universe mean that P(doom) is 1? I don’t know, P(doom) confuses me.