More Facebook discussion of this post:
Satvik Beri: I think Bayes’ Theorem is extremely hard to apply usefully, to the point that I rarely use it at all despite working in data science.
A major problem that leads people to be underconfident is the temptation to round down evidence to reasonable odds, like the post mentions. A major problem that leads people to be overconfident is applying lots of small pieces of information while discounting the correlations between them.
A comment [on LessWrong] mentions that if you have excellent returns for a year, that’s strong evidence you’re a top 1% trader. That’s not really true, the market tends to move in regimes for long periods of time, so a strategy that works well for a year is pretty likely to have average performance the next year. Studies on hedge fund managers have found it is extremely difficult to find consistent outperformers, e.g. 5-year performance on pretty much any metric is uncorrelated to the performance on that metric next year.
I didn’t say anything about what size/duration of returns would make you a top 1% trader.