The Samotsvety track record does straightforwardly look better than what I expect the median superforecaster’s track record to be (which I think is ~99th percentile in either the original Tetlock studies or on GJO), especially on AI. Though perhaps Tetlock’s team also selected for better forecasters than the median superforecaster? It’s unclear to me.
“when I know a bunch of excellent forecasters...”
Perhaps your sampling techniques are better than Tetlock’s then.
The Samotsvety track record does straightforwardly look better than what I expect the median superforecaster’s track record to be (which I think is ~99th percentile in either the original Tetlock studies or on GJO), especially on AI. Though perhaps Tetlock’s team also selected for better forecasters than the median superforecaster? It’s unclear to me.