It’s the other way around for me. Historical baseline may be somewhat arbitrary and unreliable, but so is 1:1 odds.
Agreed! To give some nuance to my recommendation, the reason I am hesitant is mainly because of lack of academic precedent (as far as I know).
If the motivation for extremizing is that different forecasters have access to independent sources of information to move them away from a common prior, but that common prior is far from 1:1 odds, then extremizing away from 1:1 odds shouldn’t work very well.
Note that the data backs this up! Using “pseudo-historical” odds is quite better than using 1:1 odds. See the appendix for more details.
[...] use past estimates of the same question.
[...] use the odds that experts gave it at some point in the past as a baseline with which to interpret more recent odds estimates provided by experts.
I’d be interested in seeing the results of such experiments using Metaculus data!
Another possibility is to use two pools of forecasters [...]
Thanks for chipping in Alex!
Agreed! To give some nuance to my recommendation, the reason I am hesitant is mainly because of lack of academic precedent (as far as I know).
Note that the data backs this up! Using “pseudo-historical” odds is quite better than using 1:1 odds. See the appendix for more details.
I’d be interested in seeing the results of such experiments using Metaculus data!
This one is trippy, I like it!