Excellent! Would it be costly to also show the results in a log-scored format? I am not predicting they would be massively different from Brier. I just get really hung up about the way Brier only punishes e.g. 99.999% negligibly more than 97% (given the event didn’t happen).
It would make the post longer and more confusing I think, and not obviously more useful. Metaculus only allows up to 99% predictions, with increments of 1%, and I already exclude predictions of 1-9% and 91-99% from much of the analysis for reasons given in the post, thus excluding the specific area where I agree log scores are meaningfully better.
Excellent! Would it be costly to also show the results in a log-scored format? I am not predicting they would be massively different from Brier. I just get really hung up about the way Brier only punishes e.g. 99.999% negligibly more than 97% (given the event didn’t happen).
It would make the post longer and more confusing I think, and not obviously more useful. Metaculus only allows up to 99% predictions, with increments of 1%, and I already exclude predictions of 1-9% and 91-99% from much of the analysis for reasons given in the post, thus excluding the specific area where I agree log scores are meaningfully better.