Greenberg 2018 lists and evaluates forecasting scoring rules. Research on additionally more complex metrics that take into account e.g.:
importance of the questions forecasted on (perhaps using an interest score such as Metaculus)
number of questions forecasted on (brier score distorts if only few forecasts)
relative performance as compared to other forecasters on similar sets of questions
Might be useful to set incentives right in forecasting tournaments. Prediction markets solve the first point by logarithmic subsidising.
Greenberg 2018 lists and evaluates forecasting scoring rules. Research on additionally more complex metrics that take into account e.g.:
importance of the questions forecasted on (perhaps using an interest score such as Metaculus)
number of questions forecasted on (brier score distorts if only few forecasts)
relative performance as compared to other forecasters on similar sets of questions
Might be useful to set incentives right in forecasting tournaments. Prediction markets solve the first point by logarithmic subsidising.