Aggregation choices (e.g. geo mean of odds would be nice)
Geo mean of odds is a good idea—it’s probably a more sensible default. How would you feel about us using that everywhere, instead of the current arithmetic mean?
Brier scores for users
You can see your own absolute and relative Brier score in the app home (click Fatebook in the sidebar). If you’re thinking of a team-wide leaderboard—that’s on our list! Though some users said they wouldn’t like this to avoid Goodharting, so I’ve not prioritised it so far, and will include a team-wide toggle if we add it.
Ah, great! I think it would be nice to offer different aggregation options, though if you do offer one I agree that geo mean of odds is the best default. But I can imagine people wanting to use medians or averages, or even specifying their own aggregation functions. Especially if you are trying to encourage uptake by less technical organizations, it seems important to offer at least one option that is more legible to less numerate people.
Great, glad to hear it!
Geo mean of odds is a good idea—it’s probably a more sensible default. How would you feel about us using that everywhere, instead of the current arithmetic mean?
You can see your own absolute and relative Brier score in the app home (click Fatebook in the sidebar). If you’re thinking of a team-wide leaderboard—that’s on our list! Though some users said they wouldn’t like this to avoid Goodharting, so I’ve not prioritised it so far, and will include a team-wide toggle if we add it.
We’ll add this soon!
Ah, great! I think it would be nice to offer different aggregation options, though if you do offer one I agree that geo mean of odds is the best default. But I can imagine people wanting to use medians or averages, or even specifying their own aggregation functions. Especially if you are trying to encourage uptake by less technical organizations, it seems important to offer at least one option that is more legible to less numerate people.
I thought of some other down-the-line feature requests
Google Sheets integration (we currently already store our forecasts in a Google sheet)
Relatedly, ability to export to CSV (does this already exist and I just missed it?)
Ability to designate a particular resolver
Different formal resolution mechanisms, like a poll of users.