Beyond these, one could build a community around finding forecasts of public figures. Alternatively, I guess GPT-3 has a good shot of being able to turn verbal forecasts into data which could then be checked.
What’s the impact
I’m only gonna sketch my argument here. As above, if this gets 20 karma I’ll write a full post (but only upvote if it’s good, let’s not waste any of our time).
We seem to think forecasting improves the accuracy of commentator
If we could build a high-status award for forecasting, more commentators would hear about it and it would serve as a nudge for others to make their forecasts more visible
I am confident this would lead to better commentary (this seems arrogant, but honestly the people I know who forecast more are more epistemically humble—I think celebrities could really benefit from more humility about their predictions)
Better commentary leads to better outcomes. Effective Altruism implicitly holds that many have priority orderings that don’t match reality. The world at large underrates the best charities, the chance of biorisk, etc. Journalism which was more accurate would be more accurate about these things too which would be a massive win
Wouldn’t the winners just be superforecasters
Not currently. I don’t think it’s too hard to make pretty robust boundaries on what a public figure is. Most superforecasters are not well enough known (and sorry to the 5 EAs I can count in metaculus’ top 50). But Yglesias is well known enough. Scott Alexander, I’m less sure but I think we could come up with some minimum amount of hits, followers, etc for someone to be eligible.
How much resource would this take
Depends on a couple of things (I have pulled these numbers out of thin air) please criticise them:
Who is giving this award its prestige? If it’s a lot of money, fine. If it’s an existing org, then it’s cheaper ( 0 - $50k)
How deeply are we looking. I think you could pay someone $50k to find say 100 public sets of forecasts and maybe another $10k to make a nice website. If you want to scrape twitter using GPT3 or crowdsource that’s maybe another $50-100k
Is there an award ceremony? If so I imagine that costs as much as a wedding so maybe $10k
That looks like $60 - $220k
If this failed, why did it fail?
It got embroiled in controversy over who was included
It was attached to some existing EA org and looked badly for them
It became a niche award that no one changed their behaviour based on
EA infrastructure idea: Best Public Forecaster Award
Gather all public forecasting track records
Present them in an easily navigable form
Award prizes one for best brier score of forecasts resolving in the last year
If this gets more than 20 karma, I’ll write a full post on it. This is rough.
Questions that come to mind
Where would we find these forecasts
To begin with I would look at those with public records:
Scott Alexander
Bryan Caplan
Matthew Yglesias
Many such cases
Beyond these, one could build a community around finding forecasts of public figures. Alternatively, I guess GPT-3 has a good shot of being able to turn verbal forecasts into data which could then be checked.
What’s the impact
I’m only gonna sketch my argument here. As above, if this gets 20 karma I’ll write a full post (but only upvote if it’s good, let’s not waste any of our time).
We seem to think forecasting improves the accuracy of commentator
If we could build a high-status award for forecasting, more commentators would hear about it and it would serve as a nudge for others to make their forecasts more visible
I am confident this would lead to better commentary (this seems arrogant, but honestly the people I know who forecast more are more epistemically humble—I think celebrities could really benefit from more humility about their predictions)
Better commentary leads to better outcomes. Effective Altruism implicitly holds that many have priority orderings that don’t match reality. The world at large underrates the best charities, the chance of biorisk, etc. Journalism which was more accurate would be more accurate about these things too which would be a massive win
Wouldn’t the winners just be superforecasters
Not currently. I don’t think it’s too hard to make pretty robust boundaries on what a public figure is. Most superforecasters are not well enough known (and sorry to the 5 EAs I can count in metaculus’ top 50). But Yglesias is well known enough. Scott Alexander, I’m less sure but I think we could come up with some minimum amount of hits, followers, etc for someone to be eligible.
How much resource would this take
Depends on a couple of things (I have pulled these numbers out of thin air) please criticise them:
Who is giving this award its prestige? If it’s a lot of money, fine. If it’s an existing org, then it’s cheaper ( 0 - $50k)
How deeply are we looking. I think you could pay someone $50k to find say 100 public sets of forecasts and maybe another $10k to make a nice website. If you want to scrape twitter using GPT3 or crowdsource that’s maybe another $50-100k
Is there an award ceremony? If so I imagine that costs as much as a wedding so maybe $10k
That looks like $60 - $220k
If this failed, why did it fail?
It got embroiled in controversy over who was included
It was attached to some existing EA org and looked badly for them
It became a niche award that no one changed their behaviour based on