Political betting had a problem relative to perfection, not relative to the actual other alternatives used; it did better than them according to accuracy studies.
Yes there are overheads to using prediction markets, but those are mainly for having any system at all. Once you have a system, the overhead to adding a new question is much lower. Since you don’t have EA prediction markets now, you face those initial costs.
For forecasting in most organizations, hiring top 30 super forecasters would go badly, as they don’t know enough about that organization to be useful. Far better to have just a handful of participants from that organization.
I assumed you didn’t mean an internal World Bank prediction market, sorry about that. As I said above, I’m more optimistic about large workplaces employing prediction markets. I don’t know how many staff the World Bank employs. Do you agree now that prediction markets are an inferior solution to forecasting problems in small organizations? If yes, what do you think is the minimum staff size of a workplace for a prediction market to be efficient enough to be better than e.g. extremized team forecasting?
Could you link to the accuracy studies you cite that show that prediction markets do better than polling on predicting election results? I don’t see any obvious big differences on a quick Google search. The next obvious alternative is asking whether people like Nate Silver did better than prediction markets.
In the GJP, individual superforecasters did sometimes better than prediction markets, but team superforecasters did consistently better. Putting Nate Silver and his kin in a room seems to have a good chance to outperform prediction markets then.
You also don’t state your opinion on the Intrade incident. Since I cannot see that prediction markets are obviously a lot better than polls or pundits (they didn’t call the 2016 surprises either), I find it questionable whether blatant attempts at voter manipulation through prediction markets are worth the cost. This is a big price to pay even if prediction markets did a bit better than polls or pundits.
I find it questionable whether blatant attempts at voter manipulation through prediction markets are worth the cost. This is a big price to pay even if prediction markets did a bit better than polls or pundits.
Robin’s position is that manipulators can actually improve the accuracy of prediction markets, by increasing the rewards to informed trading. On this view, the possibility of market manipulation is not in itself a consideration that favors non-market alternatives, such as polls or pundits.
Interesting! I am trading off accuracy with outside world manipulation in that argument, since accuracy isn’t actually the main end goal I care about (but ‘good done in the world’ for which better forecasts of the future would be pretty useful).
Feel free to ignore if you don’t think this is sufficiently important, but I don’t understand the contrast you draw between accuracy and outside world manipulation. I thought manipulation of prediction markets was concerning precisely because it reduces their accuracy. Assuming you accept Robin’s point that manipulation increases accuracy on balance, what’s your residual concern?
Political betting had a problem relative to perfection, not relative to the actual other alternatives used; it did better than them according to accuracy studies.
Yes there are overheads to using prediction markets, but those are mainly for having any system at all. Once you have a system, the overhead to adding a new question is much lower. Since you don’t have EA prediction markets now, you face those initial costs.
For forecasting in most organizations, hiring top 30 super forecasters would go badly, as they don’t know enough about that organization to be useful. Far better to have just a handful of participants from that organization.
I assumed you didn’t mean an internal World Bank prediction market, sorry about that. As I said above, I’m more optimistic about large workplaces employing prediction markets. I don’t know how many staff the World Bank employs. Do you agree now that prediction markets are an inferior solution to forecasting problems in small organizations? If yes, what do you think is the minimum staff size of a workplace for a prediction market to be efficient enough to be better than e.g. extremized team forecasting?
Could you link to the accuracy studies you cite that show that prediction markets do better than polling on predicting election results? I don’t see any obvious big differences on a quick Google search. The next obvious alternative is asking whether people like Nate Silver did better than prediction markets. In the GJP, individual superforecasters did sometimes better than prediction markets, but team superforecasters did consistently better. Putting Nate Silver and his kin in a room seems to have a good chance to outperform prediction markets then.
You also don’t state your opinion on the Intrade incident. Since I cannot see that prediction markets are obviously a lot better than polls or pundits (they didn’t call the 2016 surprises either), I find it questionable whether blatant attempts at voter manipulation through prediction markets are worth the cost. This is a big price to pay even if prediction markets did a bit better than polls or pundits.
Robin’s position is that manipulators can actually improve the accuracy of prediction markets, by increasing the rewards to informed trading. On this view, the possibility of market manipulation is not in itself a consideration that favors non-market alternatives, such as polls or pundits.
Interesting! I am trading off accuracy with outside world manipulation in that argument, since accuracy isn’t actually the main end goal I care about (but ‘good done in the world’ for which better forecasts of the future would be pretty useful).
Feel free to ignore if you don’t think this is sufficiently important, but I don’t understand the contrast you draw between accuracy and outside world manipulation. I thought manipulation of prediction markets was concerning precisely because it reduces their accuracy. Assuming you accept Robin’s point that manipulation increases accuracy on balance, what’s your residual concern?