Thanks for engaging positively! You’re correct about the crux—if the resulting prediction market worked really well, the technical complains wouldn’t matter. But the number of predictions is much less important to me than their trustworthiness and the precision of specifying exactly what is being predicted. Being well-calibrated is good, but does not necessarily indicate good precision (i.e. a good Brier score), and that calibration.city is quite misleading in presenting the orders of magnitude more questions on manifold as a larger dot, rather than using dot size to indicate uncertainty bounds in the calibration.
It’s not true that markets at any scale produce the most accurate forecasts. There’s extensive literature showing that long-term prediction markets need to worry about the time-value of money and risk aversion influencing the market valuation. Manifold’s old loan system helped alleviate the time-value problem but gave you a negative equity problem. I don’t see this time value effect in your calibration data, but I suspect that’s dominated by short-term markets. Because market participation is strongly affected by liquidity, smaller markets don’t have incentives for people to get involved in them unless they’re very wrong. Thus getting markets to scale up when they’re not intrinsically controversial and therefore interesting is a substantial problem. The incentives to make accurate predictions can just be prizes for accurate individual predictions which can be aggregated into a site prediction by any other mechanism. The key feature of a market mechanism for prediction aggregation is that the reward must be tied to the probability of the event, and must be blind to who is providing the money. There’s no reason to believe either of these are useful constraints, and I don’t believe they’re optimal.
I note that many accounts are still in negative equity, and that a few such accounts that primarily generated their wealth by betting on weird metamarkets substantially influence the price of AI extinction risk markets. The number and variety of markets is therefore potentially punitive to the accuracy of predictions, particularly given the power-law rewards to market participation. While I refer to negative equity, the fact that we can still create puppets and transfer their $200 to another user (directly or via bad bets) means the problem persists to a smaller extent without anyone’s account going negative.
Thanks for engaging positively! You’re correct about the crux—if the resulting prediction market worked really well, the technical complains wouldn’t matter. But the number of predictions is much less important to me than their trustworthiness and the precision of specifying exactly what is being predicted. Being well-calibrated is good, but does not necessarily indicate good precision (i.e. a good Brier score), and that calibration.city is quite misleading in presenting the orders of magnitude more questions on manifold as a larger dot, rather than using dot size to indicate uncertainty bounds in the calibration.
It’s not true that markets at any scale produce the most accurate forecasts. There’s extensive literature showing that long-term prediction markets need to worry about the time-value of money and risk aversion influencing the market valuation. Manifold’s old loan system helped alleviate the time-value problem but gave you a negative equity problem. I don’t see this time value effect in your calibration data, but I suspect that’s dominated by short-term markets. Because market participation is strongly affected by liquidity, smaller markets don’t have incentives for people to get involved in them unless they’re very wrong. Thus getting markets to scale up when they’re not intrinsically controversial and therefore interesting is a substantial problem. The incentives to make accurate predictions can just be prizes for accurate individual predictions which can be aggregated into a site prediction by any other mechanism. The key feature of a market mechanism for prediction aggregation is that the reward must be tied to the probability of the event, and must be blind to who is providing the money. There’s no reason to believe either of these are useful constraints, and I don’t believe they’re optimal.
I note that many accounts are still in negative equity, and that a few such accounts that primarily generated their wealth by betting on weird metamarkets substantially influence the price of AI extinction risk markets. The number and variety of markets is therefore potentially punitive to the accuracy of predictions, particularly given the power-law rewards to market participation. While I refer to negative equity, the fact that we can still create puppets and transfer their $200 to another user (directly or via bad bets) means the problem persists to a smaller extent without anyone’s account going negative.