One more point to consider, you can sell your shares before the market closes which introduces another distortion in the probability: a lot of times I am thinking not so much in what the result of a market will be, but more on what probability people are going to assign to it. So if I see a market at 50% but I know that and event that just occurred will make people update the probability to 75%, I will buy YES at 50% all the way to say 70%, wait for the market to set to 75% and then sell to make a profit, perhaps having an effect on the market by my sell. If everybody does the same, the price is more of an estimate of people’s intuition than a probability for the event. This effect is stronger in things like politics or sports, where people overprice their teams.
Yet another one: just as with stocks, big players can influence the price and sometimes make the market move the way that they want (big name is betting NO, I should probably sell my YES) which won’t reflect on the real probability.
And to your last paragraph: you can actually compare “reliability” by estimating metrics such as the log score or (I think better) the Brier score. For example, Jack, one of the most successful users at manifold.markets, compared results for the election forecasts among multiple platforms: https://firstsigma.substack.com/p/midterm-elections-forecast-comparison-analysis the results were 1. Metaculus (which is non-monetary but also has their own system of aggregating forecasts), 2. 538 (poll aggregation and statistical modeling), 3. Manifold (non-monetary* pure prediction), and then other platforms including Polymarket. So at least for the last elections, that analysis supports your argument.
*: they give you some starting “money”, you can buy with real money more, and you can cash out their “money” to real money only to send it via donations
One more point to consider, you can sell your shares before the market closes which introduces another distortion in the probability: a lot of times I am thinking not so much in what the result of a market will be, but more on what probability people are going to assign to it. So if I see a market at 50% but I know that and event that just occurred will make people update the probability to 75%, I will buy YES at 50% all the way to say 70%, wait for the market to set to 75% and then sell to make a profit, perhaps having an effect on the market by my sell. If everybody does the same, the price is more of an estimate of people’s intuition than a probability for the event. This effect is stronger in things like politics or sports, where people overprice their teams.
Yet another one: just as with stocks, big players can influence the price and sometimes make the market move the way that they want (big name is betting NO, I should probably sell my YES) which won’t reflect on the real probability.
And to your last paragraph: you can actually compare “reliability” by estimating metrics such as the log score or (I think better) the Brier score. For example, Jack, one of the most successful users at manifold.markets, compared results for the election forecasts among multiple platforms: https://firstsigma.substack.com/p/midterm-elections-forecast-comparison-analysis the results were 1. Metaculus (which is non-monetary but also has their own system of aggregating forecasts), 2. 538 (poll aggregation and statistical modeling), 3. Manifold (non-monetary* pure prediction), and then other platforms including Polymarket. So at least for the last elections, that analysis supports your argument.
*: they give you some starting “money”, you can buy with real money more, and you can cash out their “money” to real money only to send it via donations