[COI: I work at the Swift Centre as a forecaster, I have worked for a prediction market, I am very involved in forecasting. It is not my current work however, which is on community notes]
A few things points attempting to say things other commenters haven’t, though I largely agree with the critical comments and the things they agree with Marcus on:
I agree that the $100M doesn’t seem super well allocated. Not because forecasting is useless, but because the money flowed to big institutions and platforms rather than smaller, weirder, mechanism-design bets. I like Metaculus, but it has absorbed a lot of money in the last 5 years and not clearly changed much. I don’t know if I think FRI has been worth it, I am glad someone has done the research but, again, how much are we talking? I would have preferred smaller projects were funded on the margin. Coefficient’s strategy in forecasting has felt poor to me, often ignoring the community who in my view come up with the most interesting projects and going for marginal spending on incumbents.
Nobody funds mechanism design or institutional epistemics. I recently spoke to someone at a household name enormous tech company who described their institutional process. It was almost unbelievably dysfunctional to me. Who is funding the work to help institutions think better? It doesn’t promise near-term wins and frankly should’t be the priority of any non-research org. So basically no one. Forecasting is an attempt. How much value is there in the joint stock company, or in democracy. To me, that’s what we are talking about. Figuring out fundamentally better ways of making decisions. It is a problem at scale, it is neglected, and given the deregulation of prediction markets, tractable (though maybe bad, more later).
On “feels useful when it isn’t” (point 6). I don’t entirely disagree. I deliberately try not to spend time forecasting unless I’m being paid to. It can be a distraction. Where I disagree is that some forecasting is genuinely mentally sharpening, at least for me as a thinking discipline. And I think it’s a not unreasonable status hierarchy. Do I endorse the status that Peter Wildeford or Eli Leifland have gotten from forecasting? Yes. Frankly, who do I not endorse having got status from being a forecaster?
Why don’t AI 2027 and Ajeya count? Tangible forecasting outputs that demonstrably moved discourse and decision-making. Why don’t these count as valuable forecasting outputs. AI 2027 is clearly informed by judgemental forecasters and was read by (I think) the Vice President. Habryka said something like ‘too much time has been wasted down the resolution criteria mines’ and I disagree but even if one agrees, I’m not sure even he thinks the whole field is a waste of time.
Prediction markets may be net-harmful, but not useless. I’ve said publicly I’m less sure PMs are net-positive — bankruptcies and intimate partner violence are real and huge problems that may be as large as any coordinaton benefits. But ‘bad on net’ and ‘useless’ are different claims, and the later seems more obviously incorrect to me. I would be more interested in a post entitled “EA forecasting efforts have caused massive harm”.
I think it’s a bit of a cop out to suggest that the money was spent poorly and therefore it’s not fair to judge forecasting on the merits. Not sure if this is what you are saying though.
I’m not sure I agree. Blockchains were funded. Lots of academic research is funded by governments, etc.
By this logic, why aren’t we paying people in the top, say 100 on Polymarket on non-sports questions and saying “hey guys, we want to pay you to make some forecasts for us”.
I’m not sure how I feel about these. I will say, I am not nearly as big a fan of AI 2027 as others seem to be, and I think it is going to severely discredit AI risks because we have been crying wolf when frankly, most of the scary stuff they say doesn’t happen. I am very happy to make some bets with Daniel or Eli on some of these (and give them extra time).
Useless is a stretch and I didn’t intend to claim it originally. Sorry if I implied it. I merely think they are overrated, hence the title.
If it is not indiscreet, can you explain what you mean by “intimate partner violence”? Couples fighting due to different views on bets? or more something like bets on couples’ separations triggering the separation? I’m pretty curious what you mean.
Intimate partner violence refers to behaviour within an intimate relationship that causes physical, sexual or psychological harm, including acts of physical aggression, sexual coercion, psychological abuse and controlling behaviours. This definition covers violence by both current and former spouses and partners.
[COI: I work at the Swift Centre as a forecaster, I have worked for a prediction market, I am very involved in forecasting. It is not my current work however, which is on community notes]
A few things points attempting to say things other commenters haven’t, though I largely agree with the critical comments and the things they agree with Marcus on:
I agree that the $100M doesn’t seem super well allocated. Not because forecasting is useless, but because the money flowed to big institutions and platforms rather than smaller, weirder, mechanism-design bets. I like Metaculus, but it has absorbed a lot of money in the last 5 years and not clearly changed much. I don’t know if I think FRI has been worth it, I am glad someone has done the research but, again, how much are we talking? I would have preferred smaller projects were funded on the margin. Coefficient’s strategy in forecasting has felt poor to me, often ignoring the community who in my view come up with the most interesting projects and going for marginal spending on incumbents.
Nobody funds mechanism design or institutional epistemics. I recently spoke to someone at a household name enormous tech company who described their institutional process. It was almost unbelievably dysfunctional to me. Who is funding the work to help institutions think better? It doesn’t promise near-term wins and frankly should’t be the priority of any non-research org. So basically no one. Forecasting is an attempt. How much value is there in the joint stock company, or in democracy. To me, that’s what we are talking about. Figuring out fundamentally better ways of making decisions. It is a problem at scale, it is neglected, and given the deregulation of prediction markets, tractable (though maybe bad, more later).
On “feels useful when it isn’t” (point 6). I don’t entirely disagree. I deliberately try not to spend time forecasting unless I’m being paid to. It can be a distraction. Where I disagree is that some forecasting is genuinely mentally sharpening, at least for me as a thinking discipline. And I think it’s a not unreasonable status hierarchy. Do I endorse the status that Peter Wildeford or Eli Leifland have gotten from forecasting? Yes. Frankly, who do I not endorse having got status from being a forecaster?
Why don’t AI 2027 and Ajeya count? Tangible forecasting outputs that demonstrably moved discourse and decision-making. Why don’t these count as valuable forecasting outputs. AI 2027 is clearly informed by judgemental forecasters and was read by (I think) the Vice President. Habryka said something like ‘too much time has been wasted down the resolution criteria mines’ and I disagree but even if one agrees, I’m not sure even he thinks the whole field is a waste of time.
Prediction markets may be net-harmful, but not useless. I’ve said publicly I’m less sure PMs are net-positive — bankruptcies and intimate partner violence are real and huge problems that may be as large as any coordinaton benefits. But ‘bad on net’ and ‘useless’ are different claims, and the later seems more obviously incorrect to me. I would be more interested in a post entitled “EA forecasting efforts have caused massive harm”.
As promised, my reply (a couple days late).
I think it’s a bit of a cop out to suggest that the money was spent poorly and therefore it’s not fair to judge forecasting on the merits. Not sure if this is what you are saying though.
I’m not sure I agree. Blockchains were funded. Lots of academic research is funded by governments, etc.
By this logic, why aren’t we paying people in the top, say 100 on Polymarket on non-sports questions and saying “hey guys, we want to pay you to make some forecasts for us”.
I’m not sure how I feel about these. I will say, I am not nearly as big a fan of AI 2027 as others seem to be, and I think it is going to severely discredit AI risks because we have been crying wolf when frankly, most of the scary stuff they say doesn’t happen. I am very happy to make some bets with Daniel or Eli on some of these (and give them extra time).
Useless is a stretch and I didn’t intend to claim it originally. Sorry if I implied it. I merely think they are overrated, hence the title.
If it is not indiscreet, can you explain what you mean by “intimate partner violence”? Couples fighting due to different views on bets? or more something like bets on couples’ separations triggering the separation? I’m pretty curious what you mean.
There is pretty solid evidence that sports gambling legalisation leads to increased intimate partner violence.
https://cheps.sdsu.edu/_resources/docs/working-papers/cheps-wp-20251001.pdf?utm_source=perplexity
Likely the standard definition, e.g. by WHO
It’s a subset of gender-based violence. See cause area overview, TLYCS’s help women and girls fund, CE charity NOVAH, etc.
ok, but I mean ask about what the relation is of intimate partner violence with prediction markets.
Ah, sorry for misunderstanding. I don’t know.
well, you actually answered what i asked. It was me who didn’t ask what I meant to ask… :-S