I both think that some groups in EA are slightly overenthusiastic about forecasting[1] (while other subgroups in EA don’t forecast enough), and that forecasting is underused/undervalued in a lot of the world for a number of different reasons. And I’d also suggest looking at this question less from the lens of “why is EA more into forecasting than other groups?” and more from the lens of “some groups are into forecasting while others aren’t, why is that?”
More specific (but still quick!) notes:
I’d guess reasons for why other groups are less enthusiastic about forecasting can vary significantly.
E.g.:
People in governments and the media (and maybe e.g. academia) might view forecasting as not legible enough, or credentialist enough, and the relevant institutions might be slower to change. I don’t know how much different factors contribute in different cases. There might also be legal issues for some of those groups.
I think some people have argued that in corporate settings, people in power are not interested in setting up forecasting-like mechanisms because it might go against their interests. Alternatively, I’ve heard arguments that middle management systems are messed up, meaning that people who are able to set up forecasting systems in large organizations are promoted and the projects die. I don’t know how true these things are.
I think people who end up in EA are often the kinds of people who’d also be more likely to find forecasting interesting — both for reasons that are somewhat baked into common ~principles in EA, and for other reasons that look more like biases/selection effects.
I’m not sure if you’re implying that EA is one of the most extremely-into-forecasting groups, but I’m not sure that this is true. Relatedly, if you’re looking to see whether/where forecasting is especially useful, looking at the different use cases as a whole (rather than zooming into EA-vs-not-EA as a key division) might be more useful.
E.g. sports (betting), trading, and maybe even the weather might be areas/fields/subcultures that rely a lot on forecasting. Wikipedia lists lots of “applications” in its entry on forecasting, and they include business, politics, energy, etc.
[Edit: this is not as informative as I thought! See Austin’s comment below.] Manifold has groups, and while there’s an EA group, it’s only 2.2K people, while groups like sports (14.4k people) and politics (16.4k people) are bigger. This is more surprising to me since Manifold is particularly EA-aligned.[2]
The forecasting group on Reddit looks dead, but it looks like people in various subreddits are discussing forecasts on machine learning, data science, business, supply chains, politics, etc. (I also think that forecasting might be popular in crypto-enthusiast circles, but I’m not sure.)
This is potentially minor, but given that a lot of prediction-market-like setups with real money are illegal, as far as I understand, the only people who forecast on public platforms are probably those who are motivated by things like reputation, public good benefits, or maybe charity in the case of e.g. Manifold. So you might ask why Wikipedia editors don’t forecast as much as EAs.
You could claim that this is a sign that EA is more into forecasting, but I’m not sure; EA is also more into creating public-good technology, so even if lots of e.g. trading firms were really excited about forecasting, I’d be unsurprised if there weren’t many ~public forecasting platforms that were supported by those groups. (And there are in fact a number of forecasting platforms/aggregators whose relationship to EA is at least looser, unless I’m mistaken, like PredictIt, PolyMarket, Betfair, Kalshi, etc.)
3b. As a clarification, for a period of time we auto-enrolled people in a subset of groups we considered to be broadly appealing (Econ/Tech/Science/Politics/World/Culture/Sports), so those group size metrics are not super indicative of user preferences. We aren’t doing this at this point in time, but did not unenroll those users.
Thanks for these thoughts! I agree with most of what you said. Some replies to specific points:
1b: The post I mentioned discusses this point. I think it’s plausible that that’s a factor, but even if it were a major one, it still doesn’t explain the lack of demand for forecasting consultancies, which could presumably do an even better job at forecasting questions which don’t require company-specific information.
2: This matches my intuitions as well. Though I think it doesn’t say much about whether forecasting is actually useful or not, as this could mean “EAs are more keen to pay the initial (large) fixed cost to learn how to use and integrate this tool”, or “EAs use this tool because they like it, even though it isn’t actually that helpful for decisionmaking”
3: I agree with your point if we’re talking about forecasting in general! I think that all of the actors I mention make extensive use of forecasting. However, in this question, I tried to restrict my attention to Tetlock-style judgmental forecasting, as I mentioned in the first paragraph (this was my bad, I should have been clearer when specifying the question). The fact that these agents do use various forms of forecasting makes the question more intriguing for me, given that judgmental forecasting is very general and seemingly really promising.
Re: 1b (or 1aii because of my love for indenting): That makes sense. I think I agree with you, and I’m generally unsure how much of a factor what I described even is.
Re 2: Yeah, this seems right. I do think some of the selection effects might mean that we should expect that forecasting is less promising than one might think given excitement about them in EA, though?
Re: 3: Thanks for clarifying! I was indeed not narrowing things down to Tetlock-style judgmental forecasting. I agree that it’s interesting that judgement-style forecasting doesn’t seem to get used as much even in fields that do use forecasting (although I don’t know what the most common approaches to forecasting in different fields actually are, so I don’t know how far off they are from Tetlock-style things).
Also, this is mostly an aside (it’s a return to the overall topic, rather than being specific to your reply), but have you seen this report/post?
I both think that some groups in EA are slightly overenthusiastic about forecasting[1] (while other subgroups in EA don’t forecast enough), and that forecasting is underused/undervalued in a lot of the world for a number of different reasons. And I’d also suggest looking at this question less from the lens of “why is EA more into forecasting than other groups?” and more from the lens of “some groups are into forecasting while others aren’t, why is that?”
More specific (but still quick!) notes:
I’d guess reasons for why other groups are less enthusiastic about forecasting can vary significantly.
E.g.:
People in governments and the media (and maybe e.g. academia) might view forecasting as not legible enough, or credentialist enough, and the relevant institutions might be slower to change. I don’t know how much different factors contribute in different cases. There might also be legal issues for some of those groups.
I think some people have argued that in corporate settings, people in power are not interested in setting up forecasting-like mechanisms because it might go against their interests. Alternatively, I’ve heard arguments that middle management systems are messed up, meaning that people who are able to set up forecasting systems in large organizations are promoted and the projects die. I don’t know how true these things are.
I think people who end up in EA are often the kinds of people who’d also be more likely to find forecasting interesting — both for reasons that are somewhat baked into common ~principles in EA, and for other reasons that look more like biases/selection effects.
I’m not sure if you’re implying that EA is one of the most extremely-into-forecasting groups, but I’m not sure that this is true. Relatedly, if you’re looking to see whether/where forecasting is especially useful, looking at the different use cases as a whole (rather than zooming into EA-vs-not-EA as a key division) might be more useful.
E.g. sports (betting), trading, and maybe even the weather might be areas/fields/subcultures that rely a lot on forecasting. Wikipedia lists lots of “applications” in its entry on forecasting, and they include business, politics, energy, etc.
[Edit: this is not as informative as I thought! See Austin’s comment below.] Manifold has groups, and while there’s an EA group, it’s only 2.2K people, while groups like sports (14.4k people) and politics (16.4k people) are bigger. This is more surprising to me since Manifold is particularly EA-aligned.[2]
The forecasting group on Reddit looks dead, but it looks like people in various subreddits are discussing forecasts on machine learning, data science, business, supply chains, politics, etc. (I also think that forecasting might be popular in crypto-enthusiast circles, but I’m not sure.)
This is potentially minor, but given that a lot of prediction-market-like setups with real money are illegal, as far as I understand, the only people who forecast on public platforms are probably those who are motivated by things like reputation, public good benefits, or maybe charity in the case of e.g. Manifold. So you might ask why Wikipedia editors don’t forecast as much as EAs.
And specific aspects of forecasting
You could claim that this is a sign that EA is more into forecasting, but I’m not sure; EA is also more into creating public-good technology, so even if lots of e.g. trading firms were really excited about forecasting, I’d be unsurprised if there weren’t many ~public forecasting platforms that were supported by those groups. (And there are in fact a number of forecasting platforms/aggregators whose relationship to EA is at least looser, unless I’m mistaken, like PredictIt, PolyMarket, Betfair, Kalshi, etc.)
3b. As a clarification, for a period of time we auto-enrolled people in a subset of groups we considered to be broadly appealing (Econ/Tech/Science/Politics/World/Culture/Sports), so those group size metrics are not super indicative of user preferences. We aren’t doing this at this point in time, but did not unenroll those users.
Thanks! This is really useful to know. Edited my comment.
Thanks for these thoughts! I agree with most of what you said. Some replies to specific points:
1b: The post I mentioned discusses this point. I think it’s plausible that that’s a factor, but even if it were a major one, it still doesn’t explain the lack of demand for forecasting consultancies, which could presumably do an even better job at forecasting questions which don’t require company-specific information.
2: This matches my intuitions as well. Though I think it doesn’t say much about whether forecasting is actually useful or not, as this could mean “EAs are more keen to pay the initial (large) fixed cost to learn how to use and integrate this tool”, or “EAs use this tool because they like it, even though it isn’t actually that helpful for decisionmaking”
3: I agree with your point if we’re talking about forecasting in general! I think that all of the actors I mention make extensive use of forecasting. However, in this question, I tried to restrict my attention to Tetlock-style judgmental forecasting, as I mentioned in the first paragraph (this was my bad, I should have been clearer when specifying the question). The fact that these agents do use various forms of forecasting makes the question more intriguing for me, given that judgmental forecasting is very general and seemingly really promising.
Re: 1b (or 1aii because of my love for indenting): That makes sense. I think I agree with you, and I’m generally unsure how much of a factor what I described even is.
Re 2: Yeah, this seems right. I do think some of the selection effects might mean that we should expect that forecasting is less promising than one might think given excitement about them in EA, though?
Re: 3: Thanks for clarifying! I was indeed not narrowing things down to Tetlock-style judgmental forecasting. I agree that it’s interesting that judgement-style forecasting doesn’t seem to get used as much even in fields that do use forecasting (although I don’t know what the most common approaches to forecasting in different fields actually are, so I don’t know how far off they are from Tetlock-style things).
Also, this is mostly an aside (it’s a return to the overall topic, rather than being specific to your reply), but have you seen this report/post?