Not Josh, and also conflicted through the Social Science Prediction Platform (though we had pretty minimal funding from EA sources), but I wonder if it would be worth pooling non-public projects we know of and making BOTE estimates of hypothetical impact. It’s tricky because I don’t know of any RCTs (though I’m working on one now). But I’m extremely confident that across us we would think of some combination of orgs/governments that collectively spend over $100 billion per year (… I can think of that alone) that are interested in forecasts in different ways. Now, imo the vast majority of places interested in forecasting are not going to do anything substantive with it, and it’s hard to know what it means for one of these places to integrate forecasts—for example, for an org spending $X, do forecasts inform 1% of their funding or what? Of the share they inform, how much do they move the needle? If estimates from people who work on forecasts may be optimistic (I’m not paid at all for it, but I choose to work on it because I think it’s useful), happy to describe the situation to an outside observer privately.
I think the Social Science Prediction Platform (alongside a friend of mine who is doing something similar for clinical trials) are among the more interesting uses of forecasting/PMs but I’m skeptical they will be uptaken to the degree/impact you might hope for.
do forecasts inform 1% of their funding or what?
I’m skeptical of things of the form “small percentage chance * big number”. I think humans are really bad at estimating small percentages.
Would be happy to talk privately about any situations you are thinking of.
Thanks! I agree, I’m also generally skeptical of small chance * big number things—I was not intending 1% as an anchor but as an open question—and not as a probability but as a concrete percent of the funding. For example, a big org uses forecasts, but perhaps they only use them in particular workstreams responsible for X% of funding, and those workstreams could be tracked. Then out of X%, how much do they move the needle?
Not Josh, and also conflicted through the Social Science Prediction Platform (though we had pretty minimal funding from EA sources), but I wonder if it would be worth pooling non-public projects we know of and making BOTE estimates of hypothetical impact. It’s tricky because I don’t know of any RCTs (though I’m working on one now). But I’m extremely confident that across us we would think of some combination of orgs/governments that collectively spend over $100 billion per year (… I can think of that alone) that are interested in forecasts in different ways. Now, imo the vast majority of places interested in forecasting are not going to do anything substantive with it, and it’s hard to know what it means for one of these places to integrate forecasts—for example, for an org spending $X, do forecasts inform 1% of their funding or what? Of the share they inform, how much do they move the needle? If estimates from people who work on forecasts may be optimistic (I’m not paid at all for it, but I choose to work on it because I think it’s useful), happy to describe the situation to an outside observer privately.
Hi Eva,
I think the Social Science Prediction Platform (alongside a friend of mine who is doing something similar for clinical trials) are among the more interesting uses of forecasting/PMs but I’m skeptical they will be uptaken to the degree/impact you might hope for.
I’m skeptical of things of the form “small percentage chance * big number”. I think humans are really bad at estimating small percentages.
Would be happy to talk privately about any situations you are thinking of.
Thanks! I agree, I’m also generally skeptical of small chance * big number things—I was not intending 1% as an anchor but as an open question—and not as a probability but as a concrete percent of the funding. For example, a big org uses forecasts, but perhaps they only use them in particular workstreams responsible for X% of funding, and those workstreams could be tracked. Then out of X%, how much do they move the needle?
Anyway, happy to chat sometime!