I see forecasting catching on with researchers for experimental design, which could easily save a lot of money and help make more progress. Earlier this month we updated a working paper on forecasts using data from the Social Science Prediction Platform to explicitly include results demonstrating use in power calculations. If a year from now forecasts are used a similar amount to now in economics research then that would be evidence for your hypothesis but from my perspective the concept that forecasts could be used in this way has only just started to be socialized, at least in my field. I also personally know of at least a couple of large institutions seeking forecasts and am planning a RCT on how they affect decision-making in the field.
I think LLMs are making forecasting much cheaper and easier.
If humans don’t take up the use of forecasts in decision-making as much as they “should”, well, LLMs may be more likely to in their own pipelines.
That’s not to say that every project previously funded around forecasting was a good use of money. I would probably agree with you regarding most of the projects you have in mind, while disagreeing with the title and framing which is way too broad.
Since you’re the only one mentioning LLMs I thought I’d ask a couple of things I’d love to understand. Maybe you can point me somewhere to learn more.
Forecasting is supposed to help us prepare for the future. I just learnt that LLMs seem to be increasingly be used for forecasting—turns out that they’re surprisingly good at it. And LLMs-for-forecasting are used for AI safety work.
This sounds to me very close to assigning a suspected criminal to guide the team of investigators that are investigating the suspected crime.
Furthermore, I hear that this is admitted to be risky some years in the future. While also admitting that we don’t know how the current AI is already so surprisingly good at it. Why isn’t it considered to be risky right now?
Even further, I hear that the reason LLMs are used is because there’s not enough humans to work on this. But given the circularity of these things, I’ll bet that the use of LLMs is also making it difficult for new humans to grow into the job. So this would paint a future of better LLMs forecasting for less humans. Is this so?
In all, this sounds to me like a doom machine that will only fail if forecasting turns out by sheer luck to be useless and/or LLMs turn out by sheer luck to go nowhere.
Or in other words, AI safety work with a good chance of making the problem worse by spawning a double agent.
I have at least three reasons to be hopeful:
I see forecasting catching on with researchers for experimental design, which could easily save a lot of money and help make more progress. Earlier this month we updated a working paper on forecasts using data from the Social Science Prediction Platform to explicitly include results demonstrating use in power calculations. If a year from now forecasts are used a similar amount to now in economics research then that would be evidence for your hypothesis but from my perspective the concept that forecasts could be used in this way has only just started to be socialized, at least in my field. I also personally know of at least a couple of large institutions seeking forecasts and am planning a RCT on how they affect decision-making in the field.
I think LLMs are making forecasting much cheaper and easier.
If humans don’t take up the use of forecasts in decision-making as much as they “should”, well, LLMs may be more likely to in their own pipelines.
That’s not to say that every project previously funded around forecasting was a good use of money. I would probably agree with you regarding most of the projects you have in mind, while disagreeing with the title and framing which is way too broad.
Since you’re the only one mentioning LLMs I thought I’d ask a couple of things I’d love to understand. Maybe you can point me somewhere to learn more.
Forecasting is supposed to help us prepare for the future. I just learnt that LLMs seem to be increasingly be used for forecasting—turns out that they’re surprisingly good at it. And LLMs-for-forecasting are used for AI safety work.
This sounds to me very close to assigning a suspected criminal to guide the team of investigators that are investigating the suspected crime.
Furthermore, I hear that this is admitted to be risky some years in the future. While also admitting that we don’t know how the current AI is already so surprisingly good at it. Why isn’t it considered to be risky right now?
Even further, I hear that the reason LLMs are used is because there’s not enough humans to work on this. But given the circularity of these things, I’ll bet that the use of LLMs is also making it difficult for new humans to grow into the job. So this would paint a future of better LLMs forecasting for less humans. Is this so?
In all, this sounds to me like a doom machine that will only fail if forecasting turns out by sheer luck to be useless and/or LLMs turn out by sheer luck to go nowhere.
Or in other words, AI safety work with a good chance of making the problem worse by spawning a double agent.
What am I getting wrong? Where can I learn more?