Would you count Holden’s take here as a robust case for funding forecasting as an effective use of charitable funds?
It’s not controversial to say a highly general AI system, such as PASTA, would be momentous. The question is, when (if ever) will such a thing exist?
Over the last few years, a team at Open Philanthropy has investigated this question from multiple angles.
One forecasting method observes that:
No AI model to date has been even 1% as “big” (in terms of computations performed) as a human brain, and until recently this wouldn’t have been affordable—but that will change relatively soon.
And by the end of this century, it will be affordable to train enormous AI models many times over; to train human-brain-sized models on enormously difficult, expensive tasks; and even perhaps to perform as many computations as have been done “by evolution” (by all animal brains in history to date).
This method’s predictions are in line with the latest survey of AI researchers: something like PASTA is more likely than not this century.
A number of other angles have been examined as well.
One challenge for these forecasts: there’s no “field of AI forecasting” and no expert consensus comparable to the one around climate change.
It’s hard to be confident when the discussions around these topics are small and limited. But I think we should take the “most important century” hypothesis seriously based on what we know now, until and unless a “field of AI forecasting” develops.
Actually, maybe it’s also useful to just look at the biggest grants from that list:
$7,993,780 over two years to the Applied Research Laboratory for Intelligence and Security at the University of Maryland, to support the development of two forecasting platforms, in a project led by Dr. Adam Russell. The forecasting platforms will be provided as a resource to help answer questions for policymakers (writeup)
two grants totaling $6,305,675 over three years to support the Forecasting Research Institute (FRI)’s work on projects to advance the science of forecasting as a tool to improve public policy and reduce existential risk. This includes developing a new modular forecasting platform and conducting research to test different forecasting techniques. This follows our October 2021 support ($275,000) for planning work by FRI Chief Scientist Philip Tetlock, and falls within our work on global catastrophic risks (writeup)
$3,000,000 to Metaculus to support work to improve its online forecasting platform, which allows forecasters to make predictions about world events. We believe that this work will help to provide more accurate and calibrated forecasts in domains relevant to Open Philanthropy’s work, such as artificial intelligence and biosecurity and pandemic preparedness, and enable organizations and individuals working in those areas to make better decisions. This follows our May 2022 support ($5,500,000) and falls within our work on global catastrophic risks (writeup)
Thanks for sharing. It’s a start, but it’s certainly not a proven Theory of Change. For example, Tetlock himself said that nebulous long-term forecasts are hard to do because there’s no feedback loop. Hence, a prediction market on an existential risk will be inherently flawed.
I don’t think that really works. You can get feedback from 5 years in 5 years. Metaculus already has some suggestions as to people who are good 5 year forecasters.
Would you count Holden’s take here as a robust case for funding forecasting as an effective use of charitable funds?
This is my own (possibly very naive) interpretation of one motivation behind some of Open Phil’s forecasting-related grants.
Actually, maybe it’s also useful to just look at the biggest grants from that list:
Thanks for sharing. It’s a start, but it’s certainly not a proven Theory of Change. For example, Tetlock himself said that nebulous long-term forecasts are hard to do because there’s no feedback loop. Hence, a prediction market on an existential risk will be inherently flawed.
I don’t think that really works. You can get feedback from 5 years in 5 years. Metaculus already has some suggestions as to people who are good 5 year forecasters.
None of the above are prediction markets.