[OpenAI do] very little public discussion of concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).
Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.
The “Preparedness” page—linked from the top navigation menu on their website—starts:
The study of frontier AI risks has fallen far short of what is possible and where we need to be. To address this gap and systematize our safety thinking, we are adopting the initial version of our Preparedness Framework. It describes OpenAI’s processes to track, evaluate, forecast, and protect against catastrophic risks posed by increasingly powerful models.
The page mentions “cybersecurity, CBRN (chemical, biological, radiological, nuclear threats), persuasion, and model autonomy”. The framework itself goes into more detail, proposing scorecards for assessing risk in each category. They define “catastrophic risk” as “any risk which could result
in hundreds of billions of dollars in economic damage or lead to the severe harm or death of many individuals—this includes, but is not limited to, existential risk”. The phrase “millions of deaths” appears in one of the scorecards.
I agree that there’s a lot of evidence that people at OpenAI have thought that AI could be a major risk, and I think that these are good examples.
I said here, “concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).”
Just looking at the examples you posted, most feel pretty high-level and vague, and not very related to their specific products.
This was a one-sentence statement. It easily sounds to me like saying, “Someone should deal with this, but not exactly us.”
> The framework itself goes into more detail, proposing scorecards for assessing risk in each category.
I think this is a good step, but it seems pretty vague to me. There’s fairly little quantifiable content here, a lot of words like “medium risk” and “high risk”.
From what I can tell, the “teeth” in the document is, “changes get brought up to management, and our board”, which doesn’t fill me with confidence.
Related, I’d be quite surprised if they actually followed through with this much in the next 1-3 years, but I’d be happy to be wrong!
You wrote:
This doesn’t match my impression.
For example, Altman signed the CAIS AI Safety Statement, which reads:
The “Preparedness” page—linked from the top navigation menu on their website—starts:
The page mentions “cybersecurity, CBRN (chemical, biological, radiological, nuclear threats), persuasion, and model autonomy”. The framework itself goes into more detail, proposing scorecards for assessing risk in each category. They define “catastrophic risk” as “any risk which could result in hundreds of billions of dollars in economic damage or lead to the severe harm or death of many individuals—this includes, but is not limited to, existential risk”. The phrase “millions of deaths” appears in one of the scorecards.
Their “Planning for AGI & Beyond” blog post describes the risks as “existential”, I quote the relevant passage in another comment.
On their “Safety & Alignment” blog they highlight recent posts called Reimagining secure infrastructure for advanced AI and Building an early warning system for LLM-aided biological threat creation.
My sense is that there are many other examples, but I’ll stop here for now.
I agree that there’s a lot of evidence that people at OpenAI have thought that AI could be a major risk, and I think that these are good examples.
I said here, “concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).”
Just looking at the examples you posted, most feel pretty high-level and vague, and not very related to their specific products.
> For example, Altman signed the CAIS AI Safety Statement, which reads...
This was a one-sentence statement. It easily sounds to me like saying, “Someone should deal with this, but not exactly us.”
> The framework itself goes into more detail, proposing scorecards for assessing risk in each category.
I think this is a good step, but it seems pretty vague to me. There’s fairly little quantifiable content here, a lot of words like “medium risk” and “high risk”.
From what I can tell, the “teeth” in the document is, “changes get brought up to management, and our board”, which doesn’t fill me with confidence.
Related, I’d be quite surprised if they actually followed through with this much in the next 1-3 years, but I’d be happy to be wrong!