> thesis also seems like the type of thing that should be proven, rather than asserted. It would indeed be damning if OpenAI is taking contradictory positions about numerous important issues, but I don’t think you’ve shown that they are in this post. This post would be stronger if you gave concrete examples.
I agree with this. I’d also prefer that there would be work to track down more of this. I’ve been overall surprised at the response my post had, but from previous comments, I assumed that readers mostly agreed with these claims. I’d like to see more work go into this (I’ll look for some sources, and encourage others to do a better job).
> OpenAI isn’t making anything dangerous and is unlikely to do so in the future (good for public PR and government regulations).
I feel like this is one of the more implicit items listed. It’s true that this is one that I don’t remember them saying explicitly, more in the manner of which they speak. There’s also a question here of what the bar for “dangerous” is. Also, to be clear, I think OpenAI’s is stating “We are working on things that could be dangerous if not handled well, but we are handling them well, so the results of our work won’t be dangerous”, not, “We are working on things that could never be dangerous.”
Here are some predictions I’d make: - If someone were to ask Sam Altman, “Do you think that OpenAI releasing LLMs to the point it has now, has endangered over 100 lives, or has the effects of doing so in the next few years?”, he’d say no. - If someone were to ask Sam Altman, “Do you think that GPT-5 is likely to be a real threat to humanity”, he’d say something like, “This is still too early. If it’s any threat, it’s the potential for things like misinformation, not an existential threat. We’re competent at handling such threats.” - If someone were to ask Sam Altman, “Is there a substantial chance that OpenAI will create something that destroys mankind, or kills 1k+ people, in the next 10 years”, he’ll say, “We are very careful, so the chances are very low of anything like that. However, there could be other competitors....”
Their actions really don’t make it seem, to me, like they think it’s very dangerous. - Main information otherwise is the funding of the alignment team, but that was just disbanded. - Removed the main board members who were publicly concerned about risk. - Very little public discussion of concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).
I’d also flag that such a message (it’s not very dangerous / it will be handled well) seems more common from Microsoft, I believe even when asked about OpenAI.
One quote from Sam I came across recently that might be of interest to you: “What I lose the most sleep over is the hypothetical idea that we already have done something really bad by launching ChatGPT. That maybe there was something hard and complicated in there (the system) that we didn’t understand and have now already kicked it off.”
[OpenAI do] very little public discussion of concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).
Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.
The “Preparedness” page—linked from the top navigation menu on their website—starts:
The study of frontier AI risks has fallen far short of what is possible and where we need to be. To address this gap and systematize our safety thinking, we are adopting the initial version of our Preparedness Framework. It describes OpenAI’s processes to track, evaluate, forecast, and protect against catastrophic risks posed by increasingly powerful models.
The page mentions “cybersecurity, CBRN (chemical, biological, radiological, nuclear threats), persuasion, and model autonomy”. The framework itself goes into more detail, proposing scorecards for assessing risk in each category. They define “catastrophic risk” as “any risk which could result
in hundreds of billions of dollars in economic damage or lead to the severe harm or death of many individuals—this includes, but is not limited to, existential risk”. The phrase “millions of deaths” appears in one of the scorecards.
I agree that there’s a lot of evidence that people at OpenAI have thought that AI could be a major risk, and I think that these are good examples.
I said here, “concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).”
Just looking at the examples you posted, most feel pretty high-level and vague, and not very related to their specific products.
This was a one-sentence statement. It easily sounds to me like saying, “Someone should deal with this, but not exactly us.”
> The framework itself goes into more detail, proposing scorecards for assessing risk in each category.
I think this is a good step, but it seems pretty vague to me. There’s fairly little quantifiable content here, a lot of words like “medium risk” and “high risk”.
From what I can tell, the “teeth” in the document is, “changes get brought up to management, and our board”, which doesn’t fill me with confidence.
Related, I’d be quite surprised if they actually followed through with this much in the next 1-3 years, but I’d be happy to be wrong!
> thesis also seems like the type of thing that should be proven, rather than asserted. It would indeed be damning if OpenAI is taking contradictory positions about numerous important issues, but I don’t think you’ve shown that they are in this post. This post would be stronger if you gave concrete examples.
I agree with this. I’d also prefer that there would be work to track down more of this. I’ve been overall surprised at the response my post had, but from previous comments, I assumed that readers mostly agreed with these claims. I’d like to see more work go into this (I’ll look for some sources, and encourage others to do a better job).
> OpenAI isn’t making anything dangerous and is unlikely to do so in the future (good for public PR and government regulations).
I feel like this is one of the more implicit items listed. It’s true that this is one that I don’t remember them saying explicitly, more in the manner of which they speak. There’s also a question here of what the bar for “dangerous” is. Also, to be clear, I think OpenAI’s is stating “We are working on things that could be dangerous if not handled well, but we are handling them well, so the results of our work won’t be dangerous”, not, “We are working on things that could never be dangerous.”
Here are some predictions I’d make:
- If someone were to ask Sam Altman, “Do you think that OpenAI releasing LLMs to the point it has now, has endangered over 100 lives, or has the effects of doing so in the next few years?”, he’d say no.
- If someone were to ask Sam Altman, “Do you think that GPT-5 is likely to be a real threat to humanity”, he’d say something like, “This is still too early. If it’s any threat, it’s the potential for things like misinformation, not an existential threat. We’re competent at handling such threats.”
- If someone were to ask Sam Altman, “Is there a substantial chance that OpenAI will create something that destroys mankind, or kills 1k+ people, in the next 10 years”, he’ll say, “We are very careful, so the chances are very low of anything like that. However, there could be other competitors....”
Their actions really don’t make it seem, to me, like they think it’s very dangerous.
- Main information otherwise is the funding of the alignment team, but that was just disbanded.
- Removed the main board members who were publicly concerned about risk.
- Very little public discussion of concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).
I’d also flag that such a message (it’s not very dangerous / it will be handled well) seems more common from Microsoft, I believe even when asked about OpenAI.
One quote from Sam I came across recently that might be of interest to you: “What I lose the most sleep over is the hypothetical idea that we already have done something really bad by launching ChatGPT. That maybe there was something hard and complicated in there (the system) that we didn’t understand and have now already kicked it off.”
https://timesofindia.indiatimes.com/business/india-business/et-conversations-with-openai-ceo-sam-altman/amp_liveblog/100822923.cms
You wrote:
This doesn’t match my impression.
For example, Altman signed the CAIS AI Safety Statement, which reads:
The “Preparedness” page—linked from the top navigation menu on their website—starts:
The page mentions “cybersecurity, CBRN (chemical, biological, radiological, nuclear threats), persuasion, and model autonomy”. The framework itself goes into more detail, proposing scorecards for assessing risk in each category. They define “catastrophic risk” as “any risk which could result in hundreds of billions of dollars in economic damage or lead to the severe harm or death of many individuals—this includes, but is not limited to, existential risk”. The phrase “millions of deaths” appears in one of the scorecards.
Their “Planning for AGI & Beyond” blog post describes the risks as “existential”, I quote the relevant passage in another comment.
On their “Safety & Alignment” blog they highlight recent posts called Reimagining secure infrastructure for advanced AI and Building an early warning system for LLM-aided biological threat creation.
My sense is that there are many other examples, but I’ll stop here for now.
I agree that there’s a lot of evidence that people at OpenAI have thought that AI could be a major risk, and I think that these are good examples.
I said here, “concrete/specific large-scale risks of their products and the corresponding risk-mitigation efforts (outside of things like short-term malicious use by bad API actors, where they are doing better work).”
Just looking at the examples you posted, most feel pretty high-level and vague, and not very related to their specific products.
> For example, Altman signed the CAIS AI Safety Statement, which reads...
This was a one-sentence statement. It easily sounds to me like saying, “Someone should deal with this, but not exactly us.”
> The framework itself goes into more detail, proposing scorecards for assessing risk in each category.
I think this is a good step, but it seems pretty vague to me. There’s fairly little quantifiable content here, a lot of words like “medium risk” and “high risk”.
From what I can tell, the “teeth” in the document is, “changes get brought up to management, and our board”, which doesn’t fill me with confidence.
Related, I’d be quite surprised if they actually followed through with this much in the next 1-3 years, but I’d be happy to be wrong!