With the fast pace of progress in mind, we can think of AI risks as falling into three buckets:
● Short-term risks are those present in current AI systems or that imminently will be present. This includes concerns like privacy, copyright issues, bias and fairness in the model’s outputs, factual accuracy, and the potential to generate misinformation or propaganda.
● Medium-term risks are those we will face in two to three years. In that time period, Anthropic’s projections suggest that AI systems may become much better at science and engineering, to the point where they could be misused to cause large-scale destruction, particularly in the domain of biology. This rapid growth in science and engineering skills could also change the balance of power between nations.
● Long-term risks relate to where AI is ultimately going. At present, most AI systems are passive and merely converse with users, but as AI systems gain more and more autonomy and ability to directly manipulate the external world, we may face increasing challenges in controlling them. There is a spectrum of problems we could face related to this, at the extreme end of which is concerns about whether a sufficiently powerful AI, without appropriate safeguards, could be a threat to humanity as a whole – referred to as existential risk. Left unchecked, highly autonomous, intelligent systems could also be misused or simply make catastrophic mistakes.
Note that there are some concerns, like AI’s effects on employment, that don’t fit neatly in one bucket and probably take on a different form in each time period.
Short-term risks are in the news every day and are certainly important. I expect we’ll have many opportunities to discuss these in this hearing, and much of Anthropic’s research applies immediately to those risks: our constitutional AI principles include attempts to reduce bias, increase factual accuracy, and show respect for privacy, copyright, and child safety. Our red-teaming is designed to reduce a wide range of these risks, and we have also published papers on using AI systems to correct their own biases and mistakes. There are a number of proposals already being considered by the Congress relating to these risks.
The long-term risks might sound like science fiction, but I believe they are at least potentially real. Along with the CEOs of other major AI companies and a number of prominent AI academics (including my co-witnesses Professors Russell and Bengio) I have signed a statement emphasizing that these risks are a challenge humanity should not neglect. Anthropic has developed evaluations designed to measure precursors of these risks and submitted its models to independent evaluators. And our work on interpretability is also designed to someday help with long-term risks. However, the abstract and distant nature of long-term risks makes them hard to approach from a policy perspective: our view is that it may be best to approach them indirectly by addressing more imminent risks that serve as practice for them.
The medium-term risks are where I would most like to draw the subcommittee’s attention. Simply put, a straightforward extrapolation of the pace of progress suggests that, in 2-3 years, AI systems may facilitate extraordinary insights in broad swaths of many science and engineering disciplines. This will cause a revolution in technology and scientific discovery, but also greatly widen the set of people who can wreak havoc. In particular, I am concerned that AI systems could be misused on a grand scale in the domains of cybersecurity, nuclear technology, chemistry, and especially biology.
Thanks for linking Dario’s testimony. I actually found this extract which was closer to answering my question:
I wanted to answer one obvious question up front: if I truly believe that AI’s risks are so severe, why even develop the technology at all? To this I have three answers:
First, if we can mitigate the risks of AI, its benefits will be truly profound. In the next few years it could greatly accelerate treatments for diseases such as cancer, lower the cost of energy, revolutionize education, improve efficiency throughout government, and much more.
Second, relinquishing this technology in the United States would simply hand over its power, risks, and moral dilemmas to adversaries who do not share our values.
Finally, a consistent theme of our research has been that the best mitigations to the risks of powerful AI often also involve powerful AI. In other words, the danger and the solution to the danger are often coupled. Being at the frontier thus puts us in a strong position to develop safety techniques (like those I’ve mentioned above), and also to see ahead and warn about risks, as I’m doing today.
I know this statement would have been massively pre-prepared for the hearing, but I don’t feel super convinced by it:
On his point 1) such benefits have to be weighed up against the harms, both existential and not. But just as many parts of the xRisk story are speculative, so are many of the purported benefits from AI research. I guess Dario is saying ‘it could’ and not it will, but for me if you want to “improve efficiency throughout government” you’ll need political solutions, not technical ones.
Point 2) is the ‘but China’ response to AI Safety. I’m not an expert in US foreign policy strategy (funny how everyone is these days), but I’d note this response only works if you view the path to increasing capability as straightforward. It also doesn’t work, in my mind, if you think there’s a high chance of xRisk. Just because someone else might ignite the atmosphere, doesn’t mean you should too. I’d also note that Dario doesn’t sound nearly as confident making this statement as he did talking to it with Dwarkesh recently.
Point 3) makes sense if you think the value of the benefits massively outweighs the harms, so that you solve the harms as you reap the benefits. But if those harms outweigh the benefits, or you incure a substantial “risk of ruin”, then being at the frontier and expanding it further unilaterally makes less sense to me.
I guess I’d want the CEOs and those with power in these companies to actually be put under the scrutiny in the political sphere which they deserve. These are important and consequential issues we’re talking about, and I just get the vibe that the ‘kid gloves’ need to come off a bit in turns of oversight and scrutiny/scepticism.
See Dario’s Senate testimony from two months ago:
Thanks for linking Dario’s testimony. I actually found this extract which was closer to answering my question:
I know this statement would have been massively pre-prepared for the hearing, but I don’t feel super convinced by it:
On his point 1) such benefits have to be weighed up against the harms, both existential and not. But just as many parts of the xRisk story are speculative, so are many of the purported benefits from AI research. I guess Dario is saying ‘it could’ and not it will, but for me if you want to “improve efficiency throughout government” you’ll need political solutions, not technical ones.
Point 2) is the ‘but China’ response to AI Safety. I’m not an expert in US foreign policy strategy (funny how everyone is these days), but I’d note this response only works if you view the path to increasing capability as straightforward. It also doesn’t work, in my mind, if you think there’s a high chance of xRisk. Just because someone else might ignite the atmosphere, doesn’t mean you should too. I’d also note that Dario doesn’t sound nearly as confident making this statement as he did talking to it with Dwarkesh recently.
Point 3) makes sense if you think the value of the benefits massively outweighs the harms, so that you solve the harms as you reap the benefits. But if those harms outweigh the benefits, or you incure a substantial “risk of ruin”, then being at the frontier and expanding it further unilaterally makes less sense to me.
I guess I’d want the CEOs and those with power in these companies to actually be put under the scrutiny in the political sphere which they deserve. These are important and consequential issues we’re talking about, and I just get the vibe that the ‘kid gloves’ need to come off a bit in turns of oversight and scrutiny/scepticism.
Yeah, I think the real reason is we think we’re safer than OpenAI (and possibly some wanting-power but that mostly doesn’t explain their behavior).