Some underrated reasons why the AI safety community should reconsider its embrace of strict liability
Introduction
It is by now a well-known fact that existing AI systems are already causing harms like discrimination, and it’s also widely expected that the advanced AI systems which the likes of Meta and OpenAI are building could also cause significant harms in the future. Knowingly or unknowingly, innocent people have to live with the dire impacts of these systems. Today that might be a lack of equal access to certain opportunities or the distortion of democracy but in future it might escalate to more concerning security threats. In light of this, it should be uncontroversial for anyone to insist that we need to establish fair and practically sensible ways of figuring out who should be held liable for AI harms. The good news is that a number of AI safety experts have been making suggestions. The not-so-good news is that the idea of strict liability for highly capable advanced AI systems still has many devotees.
The most common anti-strict liability argument out there is that it discourages innovation. In this piece, we won’t discuss that position much because it’s already received outsize attention. Instead, we argue that the pro-strict liability argument should be reconsidered for the following trifecta of reasons: (i) In the context of highly capable advanced AI, both strict criminal liability and strict civil liability have fatal gaps, (ii) The argument for strict liability often rests on faulty analogies and (iii) Given the interests at play, strict liability will struggle to gain traction. Finally, we propose that AI safety-oriented researchers working on liability should instead focus on the most inescapably important task–figuring out how to transform good safety ideas into real legal duties.
AI safety researchers have been pushing for strict liability for certain AI harms
The few AI safety researchers who’ve tackled the question of liability in-depth seem to have taken a pro-strict liability for certain AI harms, especially harms that are a result of highly capable advanced AI. Let’s consider some examples. In a statement to the US Senate, the Legal Priorities Project recommended that AI developers and deployers be held strictly liable if their technology is used in attacks on critical infrastructure or a range of high-risk weapons that result in harm. LPP also recommended strict liability for malicious use of exfiltrated systems and open-sourced weights. Consider as well the Future of Life Institute’s feedback to the European Commission, where it calls for a strict liability regime for harms that result from high-risk and general purpose AI systems. Finally, consider Gabriel Weil’s research on the promise that tort law has for regulating highly capable advanced AI (also summarized in his recent EA Forum piece) where he notes the difficulty of proving negligence in AI harm scenarios and then argues that strict liability can be a sufficient corrective for especially dangerous AI.
The pro-strict liability argument
In the realm of AI safety, arguments for strict liability generally rest on two broad lines of reasoning. The first is that historically, strict liability has been applied for other phenomena that are somewhat similar to highly capable advanced AI, which means that it would be appropriate to apply the same regime to highly capable advanced AI. Some common examples of these phenomena include new technologies like trains and motor vehicles, activities which may cause significant harm such as the use of nuclear power and the release of hazardous chemicals into the environment and the so-called ‘abnormally dangerous activities’ such as blasting with dynamite.
The second line of reasoning is that fault-based liability (the alternative to strict liability) has weaknesses that profoundly undermine its usefulness in our attempts to regulate highly capable advanced AI. One frequently cited weakness is the idea that legislators and judges don’t have the necessary technological and risk knowledge to establish the sorts of duties that any fault-based liability regime must outline. Another is the idea that the unpredictability, opacity and autonomy of advanced AI makes it difficult for anyone to prove fault and causation, something that fault-based liability systems usually require.
AI safetyists shouldn’t be in a rush to embrace strict liability
We present three arguments to challenge any embrace of strict liability.
Argument 1
In the context of AI development, both strict criminal liability and strict civil liability have fatal gaps
For the good of readers without a background in law, it’s important to begin by clarifying that strict liability exists in two varieties: strict criminal liability and strict civil liability. The main factor that differentiates the two is the type of penalty imposed after liability is confirmed. If strict criminal liability is in play, the penalty may be jail time (although fines or other non-custodial sentences are also regularly imposed). In situations where strict civil liability is proved, the penalty is normally a non-custodial sentence, which is often the payment of damages. Unsurprisingly, AI safety researchers seem preoccupied with strict civil liability. However, it’s possible that some may be open to endorsing strict criminal liability as well.
There are two central cases that AI safetyists are typically worried about. The first is a situation in which a malicious actor uses highly capable AI in a manner that results in significant harm. The second is a situation in which some highly capable AI that developers and users honestly think is safe or fine turns out not to be as safe as they imagined. The difference is obvious: In the first scenario there is malicious intent while in the second one there is good-faith decision making. We think that strict criminal liability is inappropriate for either case. In the first central case, any malicious users of highly capable AI can be prosecuted under ordinary criminal law since their malevolent intent can be proved (no wonder we refer to them as “malicious users″) so it would in fact be laughably illogical to even consider strict criminal liability for such a case.
Strict criminal liability can be used in the second central case but we think that, as some scholars have argued, it would be unreasonable for a legal system to straight-up impose criminal sanctions, fault be damned, in a situation like that. It would be unreasonable particularly because any AI cycle has a number of actors (for example the compute provider, model developer, system deployer and user) and harm could be the result of an oversight by any of them–or, in fact, none of them. Consequently, strict criminal liability is in principle an excessively onerous regime to use for such a case. In any event, we think the adoption of strict criminal liability will almost certainly be a non-starter, as we will discuss further in our third argument.
As stated earlier, AI safety researchers are more fixated on strict civil liability so let’s now turn our attention to that. The focus on strict civil liability can probably be explained by the fact that it is less onerous (typically no jail term when liability is established) and therefore the penalty (usually the payment of damages) is more defensible. There is also a belief that strict civil liability can have a deterrent effect or can–at the very least–motivate more caution and care.
While it is true that having to pay damages seems less onerous than the possibility of jail time, that doesn’t necessarily mean that it’s fair to apply strict civil liability for highly capable advanced AI. Given that there is still no way for model developers to deterministically guarantee a model’s expected behavior to downstream actors, and given the benefits that advanced AI could have in society, we think it is unfair for an actor to be forced to pay damages regardless of any steps they’ve taken to ensure the advanced AI in question is safe. In line with this, we think there is an important lesson to be located in the architecture of discrimination law across the world. In most jurisdictions, anyone accused of indirect discrimination (where the impact of some practice is discriminatory but there is no discriminatory intent) is always allowed to offer justification before the question of their liability is decided. Given that discrimination is universally considered a very serious issue, we think it is instructive that so many societies consider it wrong to hold anyone strictly liable for discrimination if the necessary intent cannot be shown. We think the same should be true for harms caused by AI.
Beyond that, we also think that AI safetyists may be inflating the promise of strict civil liability. First, let’s dispose of an idea that could be motivating some AI safetyists’ embrace of strict civil liability–That strict civil liability will induce companies to stop doing the things in question, like the development and deployment of highly capable advanced AI. The truth is that in reality strict civil liability does not deliver on that goal, especially when the thing in question might be very lucrative (as is the case with AI development). We’ve not heard or read of any situation where a company stopped doing something because it was concerned about the threat of strict civil liability. That said, there is research which shows that strict civil liability can motivate companies to build more safe products. So we concede that strict civil liability could motivate AI developers and deployers to be more careful. However, clarifying the duties and faults of AI developers through a fault-based liability system can also motivate safety and caution, so what is the point of preferring strict civil liability? A formidable riposte to this point might be the claim that fault-based liability incentivises developers to merely meet the set duty of care while strict civil liability incentivises them to take the greatest caution possible. Still, we think that setting very high duties would likewise force developers to take the greatest caution possible. Fair point, pro-strict liability AI safetyists will say, but setting duties of care and proving fault and causation in the context of AI is very difficult. Besides, legislators, lawyers and judges are not technically equipped to be engaging in such an enterprise.
We must confess that when we encountered this argument we found it strange. It ignores the fact that legal professionals have in the past been able to develop workable legal duties and standards for some highly technical issues. For example, American courts have been able to figure out a case with complex engineering evidence on the safety of a chainsaw and another case featuring complex medical and actuarial evidence. Furthermore, when done correctly, the task of creating legal duties and standards in highly technical fields usually involves experts who possess the appropriate technical knowledge, so what’s the big worry? One could respond by claiming that proving the violation of any established duties will still be extremely difficult so we should just endorse strict civil liability. However, that concern can be addressed by tinkering with evidence discovery and disclosure rules (as the Draft EU Liability Directive in fact does) or adopting a regime of fault-based liability with a reverse burden of proof. It doesn’t necessitate the adoption of strict civil liability.
Even if strict civil liability is taken up for highly capable advanced AI, it seems likely that judges will interpret the law such that fault will still need to be proved in many circumstances, recreating a fault or wrongs-based system in all but word. Consider for example US product liability law. Even though strict liability is apparently the regime that ought to apply there, a claimant is required to first prove that the product causing harm is defective. Essentially, proving defectiveness is proving fault. As some scholars have claimed, this shows that “tort law is filled with strict liability wrongs” meaning that it’s still often the case that a wrong has to be proved for strict civil liability to apply. In our view, AI safetyists might as well save time and focus on figuring out the kinds of faults or wrongs that should trigger liability.
Of course, the strict civil liability that allows absolutely no way out for a defendant once harm is proved exists. US law, for example, applies that kind of strict civil liability for the so-called ‘abnormally dangerous activities’. However, we think highly capable advanced AI is radically different from abnormally dangerous activities and so it’s hard to defend the use of a similar liability standard in its regulation. We’ll discuss this further in our second argument.
Argument 2
The analogies used to justify strict liability are flawed in significant respects
As we showed earlier on, the pro-strict liability argument is substantially based on analogies. In effect, AI safety-oriented advocates of strict liability select examples of phenomena governed using strict liability and then explicitly or implicitly draw the implication that highly capable advanced AI has some kind of important similarity to the phenomena and should therefore also be governed using strict liability. This argumentative technique is only compelling if the phenomena being compared are sufficiently similar. We don’t think this standard has been met.
The common comparator phenomena include trains, motor vehicles, pipelines, nuclear power, the handling of industrial chemicals and the so-called abnormally dangerous activities (which has been defined more specifically in US law).
Let’s start with the comparisons made with trains, motor vehicles, nuclear power and the handling of industrial chemicals. We’re unconvinced by these comparisons because of a methodological flaw. None of the comparisons begins by reasoning out how the features whose similarity is enough to make an analogy accurate are selected. Is it the primary, most elemental features of the things being compared? Is it the secondary features created through human interaction with the primary features? Is it the primary and secondary features as understood by a certain cadre of experts or is it the primary and secondary features as understood by the general public? To counteract accusations of starting with a conclusion and then scrambling for evidence to support it, it’s critical for this methodology to be clarified.
It may be useful to look at this in a more practical way. Autonomy seems like a primary feature of the highly capable advanced AI currently being built. None of the comparators typically used shares this feature. Surely, that should matter in any analysis. And although we think that highly capable advanced AI carries significant global catastrophic risk, there is no consensus on this among experts or the public at large–while there is consensus on the sorts of risks posed by the comparators listed above. Again, shouldn’t this matter in any analogizing? And so on and so forth.
The other common analogy is with “abnormally dangerous activities” as understood in the US. For this one, we’re in luck since US law has been giving meaning to the term for years now. And the truth is, as even Gabriel Weil accepts, it’s super unlikely that the training and deployment of highly capable advanced AI would fit the longstanding legal definition of an “abnormally dangerous activity”. In making our case here we’ll stick to the key features that we think the training and deployment of highly capable AI lacks relative to the activities that have typically been understood as abnormally dangerous.
To begin with, when you consider the kinds of activities that have been found to be abnormally dangerous in the past (some common ones, for example, are blasting by dynamite or keeping wild animals), you immediately realize that (i) the risks have already materialized in reality and (ii) there is consensus in popular and expert perception of the risks. In the words of some scholars, the activities are not just abnormally dangerous, they are “conspicuously so.” Indeed, it turns out that the common law doctrine from which the concept of abnormally dangerous activities flowed was developed in Victorian England by judges who had experienced the materialization of the risks in question. Other than that, the theoretical basis for courts’ continued recognition of strict liability for abnormally dangerous activities is the idea that it makes sense because the actors in question impose substantial risk in a course of conduct typically undertaken for their own benefit. Unfortunately, the training and deployment of highly capable AI just doesn’t meet these metrics. We’re yet to see any global catastrophic risk from the training and deployment of highly capable AI, and there is no consensus among experts or the general public about the GCR risk posed. Finally, highly capable AI could have enormous societal benefits beyond profiteering for the companies. The training and development of highly capable AI can therefore only sit awkwardly next to abnormally dangerous activities.
Argument 3
Given the interests at play, strict liability will struggle to gain traction
To succeed, AI safety advocates can’t just push for the strictest regulation possible. Even if we’re convinced that the strictest regulation is the most fitting, there are other strategic reasons why such a push might be bound to fail. In light of this, we think it’s important to push for proposals that have more than a 40 pc chance of success based on the realities around us. We estimate that strict liability has less than a 15 pc chance of success because of the economic and national security pressures in the world and because there is still no consensus that highly capable advanced AI poses a sky-high level of risk.
We all know by now that countries like the US and China are in a rush to become global leaders in AI development and use due to the economic and military edge that advanced AI could give them. This ambition is destined to shape their AI regulation decisions. Countries like the US and the UK, for example, have decided to actively pursue a pro-innovation approach that seeks “to mitigate the risks that AI poses without inhibiting AI development and use.” These two countries have emphasized that any regulations put in place–including liability rules–should not inhibit AI development. No surprise then that the UK has explicitly noted its concern that allocating too much responsibility to developers might discourage them from building AI. We expect this view to undergird a hostile climate for any strict liability proposals. Unless some kind of harmful events occur as a result of the development and use of highly capable AI, the chances of strict liability being taken up are very low.
There is already evidence to confirm our suspicions. Despite recommendations from the European Parliament, EU citizens, academic institutions and consumer organisations, the European Commission’s Draft AI Liability Directive still doesn’t adopt strict liability for harms caused by high-risk systems. Instead, it proposes a lightly adjusted system of fault-based liability. The fact that a bureaucracy frequently chided for being too pro-regulation did not take strict liability surely says something about how long it might remain in the doldrums.
The other reason that strict liability is unlikely to gain traction is the fact that there’s still no expert consensus on how high the risk posed by highly capable advanced AI is. Not only are some experts unconvinced that AI poses a GCR-level risk, some regulators also remain fairly unconvinced. One of the reasons why the European Commission rejected strict liability is because it did not buy the super high risk argument. The consequences that this absence of consensus should have on AI safety strategies has been written about before. In the context of liability proposals, we think AI safety advocates would be better off trying to map out the specific faults that ought to lead to liability because such proposals have a better chance of success (and will also achieve our goals), especially if they also respond to near-term harms. At the end of the day, we must ask ourselves whether we’re more likely to reach the other side by pushing hard against an immovable wall or by finding the most vulnerable part of the wall and then pushing there. Your guess is as good as ours.
Final point
We’re aware that to some, our piece may seem like an overzealous screed that doesn’t accurately represent what’s going on in AI safety-oriented liability research. To be clear, we aren’t saying that AI safetyists are not trying to figure out the specific duties and faults that ought to trigger liability for the harms caused by highly capable advanced AI. Indeed, LPP’s statement to the US Senate contained a number of recommendations about duties that need to be legislated. We’re saying that in AI safety circles there’s a dalliance with strict liability that isn’t well thought out, and it should be abandoned in favor of more work on how best to design legal duties and standards out of the great ideas being proposed in the community. In other words, let’s invest our time in figuring out what kinds of faults are sufficiently bad, and how they should be framed in law.
When subjected to careful scrutiny, pro-strict liability arguments in AI safety appear to be insufficiently substantiated, unreasonable, unfair or unrealistic. This unnecessarily weakens our chances of gaining broad support at a time when that matters a lot. Because the stakes are so high, we should do something while we still can. As for the question of which liability regime ought to be adopted, we generally think that in scenarios where clear malicious intent can’t be proved, a fault-based system with a reverse burden of proof (reverse insofar as it will be upon the sued party to disprove liability) is the best option. The system is already applicable when airlines are sued for accidents and we think it offers great promise when it comes to governing the development and use of highly capable advanced AI.
Thanks for posting this. I think this is the kind of practical, actionable analysis that we need.
Regarding this:
It seems to me that this is begging the question. If we don’t know how to make AIs safe, that is a reason not to make AIs at all, not a reason to make unsafe AIs. This is not really any different from how the nuclear power industry has been regulated out of existence in some countries[1].
I think this analogy holds regardless of your opinions about the actual dangerousness of nuclear power.
Responding to the same quotation, I don’t think there is any way to structure a system for dealing with tort-like behavior that isn’t “unfair” to someone. If activity X imposes costs on third parties, our options are:
Impose those costs on the person performing activity X (strict liability);
Impose those costs on the third parties (immunity);
Impose those costs on the person performing activity X only if at fault, and on the third parties otherwise;
Require third parties to carry insurance to cover the harms (an indirect way of imposing costs on them);
Have the government pay for the harms (which makes everyone in society pay).
Each of these options is likely to be “unfair” in some applications, as someone is going to have to bear harms out of proportion to their responsibility in creating the harm. To credit the argument, I think we have to conclude that it is worse to be “unfair” to the AI companies than to the third parties.
Yes yes-I think the point we wanted to put across is what you say when you say “to credit the argument”. Strict liability here would be “unreasonably unfair” insofar as it doesn’t consider circumstances before imposing liability. I think it’s fine for a legal regime to be “unfair” to a party (for the reasons you’ve outlined) where there’s some kind of good-enough rationale. Fault-based liability would require the consideration of circumstances first.
As I noted in another comment, I think there is a set of cases—those with low-to-moderate harm amounts—for which the realistic options are strict liability and de facto immunity. At least for that class, I find no liability to be more “unreasonably unfair” than strict liability.
Whether a fault-based liability system is viable (or instead veers close to a “no liability” approach) for other sets of cases remains an open question for me, although I remain skeptical at this time. At least the US legal system has a poor track record of managing harms from rapidly-emerging technologies in the short to medium run, so I’d need solid evidence that it will be different with this rapidly-emerging technology.
Yeah this is sensible. But I’m still hopeful that work like Deepmind’s recent research or Clymer et al’s recent work can help us create duties for a fault-based system that can actually not lead to a de-facto zero liability regime. Worth remembering that the standard of proof will not be perfection: So long as a judge is more convinced than not, liability would be established.
Thanks for the references. The liability system needs to cover AI harms that are not catastrophes, including the stuff that goes by “AI ethics” more than “AI safety.” Indeed, those are the kinds of harms that are likely more legible to the public and will drive public support for liability rules.
(In the US system, that will often be a jury of laypersons deciding any proof issues, by the way. In the federal system at least, that rule has a constitutional basis and isn’t changeable by ordinary legislation.)
Thanks Ian. Yes, fair point. Assuming this suggests that a comparison with nuclear power makes sense, I would say: partially. I think there’s a need to justify why that’s the comparative feature that matters most given that there are other features (for example, potential benefits to humanity at large) that might lead us to conclude that the two aren’t necessarily comparable.
To the extent your proposed approach analogizes to aviation law, the nature of harms in that field strike me as different in a way that is practically relevant. In aviation safety, the harms tend to be very large—when planes crash, people die or are seriously injured, usually multiple people. That means there is usually enough money at play to incentivize litigants and lawyers to put up the resources to fight it out over fault/liability in a complex field.
In contrast, while the AI harms people on this Forum worry about are even more catastrophic than plane crashes, most AI harms will be of a rather more mundane sort. Imagine that an AI somehow causes me to break my leg, incur medical bills, and lose a few weeks of income with damages in the range of $50K-$100K. Even with the benefit of a reverse burden of proof, it’s not reasonable to expect me to litigate the fault of a complex AI system for those stakes. Few people would sue the AI companies for non-catastrophic harms, and so they would not bear the full costs of their negligent acts.
If you limit an airplane to the ground, you ~have an automobile—and I think that may be a better metaphor for most AI-harm litigation. In my view, the correct system for managing most auto-related harms is no-fault insurance (with a fault-based supplemental system for extremely high damages). There’s no expensive litigation over fault or negligence, which would consume a great portion of any potential recovery for most harms. You can see no-fault as a form of strict liability (albeit handled on a first-party insurance basis).
I think other rationales support that system as well here. You seem to have much more faith in lawmakers and regulators to adapt to rapidly changing technologies than I do. Coming up with regulations that fairly cover all uses of AI until the next time the regulations are updated will prove practically impossible. The common-law method of making law through precedent is very slow and not up to the task. Even worse, from a US perspective, many of these issues will be fact questions for a jury of laypersons to decide (and this is constitutionally required to some extent!)
To be sure, no-fault auto insurance requires all drivers to share the cost of compensating injured parties. That’s not feasible in the AI world, where most potential injured parties are more like pedestrians than fellow drivers. But it doesn’t seem unreasonable to me to expect AI users to carry insurance that compensates harmed persons on a no-fault basis—at least for harms too small to expect the injured party to litigate potential fault. I’d probably prefer it be held primarily by the end user of the AI (e.g., a company using AI to make decisions or take actions). Let the market decide which uses of AI are net positive for society and should move forward. If it’s not insurable, you probably shouldn’t be doing it.
An alternative way to deal with the disincentive effect would be awarding at least partial attorney’s fees and full litigation costs for plaintiffs who have a plausible basis for suing the AI company, win or lose (and full fees for those who prevail). That would be a bonanza for people in my profession, but might be worse for the AI companies than strict liability!
P.S. If you’re going to go with a fault-based system, you absolutely have to repeal arbitration laws to the extent that they restrict the availability of class actions. If 10,000 people have a $1,000 harm under similar circumstances, fault could potentially be addressed in a class action but certainly not in individual arbitrations.
I think that the use of insurance for moderate harms is often a commercial boondoggle for insurers, a la health insurance, which breaks incentives in many ways an leads to cost disease. And typical insurance regimes shift burden of proof about injury in damaging ways because insurers have deep pockets to deny claims in court and fight cases that establish precedents. I also don’t think that it matters for tail risks—unless explicitly mandating unlimited coverage, firms will have caps in the millions of dollars, and will ignore tail risks that will bankrupt them.
One way to address the tail, in place of strict liability, would be legislation allowing anticipated harms to be stopped via legal action, as opposed to my understanding that pursuing this type of prior restraint for uncertain harms isn’t possible in most domains.
I’d be interested in your thoughts on these points, as well as Cecil and Marie’s.
As an initial note, I don’t think my proposed model is that different from strict liability for small-to-midsize harms. But framing it in insurance terms is more politically palatable, and also allowed me to riff off the analogy between aviation law and automobile law to explain why I didn’t think the aviation-law analogy worked in a lot of cases. I didn’t intend to suggest a solution for the entire scope of AI liability issues. My focus probably comes as much from my professional interest in thinking about how modest-dollar disputes can be effectively litigated as from anything else.
That being said, I think that small/mid-size disputes would collectively be an important source of liability that would encourage more careful development and frankly would slow down development a bit. Their litigation would also bring more public attention to the harms, and provide grist for development of the common law as to AI harms. (If few people sue because proving fault is too expensive, there will be little development in caselaw.)
Health insurance is to a large extent sui generis because it lacks many of the classic features of insurance. The insured = the beneficiary, and often has substantial control over whether and how much “loss” to incur. For instance, I could decide to ignore a ~mild skin condition, use cheap last-generation remedies, or seek coverage of newfangled stuff at $900/bottle (all paid by insurance and manufacturer coupon).
Furthermore, for public policy reasons, we won’t let the insurer react against the insured for claims history. In contrast, consider condo insurance—after my condo association had a water-leak claim, our deductible doubled and our water-damage deductible went up five-fold. I told the other unitowners that we could expect to ~lose water-damage coverage if we filed another such claim in the next few years.
In contrast, you don’t see these pathologies as much in, e.g., automobile insurance (the party bearing the loss often isn’t the insured, the insured does not often “choose” the loss in the same way, and insurance companies can and will raise rates and dump risky clients altogether).
I’m not confident of this as an empirical matter—do you have a citation?
First off, under my (limited) model, fault would not be an issue so there would be no litigating that. Causation and injury could be litigated, but the potential causal and injurious mechanics for AI harm are manifold. So you probably wouldn’t get the same degree of deep-pockets motivation as in (say) asbestos or tobacco cases, where the causation questions could be common to tens of thousands of suits.
Next, as a practicing lawyer, my experience is that repeat litigants want to bury/settle cases that would risk establishing bad precedent. They are also less likely to appeal than a one-off litigant. A prime example is the U.S. Government. Let’s say the district court decision decides you can sue the Government for X. That decision only controls the outcome of that specific case and maybe that specific litigant; it isn’t even binding on the same judge in a different case. You take that case to the court of appeals and lose, and now everyone in a multi-state region knows they can sue the government for X. They don’t have to invest in litigating whether the government is immune from suit for X before getting to the merits of their case. This is why appeals by the Government require the approval of the number 3 in the whole Department of Justice (the Solicitor General).
Finally, I think what you describe is unavoidable in any scenario where a repeat litigant has a large financial stake in certain issues. If you push a ton of liability on OpenAI (and it doesn’t pass that onto an insurer), it will have ~the same incentives that an insurer has in your model.
Right. I agree with you that regulation—or pre-emptive, injunctive litigation—are the viable paths for controlling risks that would bankrupt the company. You’re generally right that injunctive relief is not generally available when money damages would be seen as compensating for any harm, and the financial ability of the risk-creating party to pay those damages may often not get much weight. See, e.g., Brown et al v. Sec’y, HHS, No. 20-14210 (11th Cir. July 14, 2021). You could fix that by statute, though.
From a theoretical perspective, one reason that preliminary injunctions are an extraordinary remedy is that there is often little practical way to compensate the enjoined party if it is later determined that they were wrongly enjoined. To do much good, your proposal would have to be done on an expedited basis. But if the standard is not demanding, there are going to be a lot of injunctions issued that will be ultimately judged to be erroneous on further review. My guess is that you will not be too troubled by that, but a lot of people will.
Other issues with injunctions to resolve:
There is a well-known problem where a lot of people across the nation have standing to bring a lawsuit. They can all bring their own lawsuit, the scope of relief is necessarily nationwide, and they only have to convince one district judge (out of several hundred) that an injunction is warranted. There’s appellate review, but it is not quick and certain aspects are deferential (factual findings reviewed for clear error, weighing reviewed for abuse of discretion). This is why you often see major U.S. government policies enjoined up front, the conservatives know which districts and divisions to file in (often in Texas) and so do the progressives (often D.C.). This could be fixed by centralizing such petitions in one court and consolidating cases.
There’s also an international element of the same. If the AI development activity is happening in the US, should any country be able to enjoin it? In other words, should any country get a legal veto on AI activity anywhere in the world? This would raise a foreign-relations nightmare. It is well-known that the judiciary in many countries (including several major world powers) is not meaningfully independent, and even where it largely is there could be political influence. The temptation to enjoin other countries’ AI companies would be strong. This seems very hard to fix.
My guess is that the U.S. would pass a federal statute preventing enforcement of foreign injunctions in the U.S., and maybe allowing the President to take retaliatory action against foreign AI companies if they deemed the anti-U.S. company action to be motivated by strategic concerns.
If your answer is some sort of world court, there is still a real problem of enforcement by a home-country government that doesn’t want to enforce the injunction against its own interests.
I find the idea of a reverse burden of proof interesting, but tbh I wasn’t really persuaded by the rest of your arguments. I guess the easiest way to respond to most of them would be “Sure, but human extinction kind of outweighs it” and then you’d reraise how these risks are abstract/speculative and then I’d respond that putting risks in two boxes, speculative and non-speculative, hinders clear thinking more than it helps. Anyway, that’s just how I see the argument play out.
In any case my main worry about strong liability laws is that we may create a situation where AI developers end up thinking primarily about dodging liability more than actually making the AI safe.Thanks. Might be more useful if you explain why the arguments weren’t persuasive to you. Our interest is in a system of liability that can meet AI safety goals and at the same time have a good chance of success in the real world. Anyway, even if we start from your premise, it doesn’t mean strict liability would work better than a fault-based liability system (as we demonstrated in Argument 1).
So my position is that most of your arguments are worth some “debate points” but that mitigating potential x-risks outweigh this.
I’ve personally made the mistake of thinking that the Overton Window is narrower than it actually was in the past. So even though such laws may not seem viable now, my strong expectation is that it will quickly change. At the same time, my intuition is that if we’re going to pursue the liability route, at least strict liability has the advantage of keeping the developer focused on preventing the issue from occurring rather than taking actions to avoid legal responsibility. Those actions won’t help, so they need to focus on preventing the issue from occurring.
I know that I wrote above:
and that this is in tension with what I’m writing now. I guess upon reflection I now feel that my concerns about strong liability laws only apply to strong fault-based liability laws, not to strict liability laws, so in retrospect I wouldn’t have included this sentence.
Regarding your discussion in point 1 - apologies for not addressing this in my initial reply—I just don’t buy that courts being able to handle chainsaws or medical or actuary evidence means that they’re equipped to handle transformative AI given how fast the situation is changing and how disputed many of the key questions are. Plus the stakes involved play a role in me not wanting to take a risk here/make an unnecessary bet on the capabilities of the courts. Even if there was a 90% chance that the courts would be fine, I’d prefer to avoid the 10% probability that they aren’t.
We shouldn’t be focused too heavily on what is politically feasible this year. A fair amount of our attention should be on what to prepare in order to handle a scenario in which there’s more of an expert consensus a couple of years from now.
This is a fair point, but we’re thinking about a scenario where such consensus takes a much much longer time to emerge. There’s no real reason to be sure that a super advanced model a couple of years from now will do the kinds of things that would produce a consensus.
Interesting post! Another potential downside (which I don’t think you mention) is that strict liability could disincentivize information sharing. For example, it could make AI labs more reluctant to disclose new dangerous capabilities or incidents (when that’s not required by law). That information could be valuable for other AI labs, for regulators, for safety researchers, and for users.
Really good point! Also just realised that what you’re saying is already playing out in cybersecurity incident reporting in many countries.
I upvoted this comment because it is a very valuable contribution to the debate. However, I also gave it an “x” vote (what is that called? disagree?) because I strongly disagree with the conclusion and recommendation.
Very briefly, everything you write here is factually (as best I know) true. There are serious obstacles to creating and to enforcing strict liability. And to do so would probably be unfair to some AI researchers who do not intend harm.
However, we need to think in a slightly more utilitarian manner. Maybe being unfair to some AI developers is the lesser of two evils in an imperfect world.
I come from the world of chemical engineering, and I’ve worked some time in Pharma. In these areas, there is not “strict liability” as such, in the sense that you typically do not go to jail if you can demonstrate that you have done everything by the book.
BUT—the “book” for chemical engineering or pharma is a much, much longer book, based on many decades of harsh lessons. Whatever project I might want to do, I would have to follow very strict, detailed guidelines every step of the way. If I develop a new drug, it might require more than a decade of testing before I can put it on the market, and if I make one flaw in that decade, I can be held criminally liable. If I build a factory and there is an accident, they can check every detail of every pump and pipe and reactor, they can check every calculation and every assumption I’ve made, and if just one of them is mistaken, or if just one time (even with a very valid reason) I have chosen not to follow the recommended standards, I can be criminally and civilly liable.
We have far more knowledge about how to create and test drugs than we have on how to create and test AI models. And in our wisdom, we believe it takes a decade to prove that a drug is safe to be released on the market.
We don’t have anything analogous to this for AI. So nobody (credible) is arguing that strict liability is an ideal solution or a fair solution. The argument is that, until we have a much better AI Governance system in place, with standards and protocols and monitoring systems and so on, then strict liability is one of the best ways we can ensure that people act responsibly in developing, testing and releasing models.
The AI developers like to argue that we’re stifling innovation if we don’t give them totally free-rein to do whatever they find interesting or promising. But this is not how the world works. There are thousands of frustrated pharmacologists who have ideas for drugs that might do wonders for some patients, but which are 3 years into a 10-year testing cycle instead of already saving lives. But they understand that this is necessary to create a world in which patients know that any drug that is prescribed by their doctor is safe for them (or that it’s potential risks are understood).
Strict liability is, in a way, telling AI model developers: “You say that your model is safe. OK, put your money where your mouth is. If you’re so sure that it’s safe, then you shouldn’t have any worries about strict-liability. If you’re not sure that it’s safe, then you shouldn’t be releasing it.”
This feels to me like a reasonable starting point. If AI-labs have a model which they believe is valuable but flawed (e.g. risk of bias), they do have the option to release it with that warning—for example to refuse to accept liability for certain identified risks. Lawmakers can then decide if that’s OK or not. It may take time, but eventually we’ll move forward.
Right now, it’s the Wild West. I can understand the frustration of people with brilliant models which could do much good in the world, but we need to apply the same safety standards that we apply to everyone else.
Strict liability is neither ideal nor fair. It’s just, right now, the best option until we find a better one.
Even with decades of development of pharma knowledge, and a complex regulatory system, things still blow up badly (e.g., Vioxx). And the pharma companies usually end up paying through the nose in liability, too. Here, we have a much weaker body of built-up knowledge and much weaker ex ante regulation.
There seems some pretty large things I disagree with in each of your arguments:
This seems exactly the sort of situation I want AI developers to think long and hard about. Frankly your counterexample looks like an example to me.
To me, where I cannot impose costs directly on the autonomous entity, autonomy again makes strict liability better, not worse. If nuclear explosions or trains were autonomous you seem to argue we shouldn’t place strict liability on their creators. This seems the opposite of what I’d expect.
I do not trust almost anyone’s ability to predict this stuff. If it’s good on its merits let’s push for it. Notably Robin Hanson and some other more “risk is low” people support strict liability (because they don’t think disasters will happen). I think there is the case for a coalition around this. I don’t buy that you can predict that this will struggle.
I am interested in what bad things you think might happen with strict liability or how you think it’s gone in the past?
I would be interested in understanding whether you think that joint-and-several liability among model training, model developers, application developers, and users would address many of the criticisms you point out against civil liability. As I said last year, “joint-and-several liability for developers, application providers, and users for misuse, copyright violation, and illegal discrimination would be a useful initial band-aid; among other things, this provides motive for companies to help craft regulation to provide clear rules about what is needed to ensure on each party’s behalf that they will not be financially liable for a given use, or misuse.”
I also think that this helps mitigate the issue with fault-based liability in proving culpability, but I’m agnostic about which liability regime is justified.
Lastly, I think that your arguments mean that there’s good reason to develop a clear proposal for some new liability standard, perhaps including requirements for uncapped liability insurance for some specific portion of eventual damages, rather than assume that the dichotomy of strict vs. fault based is immutable.
This is super interesting. Please give me a couple of days to think through it and then comment again.
[on argument 3]
I hear that, but AI liability regulation is presumably going to be governed by the country in which the harm occurs. If you’re a Chinese AI company and want to make a ton of money, you’re going to have to operate in the US, EU, and other highly developed countries. To quote Willie Sutton (allegedly) when asked why he robbed banks: “Because that’s where the money is.” That means that you’re going to have to comply with those countries’ standards when operating in them. It’s not clear why a rule that governs liabilities for harms committed in the US would have a significantly greater impact on US firms than on Chinese ones.
I would agree insofar as (e.g.) the US should not allow people harmed by AI in China to sue in the US under more favorable standards than China would apply. That would disadvantage U.S. companies—who would be held to U.S. standards of liability for their harms in China—while Chinese companies would be held to lower standards (because they would generally not be amenable to personal jurisdiction in the U.S. for non-U.S. harms).
I don’t see a clear connection here. The liability system is not meant to directly prevent (or compensate for) global catastrophic risks. In the event of a catastrophic event, the offending AI company is toast (hopefully the rest of us aren’t). It will be shut down irrespective of the outcome of any litigation about its fault.
It is true that EAs’ motivation for imposing heavier liability on AI companies is related to GCRs, but there’s no reason that has to be the chosen policy rationale. Something as mundane as “They are in the best position to shoulder the harms that will inevitably come along, and pass that risk onto their customers through higher prices” could do the trick.
Yes, I see a strong argument for the claim that the companies are in the best position to shoulder the harms that will inevitably come along, and pass that risk onto their customers through higher prices—but the other critical part is that this also changes incentives because liability insurers will demand the firms mitigate the risks. (And this is approaching the GCR argument, from a different side.)
The biggest fish—which I assume are the ones you are going to be most worried about from a GCR perspective—are very likely to self-insure.
I’m also less confident in insurers’ abilities to insist on and monitor risk from AI development than risk exposure from application of AI. For instance, it seems a lot easier for a third party (who knows much less about AI systems than the insured) to figure out “You shouldn’t let AI determine the results of that CT scan without a human overread” than “You shouldn’t use technique X to grow your AI technology.”
Thanks, really interesting.
Good point re complying everywhere, but I think the UK example shows that countries are keen to have the AI companies’ offices in their jurisdictions, and are clearly worried that having some liability regimes would discourage that.
I don’t think we connect that part and the previous one well-enough. But anyway, it’s very hard to convince anyone that strict liability ought to be the regime in place unless you can demonstrate that the risk is super high, and can be very consequential. I can’t see how your alternative works because, well, I haven’t seen any other scenario so far where strict liability has been applied on that rationale. They can pass risk to customers via higher charges but won’t the charges have to be unusually high to mitigate against possible bankruptcy?
Logically, that should only be the case if the firm is exposed to more liability from locating in that jurisdiction rather than an alternative jurisdiction. If the jurisdiction’s choice of law rule is “apply the liability rules of the jurisdiction where the harm occurred,” I don’t see how that is appreciably worse for the AI company. If they have assets and/or business in the country where the harm occurred—or any country that will enforce that country’s court judgments—they are going to be vulnerable to judgment issued by that country’s courts. I’m not up to date on who will enforce whose judgments, but exiting the US or EU would be a massive cost for any AI company. There are other countries for which exiting would be a major commercial disadvantage.
The U.S. at least has imposed no-fault compensation regimes where political and/or other realities were seen to warrant it, although their setup is admittedly different. The two that come immediately to mind are the so-called “vaccine court” and the worker’s compensation system.
So it can be done; the question is whether the political will to do it exists. (I do agree that it won’t happen through expansion of common-law doctrine.) My own view is that there’s a decent chance that the political will comes into existence once people realize that the practical alternative in many cases is de facto immunity for the AI companies. And I think that’s where the crux largely is.
“And the truth is, as even Gabriel Weil accepts, it’s super unlikely that the training and deployment of highly capable advanced AI would fit the longstanding legal definition of an “abnormally dangerous activity”.”
This is not an accurate characterization of my views. Here’s a relevant quote from the linked paper:
“Training and deploying frontier AI models are clearly not activities of common usage, at least with current technology, given the enormous computational resource requirements of these systems.117 This means that, under the Restatement Third’s test, the key issue is whether training or deploying advanced AI systems satisfies the first criterion of creating a foreseeable and highly significant risk of harm even when reasonable care is exercised. This question is likely to be controversial. While I think the available evidence does support the conclusion that reasonable care may be insufficient to reduce the risk of catastrophic AI misalignment or misuse to below levels that would qualify as “highly significant,” recognizing any software development project as abnormally dangerous would represent a substantial doctrinal innovation.”
In the following paragraph, I further clarify: “To be clear, I do think an accurate understanding of the risks of advanced AI systems supports strict liability for the training and deployment of advanced AI systems, but it is important to recognize that this is not the likely outcome of a mechanical application of existing doctrine to harms caused by these systems.”
In a later section of the paper, I say “If courts are persuaded by the arguments summarized in part I that advanced AI systems do indeed pose a significant risk of causing human extinction, they should recognize training and deploying those systems as an abnormally dangerous activity. To be sure, treating software development as abnormally dangerous would represent a significant extension of existing strict liability, but it is entirely consistent with the doctrinal rationale for the abnormally dangerous activities category.”
I don’t see how you can read me as accepting that “it’s super unlikely that the training and deployment of highly capable advanced AI would fit the longstanding legal definition of an “abnormally dangerous activity”.”
Executive summary: The AI safety community should reconsider its embrace of strict liability for AI harms because it has significant flaws and is unlikely to gain traction, and should instead focus on defining specific duties and faults that would trigger liability.
Key points:
Strict criminal liability is inappropriate for AI harms, while strict civil liability is unfair to developers taking safety precautions and unlikely to deter AI development.
Analogies used to justify strict liability for AI, such as abnormally dangerous activities, are flawed due to differences in risk materialization, consensus, and societal benefits.
Strict liability proposals have a low chance of success due to economic and national security pressures and lack of expert consensus on AI risk levels.
The AI safety community should focus on defining specific duties and faults to trigger liability, as this approach is more likely to succeed and achieve safety goals.
A fault-based liability system with a reverse burden of proof is recommended for cases where clear malicious intent cannot be proven.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.