That appears to assume the conclusion (“AI is dangerous”) to explain why the burden of proof is on the inventors to prove AI is safe. But I’m asking about where the burden of proof should lie prior to us already having the answer! If we had used this burden for every prior technology, it’s likely that a giant amount of innovation would have been stifled.
Now, you could alternatively think that the burden of proof is on other people to show that a new technology is dangerous, and this burden has already been met for AI. But I think that’s a different claim. I was responding to the idea that the burden of proof is on AI developers to prove that their product is safe.
We might be talking past each other. I think the burden of proof that AI-in-principle could be dangerous is on non-inventors, and that has already been met[1]. I think the burden of proof that specific-AI-tech-in-practice is safe should then be on AI manufacturers.
Similarly, if we know no details about nuclear power plants or nuclear weapons, the burden of proof about how scary an abstract “power plant” or “taking some ore from the ground and refining it” should be on concerned people. But after the theoretical case for nuclear scariness is demonstrated, we shouldn’t have had to wait until Hiroshima or Nagasaki, or even the Trinity Test, before the burden of proof falls on nuclear weapons manufacturers/states to demonstrate that their potential for accident is low.
I think the burden of proof that AI-in-principle could be dangerous is on non-inventors, and that has already been met. I think the burden of proof that specific-AI-tech-in-practice is safe should then be on AI manufacturers.
I think that makes sense. But I also think that the idea of asking for a “pause” looks a lot more like asking AI developers to prove that AI in the abstract can be safe, whereas an “FDA for new AIs” looks more like asking developers to prove their specific implementations are safe. The distinction between the two ideas here is blurry though, admittedly.
But insofar as this distinction makes sense, I think it should likely push us against a generic pause, and in favor of specific targeted regulations.
Nuclear chain reactions leading to massive explosions are dangerous. We don’t have separate prohibition treaties on each specific model of nuke.
Impenetrable multi-trillion-parameter neural networks are dangerous. I think it does make sense for AI developers to prove that AI (as per the current foundation model neural network paradigm) in the abstract can be safe.
“I mean a global, indefinite moratorium on the development of frontier models until it is safe to proceed.”
I think this is distinctly different from what you claim. In any actual system implementing a pause, the model developer is free “to prove their specific implementations are safe,” and go ahead. The question is what the default is—and you’ve implied elsewhere in this thread that you think that developers should be treated like pre-1938 drug manufacturers, with no rules.
If what you’re proposing is instead that there needs to be a regulatory body like the FDA to administer the rules and review cases when a company claims to have sufficient evidence of safety when planning to create a model, with a default rule that it’s illegal to develop the model until reviewed, instead of a ban with a review for exceptions when a company claims to have sufficient evidence of safety when planning to create a model, I think the gap between our positions is less blurry than it is primarily semantic.
you’ve implied elsewhere in this thread that you think that developers should be treated like pre-1938 drug manufacturers, with no rules.
I think you misread me. I’ve said across multiple comments that I favor targeted regulations that are based on foreseeable harms after we’ve gotten more acquainted with the technology. I don’t think that’s very similar to an indefinite pause, and it certainly isn’t the same as “no rules”.
That makes sense—I was confused, since you said different things, and some of them were subjunctive, and some were speaking about why you disagree with proposed analogies.
Given your perspective, is loss-of-control from more capable and larger models not a foreseeable harm? If we see a single example of this, and we manage to shut it down, would you then be in favor of a regulate-before-training approach?
Almost no past technology have a claim to be a non-Pascalian existential risk to humanity; especially one that’s known in advance.
The only exceptions I could think of are nukes, certain forms of bioweapons research, and maybe CFCs in refrigeration.
That appears to assume the conclusion (“AI is dangerous”) to explain why the burden of proof is on the inventors to prove AI is safe. But I’m asking about where the burden of proof should lie prior to us already having the answer! If we had used this burden for every prior technology, it’s likely that a giant amount of innovation would have been stifled.
Now, you could alternatively think that the burden of proof is on other people to show that a new technology is dangerous, and this burden has already been met for AI. But I think that’s a different claim. I was responding to the idea that the burden of proof is on AI developers to prove that their product is safe.
We might be talking past each other. I think the burden of proof that AI-in-principle could be dangerous is on non-inventors, and that has already been met[1]. I think the burden of proof that specific-AI-tech-in-practice is safe should then be on AI manufacturers.
Similarly, if we know no details about nuclear power plants or nuclear weapons, the burden of proof about how scary an abstract “power plant” or “taking some ore from the ground and refining it” should be on concerned people. But after the theoretical case for nuclear scariness is demonstrated, we shouldn’t have had to wait until Hiroshima or Nagasaki, or even the Trinity Test, before the burden of proof falls on nuclear weapons manufacturers/states to demonstrate that their potential for accident is low.
As demonstrated by eg, surveys.
I think that makes sense. But I also think that the idea of asking for a “pause” looks a lot more like asking AI developers to prove that AI in the abstract can be safe, whereas an “FDA for new AIs” looks more like asking developers to prove their specific implementations are safe. The distinction between the two ideas here is blurry though, admittedly.
But insofar as this distinction makes sense, I think it should likely push us against a generic pause, and in favor of specific targeted regulations.
Nuclear chain reactions leading to massive explosions are dangerous. We don’t have separate prohibition treaties on each specific model of nuke.
Impenetrable multi-trillion-parameter neural networks are dangerous. I think it does make sense for AI developers to prove that AI (as per the current foundation model neural network paradigm) in the abstract can be safe.
I think this is distinctly different from what you claim. In any actual system implementing a pause, the model developer is free “to prove their specific implementations are safe,” and go ahead. The question is what the default is—and you’ve implied elsewhere in this thread that you think that developers should be treated like pre-1938 drug manufacturers, with no rules.
If what you’re proposing is instead that there needs to be a regulatory body like the FDA to administer the rules and review cases when a company claims to have sufficient evidence of safety when planning to create a model, with a default rule that it’s illegal to develop the model until reviewed, instead of a ban with a review for exceptions when a company claims to have sufficient evidence of safety when planning to create a model, I think the gap between our positions is less blurry than it is primarily semantic.
I think you misread me. I’ve said across multiple comments that I favor targeted regulations that are based on foreseeable harms after we’ve gotten more acquainted with the technology. I don’t think that’s very similar to an indefinite pause, and it certainly isn’t the same as “no rules”.
That makes sense—I was confused, since you said different things, and some of them were subjunctive, and some were speaking about why you disagree with proposed analogies.
Given your perspective, is loss-of-control from more capable and larger models not a foreseeable harm? If we see a single example of this, and we manage to shut it down, would you then be in favor of a regulate-before-training approach?