Instead, I recommend: “My prior is [something], here’s why”.
I’m even more against “the burden of proof for [some policy] is on X”—I mean, what does “burden of proof” even mean in the context of policy? but hold that thought.
An example that I’m against:
“The burden of proof for vaccines helping should be on people who want to vaccinate, because it’s unusual to put something in your body”
I’m against it because
It implicitly assumes that vaccines should be judged as part of the group “putting something in your body”
It’s a conversation stopper. It claims one of the sides of the conversation has nothing to do.
I prefer:
“my prior for vaccines is that they’re bad, because my prior for putting things in my body is bad (but I’m open to changing my mind from evidence, and I’m open to maybe using a different prior if you have a better idea)”
I also like:
“my prior is that governments should not force people to do things, and so I’m against forcing people to be vaccinated” or “my prior is that governments are allowed to force people to do things that, by scientific consensus, protect them”. I like that we’re discussing explicitly “which priors should we use to decide which policy to accept and which not to”
What got me to write a about this now:
I don’t like the discussion about who has the “burden of proof” to decide we should or shouldn’t have an AI pause. I would prefer discussing which prior to use for it.
For example,
Should our prior be “should we pause any new technology”, and so AI is “just” another new technology?
Should our prior be that an AI is an extinction risk like a meteor in “don’t look up”, and so should be paused unless we have further evidence showing reasons to not-pause it?
Should our priors be based on expert polls (do experts recommend a pause), and should we require evidence in order to change our mind from those polls?
My opinion: we should explicitly discuss which priors to use (which isn’t an easy question), and not just assume that one “side” has the “burden of proof”
So, I’ll give two more examples of how burden of proof gets used typically:
You claim that you just saw a unicorn ride past. I say that the burden of proof is on you to prove it, as unicorns do not exist (as far as we know).
As prime minister, you try and combat obesity by taxing people in proportion to their weight. I say that the burden of proof is on you to prove that such a policy would do more good than harm.
I think in both these cases, the statements made are quite reasonable. Let me try to translate the objections into your language:
my prior of you seeing a unicorn is extremely low, because unicorns do not exist (as far as we know)
My prior of this policy being a good idea is low, because most potential interventions are not helpful.
These are fine, but I’m not sure I prefer either of these. It seems like the other party can just say “well my priors are high, so I guess both our beliefs are equally valid”.
I think “burden of proof” translates to “you should provide a lot of proof for your position in order for me or anyone else to believe you”. It’s a statement of what peoples priors should be.
“We should avoid building more powerful AI because it might kill us all” breaks to
No prior AI system has tried to kill us all
We are not sure how powerful a system we can really make scaling known techniques and adjacent to known techniques in the next 10-20 years. A system 20 years from now might not actually be “AGI” we don’t know.
This sounds like someone should have the burden of proof of showing near future AI systems are (1) lethal (2) powerful in a utility way, not just a trick but actually effective at real world tasks
And like the absence of unicorns caught on film someone could argue that 1⁄2 are unlikely by prior due to AI hype that did not pan out.
The counter argument seems to be “we should pause now, I don’t have to prove anything because an AI system might be so smart it can defeat any obstacles even though I don’t know how it could do that, it will be so smart it finds a way”. Or “by the time there is proof we will be about to die”.
I’ve always viewed burden of proof as a dialectical tool. To say one has the burden proof is to say that if they meet the following set of necessary and jointly sufficient conditions:
1. You’ve made a claim 2. You’re attempting to convince another of the claim.
They have the obligation in the discussion to provide justification for the claim. If (1) isn’t the case, then of course you don’t have any burden to provide justification. If (2) isn’t the case (Say, everyone already agrees with the claim or someone just wants your opinion on something) it’s not clear to me you have some obligation to provide justification either.
On this account, it’s not like burden of proof talk favors a side. And I’m not sure it implicitly assumes anything or is a conversation stopper. So maybe we can “keep burden of proof talk” by using this construal while also focusing more on explicit discussion of priors. Idk, just a thought I had while reading this.
Against “the burden of proof is on X”
Instead, I recommend: “My prior is [something], here’s why”.
I’m even more against “the burden of proof for [some policy] is on X”—I mean, what does “burden of proof” even mean in the context of policy? but hold that thought.
An example that I’m against:
“The burden of proof for vaccines helping should be on people who want to vaccinate, because it’s unusual to put something in your body”
I’m against it because
It implicitly assumes that vaccines should be judged as part of the group “putting something in your body”
It’s a conversation stopper. It claims one of the sides of the conversation has nothing to do.
I prefer:
“my prior for vaccines is that they’re bad, because my prior for putting things in my body is bad (but I’m open to changing my mind from evidence, and I’m open to maybe using a different prior if you have a better idea)”
I also like:
“my prior is that governments should not force people to do things, and so I’m against forcing people to be vaccinated” or “my prior is that governments are allowed to force people to do things that, by scientific consensus, protect them”. I like that we’re discussing explicitly “which priors should we use to decide which policy to accept and which not to”
What got me to write a about this now:
I don’t like the discussion about who has the “burden of proof” to decide we should or shouldn’t have an AI pause. I would prefer discussing which prior to use for it.
For example,
Should our prior be “should we pause any new technology”, and so AI is “just” another new technology?
Should our prior be that an AI is an extinction risk like a meteor in “don’t look up”, and so should be paused unless we have further evidence showing reasons to not-pause it?
Should our priors be based on expert polls (do experts recommend a pause), and should we require evidence in order to change our mind from those polls?
My opinion: we should explicitly discuss which priors to use (which isn’t an easy question), and not just assume that one “side” has the “burden of proof”
</rant>
So, I’ll give two more examples of how burden of proof gets used typically:
You claim that you just saw a unicorn ride past. I say that the burden of proof is on you to prove it, as unicorns do not exist (as far as we know).
As prime minister, you try and combat obesity by taxing people in proportion to their weight. I say that the burden of proof is on you to prove that such a policy would do more good than harm.
I think in both these cases, the statements made are quite reasonable. Let me try to translate the objections into your language:
my prior of you seeing a unicorn is extremely low, because unicorns do not exist (as far as we know)
My prior of this policy being a good idea is low, because most potential interventions are not helpful.
These are fine, but I’m not sure I prefer either of these. It seems like the other party can just say “well my priors are high, so I guess both our beliefs are equally valid”.
I think “burden of proof” translates to “you should provide a lot of proof for your position in order for me or anyone else to believe you”. It’s a statement of what peoples priors should be.
Why doesn’t this translate to AI risk.
“We should avoid building more powerful AI because it might kill us all” breaks to
No prior AI system has tried to kill us all
We are not sure how powerful a system we can really make scaling known techniques and adjacent to known techniques in the next 10-20 years. A system 20 years from now might not actually be “AGI” we don’t know.
This sounds like someone should have the burden of proof of showing near future AI systems are (1) lethal (2) powerful in a utility way, not just a trick but actually effective at real world tasks
And like the absence of unicorns caught on film someone could argue that 1⁄2 are unlikely by prior due to AI hype that did not pan out.
The counter argument seems to be “we should pause now, I don’t have to prove anything because an AI system might be so smart it can defeat any obstacles even though I don’t know how it could do that, it will be so smart it finds a way”. Or “by the time there is proof we will be about to die”.
I’ve always viewed burden of proof as a dialectical tool. To say one has the burden proof is to say that if they meet the following set of necessary and jointly sufficient conditions:
1. You’ve made a claim
2. You’re attempting to convince another of the claim.
They have the obligation in the discussion to provide justification for the claim. If (1) isn’t the case, then of course you don’t have any burden to provide justification. If (2) isn’t the case (Say, everyone already agrees with the claim or someone just wants your opinion on something) it’s not clear to me you have some obligation to provide justification either.
On this account, it’s not like burden of proof talk favors a side. And I’m not sure it implicitly assumes anything or is a conversation stopper. So maybe we can “keep burden of proof talk” by using this construal while also focusing more on explicit discussion of priors. Idk, just a thought I had while reading this.