Where have I ever claimed that there is no evidence worth considering?
In your OP, you write:
In this post, I’ve criticized non-evidence-based arguments, which hangs on the idea that evidence is something that is inherently required. Yet it has become commonplace to claim the opposite. One example of this argument is presented in the International AI Safety Report
You then quote the following:
Given sometimes rapid and unexpected advancements, policymakers will often have to weigh potential benefits and risks of imminent AI advancements without having a large body of scientific evidence available. In doing so, they face a dilemma. On the one hand, pre-emptive risk mitigation measures based on limited evidence might turn out to be ineffective or unnecessary. On the other hand, waiting for stronger evidence of impending risk could leave society unprepared or even make mitigation impossible – for instance if sudden leaps in AI capabilities, and their associated risks, occur.
Your summary of the quoted text is inaccurate. You claim that this is an arguement that evidence is not something that in inherently required, but the quote says no such thing. Instead, it references “a large body of scientific evidence” and “stronger evidence” vs “limited evidence”. This quote essential makes the same arguement I do above. How can we square the differences in these interpretations?
In response to me, you write:
In my post, I referred to the concept of “evidence-based policy making”. In this context, evidence refers specifically to rigorous, scientific evidence, as opposed to intuitions, unsubstantiated beliefs and anecdotes. Scientific evidence, as I said, referring to high-quality studies corroborated by other studies.
So, as used in your post, “evidence” means “rigorous, scientific evidence, as opposed to intuitions, unsubstantiated beliefs and anecdotes”. This is why I find your reference to “scientific evidence” frustrating. You draw a distinct between two categories of evidence and claim policy should be based on only one. I disagree, I think policy should be based on all available evidence, including intuition and anecdote (“unsubtantiated belief” obviously seems definitionally not evidence). I also think your argument relies heavily on contrasting with a hypothetical highly rigorous body of evidence that isn’t often achieved, which is why I have pointed out what I see as the “messiness” of lots of published scientific research.
The distinction you draw and how you defined “evidence” results in an equivocation. Your caracterization of the quote above only makes sense if you are claiming that AI risk can only claim to be “evidence-based” if is is backed by “high-quality studies that withstand scrutiny”. In other words, as I said in one of my comments:
It seems like the core of your argument is saying that there is a high burden of proof that hasn’t been met.
So, where do we disagreee? As I say immediately after:
I agree that arguments for short timelines haven’t met a high burden of proof but I don’t believe that there is such a burden.
I believe that we should compare E[benefits(AI)] with E[benefits(GHD)] and any other possible alternative cause areas, with no area having any specific burden of proof. The quality of the evidence plays out in taking those expectations. Different people may disagree on the results based on their interpretations of the evidence. People might weigh different sources of evidence differently. But there is no specific burden to have “high-quality studies that withstand scrutiny”, although this obviously weighs in favor of a cause that does have those studies. I don’t think having high quality studies amounts to “style points”. What I think would amount to “style points” is if someone concluded that E[benefits(AI)] > E[benefits(GHD)] but went with GHD anyway because they think AI is off limits due to the lack of “high-quality studies that withstand scrutiny” (i.e. if there is a burden of proof where “high-quality studies that withstand scrutiny” are required).
If you believe that evidence that does not withstand scrutiny (that is, evidence that does not meet basic quality standards, contains major methodological errors, is statistically insignificant, is based on fallacious reasoning, or any other reason why the evidence is scrutinized) is evidence that we should use, then you are advocating for pseudoscience. The expected value of benefits based on such evidence is near zero.
I’m sorry if criticizing pseudoscience is frustrating, but that kind of thinking has no place in rational decision-making.
Your summary of the quoted text is inaccurate. You claim that this is an arguement that evidence is not something that in inherently required, but the quote says no such thing. Instead, it references “a large body of scientific evidence” and “stronger evidence” vs “limited evidence”. This quote essential makes the same arguement I do above. How can we square the differences in these interpretations?
The quoted text implies that the evidence would not be sufficient under normal circumstances, hence the “evidence dilemma”. If the amount of evidence was sufficient, there would be no question about what is the correct action. While the text washes its hands from making the actual decision to rely on insufficient evidence, it clearly considers this as a serious possibility, which is not something that I believe anyone should advocate.
You are splitting hairs about the difference between “no evidence” and “limited evidence”. The report considers a multitude of different AI risks, some of which have more evidence and some of which have less. What is important is that they bring up the idea that policy should be made without proper evidence.
If you believe that evidence that does not withstand scrutiny (that is, evidence that does not meet basic quality standards, contains major methodological errors, is statistically insignificant, is based on fallacious reasoning, or any other reason why the evidence is scrutinized) is evidence that we should use, then you are advocating for pseudoscience. The expected value of benefits based on such evidence is near zero.
I don’t think evidence which is based on something other than “high-quality studies that withstand scrutiny” is pseudoscience. You could have moderate-quality studies that withstand scutiny, you could have preliminary studies which are suggestive but which haven’t been around long enough for scrutiny to percolate up. I don’t think these things have near zero evidential value.
This is my issue with your use of the term “scientific evidence” and related concepts. Its role in the argument is mostly rhetorical, having the effective of charcterizing other arguments or positions as not worthy of consideration without engaging with the messy question of what value various pieces of evidence actually have. It causes confusion and results in you equivocating about what counts as “evidence”.
My view, and where we seem to disagree, is that I think there are types of evidence other than “high-quality studies that withstand scrutiny” and pseudoscience. Look, I agree that if something has basically zero evidential value we can reasonably round that off to zero. But “limited evidence” isn’t the same as near-zero evidence. I think there is a catgory of evidence between pseudoscience/near-zero evidence and “high-quality studies that withstand scrutiny”. When we don’t have access to the highest quality evidence, it is acceptable in my view to make policy based on the best evidence that we have, including if it is in that imtermediate category. This is the same argument made in the quote from the report.
The quoted text implies that the evidence would not be sufficient under normal circumstances
This is exactly what I mean when I say this approach results in you equivocating. In your OP, you explicitly claim that this quote argues that evidence is not something that is needed. You clarify in your comments with me and in a clarification at the top of your post that only “high-quality studies that withstand scrutiny” really count as evidence as you use the term. The fact that you are using the word “evidence” in this way is causing you to misinterpret the quoted statement. The quote is saying that even if we don’t have the ideal, high-quality evidence that we would like and that might be need for us to be highly confident and establish a strong consensus that in situations of uncertainty it is acceptable to make policy based on more limited or moderate evidence. I share this view and think it is reasonable nad not pseudoscientific or somehow a claim that evidence of some kind isn’t required.
If the amount of evidence was sufficient, there would be no question about what is the correct action.
Uncetainty exists! You can be in a situation where the correct decision isn’t clear because the available information isn’t ideal. This is extremely common in real-world decision making. The entire point of this quote and my own comments is that when these situations arise the reasonable thing to do is to make the best possible decision with the information you have (which might involve trying to get more information) rather than declaring some policies off the table because they don’t have the highest quailty evidence supporting them. Making decisions under uncertainty means making decisions based on limited evidence sometimes.
Your argument is very similar to creationist and other pseudoscientific/conspiracy theory-style arguments.
A creationist might argue that the existence of life, humanity, and other complex phenomena is “evidence” for intelligent design. If we allow this to count as “limited” evidence (or whatever term we choose to use), it is possible to follow through a Pascal’s wager-style argument and posit that this “evidence”, even if it has high uncertainty, is enough to merit an action.
It is always possible to come up with “evidence” for any claim. In evidence-based decision making, we must set a bar for evidence. Otherwise, the word “evidence” would lose it’s meaning, and we’d be wasting our resources considering every piece of knowledge there exists as “evidence”.
You could have moderate-quality studies that withstand scutiny
If the studies withstand scrutiny, then they are high-quality studies. Of course, it is possible that the study has multiple conclusions, and some of them are undermined by scrutiny and some are not, or that there are errors that do not undermine the conclusions. These studies can of course be used as evidence. I used “high-quality” as the opposite of “low-quality”, and splitting hair about “moderate-quality” is uninteresting.
you could have preliminary studies which are suggestive but which haven’t been around long enough for scrutiny to percolate up
This is a good basis when, e.g., funding new research, as confirming and replicating recent studies is an important part of science. In this case, it doesn’t matter that much if the study’s conclusions end up being true or false, as confirming either way is valuable. Researching interesting things is good, and even bad studies are evidence that the topic is interesting. But they are not evidence that should be used for other kind of decision-making.
The fact that you are using the word “evidence” in this way is causing you to misinterpret the quoted statement.
You are again splitting a hair about the meanings of words. The important thing is that they are advocating for making decisions without sufficient evidence, which is something I oppose. Their report is long and contains many AI risks, some of which (like deepfakes) have high-quality studies behind them, while others (like X-risks) do not. As a whole, the report “has some evidence” that there are risks associated with AI. So they talk about “limited evidence”. What is important is that they imply this “limited evidence” is not sufficient for making decisions.
But “limited evidence” isn’t the same as near-zero evidence
Splitting a hair. You can call your evidence limited evidence if you want. It won’t get you a free pass that your argument should be considered. If it has too much uncertainty or doesn’t withstand scrutiny, it shouldn’t be taken in as evidence. Otherwise we end up in the creationist situation.
In your OP, you write:
You then quote the following:
Your summary of the quoted text is inaccurate. You claim that this is an arguement that evidence is not something that in inherently required, but the quote says no such thing. Instead, it references “a large body of scientific evidence” and “stronger evidence” vs “limited evidence”. This quote essential makes the same arguement I do above. How can we square the differences in these interpretations?
In response to me, you write:
You also have added as a clarfication to your OP:
So, as used in your post, “evidence” means “rigorous, scientific evidence, as opposed to intuitions, unsubstantiated beliefs and anecdotes”. This is why I find your reference to “scientific evidence” frustrating. You draw a distinct between two categories of evidence and claim policy should be based on only one. I disagree, I think policy should be based on all available evidence, including intuition and anecdote (“unsubtantiated belief” obviously seems definitionally not evidence). I also think your argument relies heavily on contrasting with a hypothetical highly rigorous body of evidence that isn’t often achieved, which is why I have pointed out what I see as the “messiness” of lots of published scientific research.
The distinction you draw and how you defined “evidence” results in an equivocation. Your caracterization of the quote above only makes sense if you are claiming that AI risk can only claim to be “evidence-based” if is is backed by “high-quality studies that withstand scrutiny”. In other words, as I said in one of my comments:
So, where do we disagreee? As I say immediately after:
I believe that we should compare E[benefits(AI)] with E[benefits(GHD)] and any other possible alternative cause areas, with no area having any specific burden of proof. The quality of the evidence plays out in taking those expectations. Different people may disagree on the results based on their interpretations of the evidence. People might weigh different sources of evidence differently. But there is no specific burden to have “high-quality studies that withstand scrutiny”, although this obviously weighs in favor of a cause that does have those studies. I don’t think having high quality studies amounts to “style points”. What I think would amount to “style points” is if someone concluded that E[benefits(AI)] > E[benefits(GHD)] but went with GHD anyway because they think AI is off limits due to the lack of “high-quality studies that withstand scrutiny” (i.e. if there is a burden of proof where “high-quality studies that withstand scrutiny” are required).
If you believe that evidence that does not withstand scrutiny (that is, evidence that does not meet basic quality standards, contains major methodological errors, is statistically insignificant, is based on fallacious reasoning, or any other reason why the evidence is scrutinized) is evidence that we should use, then you are advocating for pseudoscience. The expected value of benefits based on such evidence is near zero.
I’m sorry if criticizing pseudoscience is frustrating, but that kind of thinking has no place in rational decision-making.
The quoted text implies that the evidence would not be sufficient under normal circumstances, hence the “evidence dilemma”. If the amount of evidence was sufficient, there would be no question about what is the correct action. While the text washes its hands from making the actual decision to rely on insufficient evidence, it clearly considers this as a serious possibility, which is not something that I believe anyone should advocate.
You are splitting hairs about the difference between “no evidence” and “limited evidence”. The report considers a multitude of different AI risks, some of which have more evidence and some of which have less. What is important is that they bring up the idea that policy should be made without proper evidence.
I don’t think evidence which is based on something other than “high-quality studies that withstand scrutiny” is pseudoscience. You could have moderate-quality studies that withstand scutiny, you could have preliminary studies which are suggestive but which haven’t been around long enough for scrutiny to percolate up. I don’t think these things have near zero evidential value.
This is my issue with your use of the term “scientific evidence” and related concepts. Its role in the argument is mostly rhetorical, having the effective of charcterizing other arguments or positions as not worthy of consideration without engaging with the messy question of what value various pieces of evidence actually have. It causes confusion and results in you equivocating about what counts as “evidence”.
My view, and where we seem to disagree, is that I think there are types of evidence other than “high-quality studies that withstand scrutiny” and pseudoscience. Look, I agree that if something has basically zero evidential value we can reasonably round that off to zero. But “limited evidence” isn’t the same as near-zero evidence. I think there is a catgory of evidence between pseudoscience/near-zero evidence and “high-quality studies that withstand scrutiny”. When we don’t have access to the highest quality evidence, it is acceptable in my view to make policy based on the best evidence that we have, including if it is in that imtermediate category. This is the same argument made in the quote from the report.
This is exactly what I mean when I say this approach results in you equivocating. In your OP, you explicitly claim that this quote argues that evidence is not something that is needed. You clarify in your comments with me and in a clarification at the top of your post that only “high-quality studies that withstand scrutiny” really count as evidence as you use the term. The fact that you are using the word “evidence” in this way is causing you to misinterpret the quoted statement. The quote is saying that even if we don’t have the ideal, high-quality evidence that we would like and that might be need for us to be highly confident and establish a strong consensus that in situations of uncertainty it is acceptable to make policy based on more limited or moderate evidence. I share this view and think it is reasonable nad not pseudoscientific or somehow a claim that evidence of some kind isn’t required.
Uncetainty exists! You can be in a situation where the correct decision isn’t clear because the available information isn’t ideal. This is extremely common in real-world decision making. The entire point of this quote and my own comments is that when these situations arise the reasonable thing to do is to make the best possible decision with the information you have (which might involve trying to get more information) rather than declaring some policies off the table because they don’t have the highest quailty evidence supporting them. Making decisions under uncertainty means making decisions based on limited evidence sometimes.
Your argument is very similar to creationist and other pseudoscientific/conspiracy theory-style arguments.
A creationist might argue that the existence of life, humanity, and other complex phenomena is “evidence” for intelligent design. If we allow this to count as “limited” evidence (or whatever term we choose to use), it is possible to follow through a Pascal’s wager-style argument and posit that this “evidence”, even if it has high uncertainty, is enough to merit an action.
It is always possible to come up with “evidence” for any claim. In evidence-based decision making, we must set a bar for evidence. Otherwise, the word “evidence” would lose it’s meaning, and we’d be wasting our resources considering every piece of knowledge there exists as “evidence”.
If the studies withstand scrutiny, then they are high-quality studies. Of course, it is possible that the study has multiple conclusions, and some of them are undermined by scrutiny and some are not, or that there are errors that do not undermine the conclusions. These studies can of course be used as evidence. I used “high-quality” as the opposite of “low-quality”, and splitting hair about “moderate-quality” is uninteresting.
This is a good basis when, e.g., funding new research, as confirming and replicating recent studies is an important part of science. In this case, it doesn’t matter that much if the study’s conclusions end up being true or false, as confirming either way is valuable. Researching interesting things is good, and even bad studies are evidence that the topic is interesting. But they are not evidence that should be used for other kind of decision-making.
You are again splitting a hair about the meanings of words. The important thing is that they are advocating for making decisions without sufficient evidence, which is something I oppose. Their report is long and contains many AI risks, some of which (like deepfakes) have high-quality studies behind them, while others (like X-risks) do not. As a whole, the report “has some evidence” that there are risks associated with AI. So they talk about “limited evidence”. What is important is that they imply this “limited evidence” is not sufficient for making decisions.
Splitting a hair. You can call your evidence limited evidence if you want. It won’t get you a free pass that your argument should be considered. If it has too much uncertainty or doesn’t withstand scrutiny, it shouldn’t be taken in as evidence. Otherwise we end up in the creationist situation.