EDIT: I gave this a mild rewrite about an hour after writing it to make a few points clearer. I notice I already got one strong disagreement. If anyone would like to offer a comment as well, I’m interested in disagreements, particularly around the value of statements of epistemic confidence. Perhaps they serve a purpose that I do not see? I’d like to know, if so.
Hmm. While I agree that it is helpful to include references for factual claims when it is in the author’s interest[1], I do not agree that inclusion of those references is necessarily useful to the reader.
For example, any topic about which the reader is unfamiliar or has a strongly held point of view is also one that likely has opposing points of view. While the reader might be interested in exploring the references for an author’s point of view, I think it would make more sense to put the responsibility on the reader to ask for those references than to force the author to presume a reader without knowledge or agreement with the author’s point of view.
Should the author be responsible for offering complete coverage of all arguments and detail their own decisions about what sources to trust or lines of argument to pursue? I think not. It’s not practical or needed.
However, what seems to be the narrative here is that, if an author does not supply references, the reader assumes nothing and moves on. After all, the author didn’t use good practices and the reader is busy. On the other hand, so the narrative goes, by offering a self-written statement of my epistemic confidence, I am letting you know how much to trust my statements whether I offer extensive citations or not.
The EA approach of putting the need for references on authors up-front (rather than by request) is not a good practice and neither is deferring to experts or discounting arguments simply because they are not from a recognized or claimed expert.
In the case of a scientific paper, if I am critiquing its claims, then yes, I will go through its citations. But I do so in order to gather information about the larger arguments or evidence for the paper’s claims, regardless of the credentials or expertise that the author claims. Furthermore, my rebuttal might include counterarguments with the same or other citations.
I can see this as valuable where I know that I (could) disagree with the author. Obviously, the burden is on me to collect the best evidence for the claims an author makes as well as the best evidence against them. If I want to “steelman” the author, as you folks put it, or refute the author’s claims definitively, then I will need to ask for citations from the author and collect additional information as well.
The whole point of providing citations up-front is to allow investigation of the author’s claims, but not to provide comfort that the claims are true or that I should trust the author. Furthermore, I don’t see any point to offering an epistemic confidence statement because such a statement contributes nothing to my trust in the author. However, EA folks seem to think that proxies for rigor and the reader’s epistemic confidence in the author are valid.[2]
With our easy access to:
scientific literature
think-tank distillations of research or informed opinion
public statements offered by scientists and experts on topics both inside and outside their areas of expertise
books (with good bibliographies) written about most topics
freely accessible journalism
thousands of forum posts on EA topics
and other sources
I don’t find it necessary to cite sources explicitly in forum posts, unless I particularly want to point readers to those sources. I know they can do the research themselves. If you want to steelman my arguments or make definitive arguments against them, then you should, is my view. It’s not my job to help you nor should I pretend that I am by offering you whatever supports my claims.
In some cases, you can’t provide much of the reasoning for your view, and it’s most transparent to simply say so.
Well, whether a person chooses not to offer their reasoning, or can’t offer their reasoning, you can conclude, if they don’t offer their reasoning, and you want to know what it is, that you should ask for it. After all, if I assert something you find disputable, why not dispute it? And the first step in disputing it is to ask for my reasoning. This is a much better approach than deciding whether you trust my presentation based on my write-up of my epistemic status.
For example, here is a plausible epistemic status for this comment:
a confident epistemic status: I spent several hours thinking over this comment’s content in particular. I thought it through after browsing more than 30 statements of epistemic status presented by EA folks in the last six months. I have more than a decade of experience judging internet posts and articles on their argumentation quality (not in any professional capacity) and have personally authored about 1500-2000 posts and comments (not tweets) on academic topics in the last 20 years. My background includes a year of training in formal and informal logic, more than a year of linguistics study, and about a year of personal study of AI and semantic networks topics and an additional (approximate) year of self-study of research methods, pragmatics and argumentation, all devoted to assertion deconstruction and development, and most applied in the context of responding to or writing forum/blog posts.
Actually, that epistemic status is accurate, but I don’t see it as relevant to my claims here. Plus how do you know its true? But suppose you thought it was true and felt reassured that my claims might have merit based on that epistemic status. I think you would be mistaken but I won’t argue it now. Interestingly, it does not appear representative of the sorts of epistemic status that I have actually read on this forum.
Here is an alternative epistemic status statement for my comment that looks like others on this forum:
a less-confident epistemic status: I only spent a few minutes thinking over my position before I decided to write it down[implying that I’m brash]. Plus I wrote it when I was really tired. I’m not an acknowledged expert on this topic, so my unqualified statements are not reliable. Also, I don’t know of anybody else saying this. Finally, I’m not sure if I can articulate why I think what I do, at least to meet your standards, which seem high on this forum [implying that I’m intimidated]. So I guess I’m not that confident about my position and don’t want you to agree with it easily, particularly since you’ve been using your approach for so long. I’m just trying to stimulate conversation with this comment.
It seems intended to acknowledge and validate potential arguments against it that are:
ad hominem: “[I’m brash and] I wrote this when I was really tired”
appeals to authority: “I’m not an acknowledged expert on this topic”
ad populum :”I don’t know of anybody else saying this”
sunk cost: “you’ve been using your approach for so long”
and the discussion of confidence is confusing. After acknowledging that I:
am brash and tired
didn’t spend long formulating my position
am not an expert
feel intimidated by your high standards
wouldn’t want you to reverse your position too quickly
I let you know that I’m not that confident in my assertions here. And then I go make them anyway with the stated goal that “I’m just trying to stimulate conversation.” Turned into a best practice as it is here, though, I see a different consequence for these statements of epistemic confidence.
What I have learned over my years on various forums (and blogs) includes:
an author will offer information that is appealing to readers or that helps the readers see the author in a better light.
Either of the epistemic status offerings I gave might convince different people to see me in a better light. The first might convince readers that I am knowledgeable and have strong and well-formed opinions. The second might convince readers that I am honest and humble and would like more information. The second also compliments the reader’s high standards.
readers that are good critical thinkers are aware of fallacies like ad hominem, ad populum, appeals to authority, and sunk cost. They also distrust flattery.
They know that to get at the truth, you need more information than is typically available in a post or comment. If they have a strong disagreement with an author, they have a forum to contest the author’s claims. If the author is really brash, tired, ill-informed, inarticulate, and has no reason for epistemic confidence in his claims , then the way to find out is not to take the author’s word for it, but to do your own research and then challenge the author’s claims directly. You wouldn’t want to encourage the author to validate irrelevant arguments against his claims by asking him for his epistemic confidence and his reasons for it. Even if the author chose on his own to hedge his claims with these irrelevant statements about his confidence, when you decide to steelman or refute the author’s argument, you will need different arguments than the four offered by the author (ad hominem, ad populum, appeals to authority, and sunk cost) .
I agree that an author should collect their data and offer reliable access to their sources, including quotes and page numbers for citations and archived copies, when that is in the author’s interest. In my experience, few people have the patience or time to go through your sources just to confirm your claims. However, if the information is important enough to warrant assessment for rigor, and you as the author care about that assessment, then yeah. Supply as much source material as possible as effectively as possible. And do a good job in the first place, that is, reach the right conclusions with your best researcher cap on. Help yourself out.
It should seem obvious (this must have come up at least a few times on the forum), but there’s a dark side to using citations and sources, and you’ll see it from think tanks, governments, and NGO’s, and here. The author will present a rigorous-looking but bogus argument. You have to actually examine the sources and go through them to see if they’re characterized correctly in the argument or if they are unbiased or representative of expert research in the topic area. Use of citations and availability of sources is not a proxy for correct conclusions or argument rigor, but authors commonly use just the access to sources and citations to help them persuade readers of their correctness and rigor, expecting that no one will follow up.
EDIT: I gave this a mild rewrite about an hour after writing it to make a few points clearer. I notice I already got one strong disagreement. If anyone would like to offer a comment as well, I’m interested in disagreements, particularly around the value of statements of epistemic confidence. Perhaps they serve a purpose that I do not see? I’d like to know, if so.
Hmm. While I agree that it is helpful to include references for factual claims when it is in the author’s interest[1], I do not agree that inclusion of those references is necessarily useful to the reader.
For example, any topic about which the reader is unfamiliar or has a strongly held point of view is also one that likely has opposing points of view. While the reader might be interested in exploring the references for an author’s point of view, I think it would make more sense to put the responsibility on the reader to ask for those references than to force the author to presume a reader without knowledge or agreement with the author’s point of view.
Should the author be responsible for offering complete coverage of all arguments and detail their own decisions about what sources to trust or lines of argument to pursue? I think not. It’s not practical or needed.
However, what seems to be the narrative here is that, if an author does not supply references, the reader assumes nothing and moves on. After all, the author didn’t use good practices and the reader is busy. On the other hand, so the narrative goes, by offering a self-written statement of my epistemic confidence, I am letting you know how much to trust my statements whether I offer extensive citations or not.
The EA approach of putting the need for references on authors up-front (rather than by request) is not a good practice and neither is deferring to experts or discounting arguments simply because they are not from a recognized or claimed expert.
In the case of a scientific paper, if I am critiquing its claims, then yes, I will go through its citations. But I do so in order to gather information about the larger arguments or evidence for the paper’s claims, regardless of the credentials or expertise that the author claims. Furthermore, my rebuttal might include counterarguments with the same or other citations.
I can see this as valuable where I know that I (could) disagree with the author. Obviously, the burden is on me to collect the best evidence for the claims an author makes as well as the best evidence against them. If I want to “steelman” the author, as you folks put it, or refute the author’s claims definitively, then I will need to ask for citations from the author and collect additional information as well.
The whole point of providing citations up-front is to allow investigation of the author’s claims, but not to provide comfort that the claims are true or that I should trust the author. Furthermore, I don’t see any point to offering an epistemic confidence statement because such a statement contributes nothing to my trust in the author. However, EA folks seem to think that proxies for rigor and the reader’s epistemic confidence in the author are valid.[2]
With our easy access to:
scientific literature
think-tank distillations of research or informed opinion
public statements offered by scientists and experts on topics both inside and outside their areas of expertise
books (with good bibliographies) written about most topics
freely accessible journalism
thousands of forum posts on EA topics
and other sources
I don’t find it necessary to cite sources explicitly in forum posts, unless I particularly want to point readers to those sources. I know they can do the research themselves. If you want to steelman my arguments or make definitive arguments against them, then you should, is my view. It’s not my job to help you nor should I pretend that I am by offering you whatever supports my claims.
Well, whether a person chooses not to offer their reasoning, or can’t offer their reasoning, you can conclude, if they don’t offer their reasoning, and you want to know what it is, that you should ask for it. After all, if I assert something you find disputable, why not dispute it? And the first step in disputing it is to ask for my reasoning. This is a much better approach than deciding whether you trust my presentation based on my write-up of my epistemic status.
For example, here is a plausible epistemic status for this comment:
a confident epistemic status: I spent several hours thinking over this comment’s content in particular. I thought it through after browsing more than 30 statements of epistemic status presented by EA folks in the last six months. I have more than a decade of experience judging internet posts and articles on their argumentation quality (not in any professional capacity) and have personally authored about 1500-2000 posts and comments (not tweets) on academic topics in the last 20 years. My background includes a year of training in formal and informal logic, more than a year of linguistics study, and about a year of personal study of AI and semantic networks topics and an additional (approximate) year of self-study of research methods, pragmatics and argumentation, all devoted to assertion deconstruction and development, and most applied in the context of responding to or writing forum/blog posts.
Actually, that epistemic status is accurate, but I don’t see it as relevant to my claims here. Plus how do you know its true? But suppose you thought it was true and felt reassured that my claims might have merit based on that epistemic status. I think you would be mistaken but I won’t argue it now. Interestingly, it does not appear representative of the sorts of epistemic status that I have actually read on this forum.
Here is an alternative epistemic status statement for my comment that looks like others on this forum:
a less-confident epistemic status: I only spent a few minutes thinking over my position before I decided to write it down[implying that I’m brash]. Plus I wrote it when I was really tired. I’m not an acknowledged expert on this topic, so my unqualified statements are not reliable. Also, I don’t know of anybody else saying this. Finally, I’m not sure if I can articulate why I think what I do, at least to meet your standards, which seem high on this forum [implying that I’m intimidated]. So I guess I’m not that confident about my position and don’t want you to agree with it easily, particularly since you’ve been using your approach for so long. I’m just trying to stimulate conversation with this comment.
It seems intended to acknowledge and validate potential arguments against it that are:
ad hominem: “[I’m brash and] I wrote this when I was really tired”
appeals to authority: “I’m not an acknowledged expert on this topic”
ad populum :”I don’t know of anybody else saying this”
sunk cost: “you’ve been using your approach for so long”
and the discussion of confidence is confusing. After acknowledging that I:
am brash and tired
didn’t spend long formulating my position
am not an expert
feel intimidated by your high standards
wouldn’t want you to reverse your position too quickly
I let you know that I’m not that confident in my assertions here. And then I go make them anyway with the stated goal that “I’m just trying to stimulate conversation.” Turned into a best practice as it is here, though, I see a different consequence for these statements of epistemic confidence.
What I have learned over my years on various forums (and blogs) includes:
an author will offer information that is appealing to readers or that helps the readers see the author in a better light.
Either of the epistemic status offerings I gave might convince different people to see me in a better light. The first might convince readers that I am knowledgeable and have strong and well-formed opinions. The second might convince readers that I am honest and humble and would like more information. The second also compliments the reader’s high standards.
readers that are good critical thinkers are aware of fallacies like ad hominem, ad populum, appeals to authority, and sunk cost. They also distrust flattery.
They know that to get at the truth, you need more information than is typically available in a post or comment. If they have a strong disagreement with an author, they have a forum to contest the author’s claims. If the author is really brash, tired, ill-informed, inarticulate, and has no reason for epistemic confidence in his claims , then the way to find out is not to take the author’s word for it, but to do your own research and then challenge the author’s claims directly. You wouldn’t want to encourage the author to validate irrelevant arguments against his claims by asking him for his epistemic confidence and his reasons for it. Even if the author chose on his own to hedge his claims with these irrelevant statements about his confidence, when you decide to steelman or refute the author’s argument, you will need different arguments than the four offered by the author (ad hominem, ad populum, appeals to authority, and sunk cost) .
I agree that an author should collect their data and offer reliable access to their sources, including quotes and page numbers for citations and archived copies, when that is in the author’s interest. In my experience, few people have the patience or time to go through your sources just to confirm your claims. However, if the information is important enough to warrant assessment for rigor, and you as the author care about that assessment, then yeah. Supply as much source material as possible as effectively as possible. And do a good job in the first place, that is, reach the right conclusions with your best researcher cap on. Help yourself out.
It should seem obvious (this must have come up at least a few times on the forum), but there’s a dark side to using citations and sources, and you’ll see it from think tanks, governments, and NGO’s, and here. The author will present a rigorous-looking but bogus argument. You have to actually examine the sources and go through them to see if they’re characterized correctly in the argument or if they are unbiased or representative of expert research in the topic area. Use of citations and availability of sources is not a proxy for correct conclusions or argument rigor, but authors commonly use just the access to sources and citations to help them persuade readers of their correctness and rigor, expecting that no one will follow up.