It seems clear to me that epistemic-status disclaimers don’t work for the purpose of mitigating the negative externalities of people saying wrong things, especially wrong things in domains where people naturally tend towards overconfidence (I have in mind anything that has political implications, broadly construed). This follows straightforwardly from the phenomenon of source amnesia, and anecdotally, there doesn’t seem to be much correlation between how much, say, Scott Alexander (whom I’m using here because his blog is widely read) hedges in the disclaimer of any given post and how widely that post winds up being cited later on.
Is there particular evidence for source amnesia you have in mind? The abstract for the first Wikipedia citation says:
Experiment 2 demonstrated that when normal subjects’ level of item recall was equivalent to that of amnesics, they exhibited significantly less source amnesia: Normals rarely failed to recollect that a retrieved item derived from either of the two sources, although they often forgot which of the two experimenters was the correct source. The results are discussed in terms of their implications for theories of normal and abnormal memory.
So I guess the question is whether the epistemic status disclaimer falls into the category of source info that people will remember (“an experimenter told me X”) or source info that people often forget (“Experimenter A told me X”). (Or whether it even makes sense to analyze epistemic status in the paradigm of source info at all—for example, including an epistemic status could cause readers to think “OK, these are just ideas to play with, not solid facts” when they read the post, and have the memory encoded that way, even if they aren’t able to explicitly recall a post’s epistemic status. And this might hold true regardless of how widespread a post is shared. Like, for all we know, certain posts get shared more because people like playing with new ideas more than they like reading established facts, but they’re pretty good at knowing that playing with new ideas is what they’re doing.)
I think if you fully buy into the source amnesia idea, that could be considered an argument for posting anything to the EA Forum which is above average quality relative to a typical EA information diet for that topic area—if you really believe this source amnesia thing, people end up taking Facebook posts just as seriously as papers they read on Google Scholar.
Epistemic status: During my psychology undergrad, I did a decent amount of reading on relevant topics, in particular under the broad label of the “continued influence effect” (CIE) of misinformation. My Honours thesis (adapted into this paper) also partially related to these topics. But I’m a bit rusty (my Honours was in 2017).
Information that initially is presumed to be correct, but that is later retracted or corrected, often continues to influence memory and reasoning. This occurs even if the retraction itself is well remembered.The present study investigated whether the continued influence of misinformation can be reduced by explicitly warning people at the outset that they may be misled. A specific warning—giving detailed information about the continued influence effect (CIE)--succeeded in reducing the continued reliance on outdated information but did not eliminate it. A more general warning—reminding people that facts are not always properly checked before information is disseminated—was even less effective. In an additional experiment, a specific warning was combined with the provision of a plausible alternative explanation for the retracted information. This combined manipulation further reduced the CIE but still failed to eliminate it altogether. (emphasis added)
This seems to me to suggest some value in including “epistemic status” messages up front, but that this don’t make it totally “safe” to make posts before having familiarised oneself with the literature and checked one’s claims.
From memory, this paper reviews research on CIE, and I perceived it to be high-quality and a good intro to the topic.
Here’s a couple other seemingly relevant quotes from papers I read back then:
“retractions [of misinformation] are less effective if the misinformation is congruent with a person’s relevant attitudes, in which case the retractions can even backfire [i.e., increase belief in the misinformation].” (source) (see also this source)
“we randomly assigned 320 undergraduate participants to read a news article presenting either claims both for/against an autism-vaccine link [a “false balance”], link claims only, no-link claims only or non-health-related information. Participants who read the balanced article were less certain that vaccines are safe, more likely to believe experts were less certain that vaccines are safe and less likely to have their future children vaccinated. Results suggest that balancing conflicting views of the autism-vaccine controversy may lead readers to erroneously infer the state of expert knowledge regarding vaccine safety and negatively impact vaccine intentions.” (emphasis added) (source)
This seems relevant to norms around “steelmanning” and explaining reasons why one’s own view may be inaccurate. Those overall seem like very good norms to me, especially given EAs typically write about issues where there truly is far less consensus than there is around things like the autism-vaccine “controversy” or climate change. But it does seem those norms could perhaps lead to overweighting of the counterarguments when they’re actually very weak, perhaps especially when communicating to wider publics who might read and consider posts less carefully than self-identifying EAs/rationalists would. But that’s all my own speculative generalisations of the findings on “falsely balanced” coverage.
I’ve been considering brushing up on this literature to write a post for the forum on how to balance risks of spreading misinformation/flawed ideas with norms among EAs and rationalists around things like just honestly contributing your views/data points to the general pool and trusting people will update on them only to the appropriate degree. Reactions to this comment with inform whether I decide investing time into that would be worthwhile.
Yeah, I should have known I’d get called out for not citing any sources. I’m honestly not sure I’d particularly believe most studies on this no matter what side they came out on; too many ways they could fail to generalize. I am pretty sure I’ve seen LW and SSC posts get cited as more authoritative than their epistemic-status disclaimers suggested, and that’s most of why I believe this; generalizability isn’t a concern here since we’re talking about basically the same context. Ironically, though, I can’t remember which posts. I’ll keep looking for examples.
Another thought is that even if the original post had a weak epistemic status, if the original post becomes popular and gets the chance to receive widespread scrutiny, which it survives, it could be reasonable to believe its “de facto” epistemic status is higher than what’s posted at the top. But yes, I guess in that case there’s the risk that none of the people who scrutinized it had familiarity with relevant literature that contradicted the post.
Maybe the solution is to hire someone to do lit reviews to carefully examine posts with epistemic status disclaimers that nonetheless became popular and seem decision relevant.
It seems clear to me that epistemic-status disclaimers don’t work for the purpose of mitigating the negative externalities of people saying wrong things, especially wrong things in domains where people naturally tend towards overconfidence (I have in mind anything that has political implications, broadly construed). This follows straightforwardly from the phenomenon of source amnesia, and anecdotally, there doesn’t seem to be much correlation between how much, say, Scott Alexander (whom I’m using here because his blog is widely read) hedges in the disclaimer of any given post and how widely that post winds up being cited later on.
Interesting thought, upvoted!
Is there particular evidence for source amnesia you have in mind? The abstract for the first Wikipedia citation says:
So I guess the question is whether the epistemic status disclaimer falls into the category of source info that people will remember (“an experimenter told me X”) or source info that people often forget (“Experimenter A told me X”). (Or whether it even makes sense to analyze epistemic status in the paradigm of source info at all—for example, including an epistemic status could cause readers to think “OK, these are just ideas to play with, not solid facts” when they read the post, and have the memory encoded that way, even if they aren’t able to explicitly recall a post’s epistemic status. And this might hold true regardless of how widespread a post is shared. Like, for all we know, certain posts get shared more because people like playing with new ideas more than they like reading established facts, but they’re pretty good at knowing that playing with new ideas is what they’re doing.)
I think if you fully buy into the source amnesia idea, that could be considered an argument for posting anything to the EA Forum which is above average quality relative to a typical EA information diet for that topic area—if you really believe this source amnesia thing, people end up taking Facebook posts just as seriously as papers they read on Google Scholar.
Epistemic status: During my psychology undergrad, I did a decent amount of reading on relevant topics, in particular under the broad label of the “continued influence effect” (CIE) of misinformation. My Honours thesis (adapted into this paper) also partially related to these topics. But I’m a bit rusty (my Honours was in 2017).
From this paper’s abstract:
This seems to me to suggest some value in including “epistemic status” messages up front, but that this don’t make it totally “safe” to make posts before having familiarised oneself with the literature and checked one’s claims.
From memory, this paper reviews research on CIE, and I perceived it to be high-quality and a good intro to the topic.
Here’s a couple other seemingly relevant quotes from papers I read back then:
“retractions [of misinformation] are less effective if the misinformation is congruent with a person’s relevant attitudes, in which case the retractions can even backfire [i.e., increase belief in the misinformation].” (source) (see also this source)
“we randomly assigned 320 undergraduate participants to read a news article presenting either claims both for/against an autism-vaccine link [a “false balance”], link claims only, no-link claims only or non-health-related information. Participants who read the balanced article were less certain that vaccines are safe, more likely to believe experts were less certain that vaccines are safe and less likely to have their future children vaccinated. Results suggest that balancing conflicting views of the autism-vaccine controversy may lead readers to erroneously infer the state of expert knowledge regarding vaccine safety and negatively impact vaccine intentions.” (emphasis added) (source)
This seems relevant to norms around “steelmanning” and explaining reasons why one’s own view may be inaccurate. Those overall seem like very good norms to me, especially given EAs typically write about issues where there truly is far less consensus than there is around things like the autism-vaccine “controversy” or climate change. But it does seem those norms could perhaps lead to overweighting of the counterarguments when they’re actually very weak, perhaps especially when communicating to wider publics who might read and consider posts less carefully than self-identifying EAs/rationalists would. But that’s all my own speculative generalisations of the findings on “falsely balanced” coverage.
I’ve been considering brushing up on this literature to write a post for the forum on how to balance risks of spreading misinformation/flawed ideas with norms among EAs and rationalists around things like just honestly contributing your views/data points to the general pool and trusting people will update on them only to the appropriate degree. Reactions to this comment with inform whether I decide investing time into that would be worthwhile.
Yeah, I should have known I’d get called out for not citing any sources. I’m honestly not sure I’d particularly believe most studies on this no matter what side they came out on; too many ways they could fail to generalize. I am pretty sure I’ve seen LW and SSC posts get cited as more authoritative than their epistemic-status disclaimers suggested, and that’s most of why I believe this; generalizability isn’t a concern here since we’re talking about basically the same context. Ironically, though, I can’t remember which posts. I’ll keep looking for examples.
Another thought is that even if the original post had a weak epistemic status, if the original post becomes popular and gets the chance to receive widespread scrutiny, which it survives, it could be reasonable to believe its “de facto” epistemic status is higher than what’s posted at the top. But yes, I guess in that case there’s the risk that none of the people who scrutinized it had familiarity with relevant literature that contradicted the post.
Maybe the solution is to hire someone to do lit reviews to carefully examine posts with epistemic status disclaimers that nonetheless became popular and seem decision relevant.