I think I’ve got similar concerns and thoughts on this. I’m vaguely aware of various ideas for dealing with these issues, but I haven’t kept up with that, and I’m not sure how effective they are or will be in future.
The idea of making captcha requirements before things like commenting very widespread is one I haven’t heard before, and seems like it could plausibly cut off part of the problem at relatively low cost.
I would also quite like it if there were much better epistemic norms widespread across society, such as people feeling embarrassed if people point out they stated something non-obvious as a fact without referencing sources. (Whereas it could still be fine to state very obvious things as facts without sharing sources all the time, or to state non-obvious things as fairly confident conjectures rather than as facts.)
But some issues also come to mind (note: these are basically speculation, rather than drawing on research I’ve read):
It seems somewhat hard to draw the line between ok and not ok behaviours (e.g., what claims are self-evident enough that it’s ok to omit a source? What sort of tone and caveats are sufficient for various sorts of claims?)
And it’s therefore conceivable that these sorts of norms could be counterproductive in various ways. E.g., lead to (more) silencing or ridicule of people raising alarm bells about low probability high stakes events, because there’s not yet strong evidence about that, but no one will look for the evidence until someone starts raising the alarm bells.
Though I think there are some steps that seems obviously good, like requiring sources for specific statistical claims (e.g., “67% of teenagers are doing [whatever]”).
This is a sociological/psychological rather than technological fix, which does seem quite needed, but also seems quite hard to implement. Spreading norms like that widely seems hard to do.
With a lot of solutions, it seems not too hard to imagine ways they could be (at least partly) circumvented by people or groups who are actively trying to spread misinformation. (At least when those people/groups are quite well-resourced.)
E.g., even if society adopted a strong norm that people must include sources when making relatively specific, non-obvious claims, there could then perhaps be large-scale human- or AI-generated sources being produced, and made to look respectable at first glance, which can then be shared alongside the claims being made elsewhere.
We could probably also think of things like more generally improving critical thinking or rationality as similar broad, sociological approaches to mitigating the spread/impacts of misinformation. I’d guess that those more general approaches may better avoid the issue of difficulty drawing lines in the appropriate places and being circumventable by active efforts, but may suffer more strongly from being quite intractable or crowded. (But this is just a quick guess.)
One thing that you didn’t raise, but which seems related and important, is how advancements in certain AI capabilities could affect the impacts of misinformation. I find this concerning, especially in connection with the point you make with this statement:
warning about it wil increase mistrust and polarization, which might be the goal of the campaign
Early last year, shortly after learning about EA, I wrote a brief research proposal related to the combination of these points. I never pursued the research project, and have now learned of other problems I see as likely more important, but I still do think it’d be good for someone to pursue this sort of research. Here it is:
AI will likely allow for easier creation of fake news, videos, images, and audio (AI-generated misinformation; AIGM) [note: this is not an established term]. This may be hard to distinguish from genuine information. Researchers have begun exploring potential political security ramifications of this (e.g., Brundage et al., 2018). Such explorations could valuably draw on the literatures on the continued influence effect of misinformation (CIE; e.g., Lewandowsky, Ecker, Seifert, Schwarz, & Cook, 2012), motivated reasoning (e.g., Nyhan & Reifler, 2010), and the false balance effect (e.g., Koehler, 2016).
For example, CIE refers to the finding that corrections of misinformation don’t entirely eliminate the influence of that misinformation on beliefs and behaviours, even among people who remember and believe the corrections. For misinformation that aligns with one’s attitudes, corrections are particularly ineffective, and may even “backfire”, strengthening belief in the misinformation (Nyhan & Reifler, 2010). Thus, even if credible messages debunking AIGM can be rapidly disseminated, the misinformation’s impacts may linger or even be exacerbated. Furthermore, as the public becomes aware of the possibility or prevalence of AIGM, genuine information may be regularly argued to be fake. These arguments could themselves be subject to the CIE and motivated reasoning, with further and complicated ramifications.
Thus, it’d be valuable to conduct experiments exposing participants to various combinations of fake articles, fake images, fake videos, fake audio, and/or a correction of one or more of these. This misinformation could vary in how indistinguishable from genuine information it is; whether it was human- or AI-generated; and whether it supports, challenges, or is irrelevant to participants’ attitudes. Data should be gathered on participants’ beliefs, attitudes, and recall of the correction. This would aid in determining how much the issue of CIE is exacerbated by the addition of video, images, or audio; how it varies by the quality of the fake or whether it’s AI-generated; and how these things interact with motivated reasoning.
Such studies could include multiple rounds, some of which would use genuine rather than fake information. This could explore issues akin to false balance or motivated dismissal of genuine information. Such studies could also measure the effects of various “treatments”, such as explanations of AIGM capabilities or how to distinguish such misinformation from genuine information. Ideally, these studies would be complemented by opportunistic evaluations of authentic AIGM’s impacts.
One concern regarding this idea is that I’m unsure of the current capabilities of AI relevant to generating misinformation, and thus of what sorts of simulations or stimuli could be provided to participants. Thus, the study design sketched above is preliminary, to be updated as I learn more about relevant AI capabilities. Another concern is that relevant capabilities may currently be so inferior to how they’ll later be that discoveries regarding how people react to present AIGM would not generalise to their reactions to later, stronger AIGM.
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., … & Anderson, H. (2018). The malicious use of artificial intelligence: forecasting, prevention, and mitigation. arXiv preprint arXiv:1802.07228.
Lewandowsky, S., Ecker, U. K., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106-131.
Koehler, D. J. (2016). Can journalistic “false balance” distort public perception of consensus in expert opinion?. Journal of experimental psychology: Applied, 22(1), 24-38.
Nyhan, B., & Reifler, J. (2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303-330.
Unfortunately, I think these specific questions are mostly about stuff that people started talking about a lot more after 2017. (Or at least, I didn’t pick up on much writing and discussion about these points.) So it’s a bit beyond my area.
But I can offer some speculations and related thoughts, informed in a general sense by the things I did learn:
I suspect misinformation at least could be an “effective weapon” against countries or peoples, in the sense of causing them substantial damage.
I’d see (unfounded) conspiracy theories and smear campaigns as subtypes of spreading misinformation, rather than as something qualitatively different. But I think today’s technology allows for spreading misinformation (of any type) much more easily and rapidly than people could previously.
At the same time, today’s technology also makes flagging, fact-checking, and otherwise countering misinformation easier.
I’d wildly speculate that, overall, the general public are much better informed than they used to be, but that purposeful efforts to spread misinformation will more easily have major effects now than previously.
This is primarily based on the research I’ve seen (see my other comment on this post) that indicates that even warnings about misinfo and (correctly recalled!) corrections of misinfo won’t stop that misinfo having an effect.
But I don’t actually know of research that’s looked into this. We could perhaps call this question: How does the “offense-defense” balance of (mis)information spreading scale with better technology, more interconnectedness, etc.? (I take the phrase “offense-defense balance” from this paper, though it’s possible my usage here is not in line with what the phrase should mean.)
My understanding is that, in general, standard ways of counteracting misinfo (e.g., fact-checking, warnings) tend to be somewhat but not completely effective in countering misinfo. I expect this would be true for accidentally spread misinfo, misinfo spread deliberately by e.g. just a random troll, or misinfo spread deliberately by e.g. a major effort on the part of a rival country.
But I’d expect that the latter case would be one where the resources dedicated to spreading the misinfo will more likely overwhelm the resources dedicated towards counteracting it. So the misinfo may end up having more influence for that reason.
We could also perhaps wonder about how the “offense-defense” balance of (mis)information spreading scales with more resources. It seems plausible that, after a certain amount of resources dedicated by both sides, the public are just saturated with the misinfo to such an extent that fact-checking doesn’t help much anymore. But I don’t know of any actual research on that.
Thanks for adding those links, Jamie!
I’ve now added the first few into my lists above.
(Just FYI, your comment doesn’t seem to have a link to the podcast mentioned.)
1. To clarify, I don’t necessarily see status-adjusted welfare as a bad term. I’d actually say it seems pretty good, as it seems to state what it’s about fairly explicitly and intuitively.
I was just responding to the claim that it’s better than “moral weight” in that it sounds more agnostic between unitarian and hierarchical approaches. I see it as perhaps scoring worse than “moral weight” on that particular criterion, or about the same.
(But I also still think it means a somewhat different thing to “moral weight” anyway, as best I can tell.)
2. I’m not confident about whether Muehlhauser meant moral status or capacity for welfare, and would guess your interpretation is more accurate than my half-remembered interpretation. Though looking again at his post on the matter, I see this sentence:
This depends (among other things) on how much “moral weight” we give to the well-being of different kinds of moral patients.
This sounds to me most intuitively like it’s about adjusting a given unit of wellbeing/welfare by some factor that “we’re giving” them, which therefore sounds like moral status. But that’s just my reading of one sentence.
In any case, I think I poorly expressed what I actually meant, which was related to my third point: It seems like “status-adjusted welfare” is the product of moral status and welfare, whereas “moral weight” is either (a) some factor by which we adjust the welfare of a being, or (b) some factor that captures how intense the welfare levels of the being will tend to be (given particular experiences/events), or some mix of (a) and (b). So “moral weight” doesn’t seem to include the being’s actual welfare, and thus doesn’t seem to be a synonym for “status-adjusted welfare”.
(Incidentally, having to try to describe in the above paragraph what “moral weight” seems to mean has increased my inclination to mostly ditch that term and to stick with the “moral status vs capacity for welfare” distinction, as that does seem conceptually clearer.)
3. That makes sense to me.
Thanks for that clarification and that answer!
I don’t know of any serious contemporary philosopher who has denied that the conjunction of sentience and agency is sufficient for moral standing, though there are philosophers who deny that agency is sufficient and a small number who deny that sentience is sufficient.
But I don’t know anybody who holds that view. Do you?
I don’t (but I know very little about the area as a whole, so I’d wouldn’t update on that in particular).
I can see why, if practically no one holds that view, “even most theologians will agree that all sentient agents have moral standing”. I guess I asked my question because I interpreted the passage as saying that that followed logically from the prior statements alone, whereas it sounds like instead it follows given the prior statements plus a background empirical fact about theologians’ view.
I quite appreciate the way you’ve engaged with hierarchical approaches and ensured your conceptual framework was open to such approaches, even if you personally aren’t very sympathetic towards them.
That said, I think I can see how a reader might get the impression that you’re more sympathetic to such approaches than it sounds like you are. E.g., you write:
I have suggested that we should frame the value of interventions in terms of status-adjusted welfare. If we were to compare the value of an intervention that targeted pigs with an intervention that targeted silkworms, we should consider not only the amount of welfare to be gained but also the moral status of the creatures who would gain the welfare.
To me, this reads like you’re saying not just that we should have this terminology at hand, nor just that we should be ready to ignore the welfare of entities with 0 moral status, but also that we should adjust things by moral status that varies by degrees.
And as I mentioned in another comment, to me, the term “status-adjusted welfare” also gives that impression. (I’m not saying that’s actually the literal meaning of your claims or terms, just that I can see how one might come to that impression.)
I found this whole comment thread interesting.
I agree that many thoughtful people reject impartiality (the majority of human beings probably reject impartiality). But this is not necessarily a reason to think there may be a sound epistemic case not to completely reject partialism.
I think two broad (though not necessarily knock-down) arguments against (some version of) those claims are considerations of epistemic modesty/humility and moral uncertainty. More specifically, I see that as at least a reason why it’s useful to engage with the idea of non-impartial views, and to try to leave one’s conceptual framework open to such views.
(That said, I also think there’s clear value in sometimes having discussions that are just about one’s “independent impressions”—i.e., what one would believe without updating on the views of others. For example, that helps avoids information cascades. And I do personally share strong intuitions towards an impartial/unitarian approach.)
Nevertheless, I think most of us are committed to taking status-adjusted welfare seriously. If one is uncomfortable with degrees of moral status, unitarianism is a live option. Denying that any creatures have moral status, however, implies that there is no moral difference between harming a person and harming a coffee mug. But most of us feel there is a moral difference, and this difference is explained by the fact that the person has moral standing and the coffee mug does not.
I found I felt like I disagreed with this, and it was interesting to try to work out why, and how I’d look at things instead. Here’s what I came up with (which is meant as more like a report on my intuitive way of looking at things than a sound philosophical theory):
In essence, I’d naturally say one is simply not harming the coffee mug, because the coffee mug can’t be harmed. I wouldn’t naturally say that one is harming the coffee mug, but that this doesn’t matter because the coffee mug lacks some special property that would make its welfare matter.
To expand: That passage seems to assume that we have to look at things the unit of analysis being an “individual” of some sort, or an object or a being or whatever. Taking that perspective, for all individuals/objects/beings in the world, we determine whether they have moral status (or how much moral status they have), how much welfare they’re currently experiencing, how much we can change their welfare, etc., and we make moral judgements and decisions based on that. A coffee mug clearly doesn’t have moral status. If we rejected the idea of moral status, then we’d be committed to saying people also don’t have moral status, and thus that there’s no moral difference between harming a person and harming a coffee mug.
The way I think I want to look at things is using welfare itself as the unit of analysis. Any and all welfare matters. And each unit of welfare matters equally. It’s not that a coffee mug’s welfare doesn’t matter, but rather that it has no welfare, and one can’t affect its welfare. So damaging it doesn’t count as “harming” it in a morally relevant sense. Whereas humans can have welfare, so actions that affect their welfare matter morally.
Perhaps another way to put this is that I’d give each unit of welfare a moral status of 1. And wouldn’t give moral status to any experiencers of welfare.
That said, I think that this post’s way of describing things can essentially capture the outputs of this way of looking at things I have, while also capturing other moral theories and ways of looking at things. And that seems quite valuable, both for communication purposes and for reasons of moral uncertainty. (Also, I’m far from an expert in the relevant areas of philosophy, so there may be reasons why this way of looking at things is conceptually confused.)
Even before considering moral status, we can say that lives that contain more and more of non-instrumental goods are more valuable than lives that contain fewer and less of those non-instrumental goods.
Did this mean something like:
Even before considering moral status, we can say that lives that contain more types of non-instrumental goods, and/or more of each type, are more valuable than lives that contain fewer types of those non-instrumental goods, and/or less of each type.
Also, footnote 59 appears to be missing a quotation mark to open the quote.
The farming of cochineal may cause an additional 4.6 to 21 trillion deaths, primarily nymphs that do not survive to adulthood.
I assume this is the annual number of deaths?
Almost certainly, all sentient agents have moral standing. It’s likely that sentience is sufficient on its own for moral standing, though that view is just slightly more controversial.
I found that first sentence slightly surprising. That’d be my preferred stance, but I’d guess that a great many people would disagree. Though I don’t know how many people who’ve thought about it a lot disagree. I’d be interested to know whether this sentence like your own considered judgement, or a reflection of the consensus view among philosophers.
Or was that sentence actually meant to indicate that “Almost certainly, all beings with moral standing are sentient” (i.e., that sentience is almost certainly necessary, rather than sufficient, for moral standing)?
The theological-minded might prefer a view on which moral standing is grounded in the possession of a Cartesian soul. But on most such accounts, the possession of a Cartesian soul grants sentience or agency or both. So even most theologians will agree that all sentient agents have moral standing because they will thank that the class of moral agents is coextensive with the class of beings with Cartesian souls.
1. Was that last sentence meant to say they will think “that the class of moral patients is coextensive with the class of beings with Cartesian souls”?
2. It seems that one could believe that “the possession of a Cartesian soul grants sentience or agency or both”, but that there are also other ways of gaining sentience or agency or both, and thus that there may be sentient beings who aren’t moral patients (if possession of a Cartesian soul is required for moral patienthood). Was the second sentence meant to imply something like “the possession of a Cartesian soul is necessary for sentience or agency or both”?
Minor matter: Do you see a reason to prefer the term “welfare subject” and “moral standing” to “moral patient” and “moral patienthood”? For example, are the former terms more popular in the philosophical literature?
I see five potential perks of the latter pair of terms:
Their relationship to each other is obvious from the terms themselves (whereas with “welfare subject” and “moral standing”, you’d have to explain to someone new to the topic that there’s a relationship between those terms)
Their relationship with “moral agent”/”moral agency” seems more obvious from the terms themselves.
Compared to “moral standing”, “moral patient” seems less likely to end up getting confused with “moral status”
“moral patient” doesn’t have to take a stand on whether welfare is the only thing that’s non-instrumentally morally good (or whether it’s non-instrumentally morally good at all), whereas focusing on whether something is a “welfare subject” could arguably be seen as implying that.
Although in practice EAs probably will be focusing on welfare as the only non-instrumentally morally good thing, and I’m ok with that myself.
I feel a vague sense that “welfare” (and thus “welfare subject”) might sound to some people like it’s focusing on a hedonistic view of wellbeing, rather than on a desire-fulfilment or objective list view. But I could very well be wrong about that.
Thanks for this post—I found it very interesting and very clearly written and reasoned! I learned a lot, and have added it to my list of sources relevant to the idea of “moral weight”.
Another term that might be used to capture both moral status and capacity for welfare is ‘moral weight.’ Although ‘status-adjusted welfare’ isn’t a perfect term, I think ‘moral weight’ suffers from two problems. First, to my ear, it doesn’t sound agnostic between the hierarchical approach and the unitarian approach. One informal way of describing unitarianism is ‘the view that rejects moral weights.’
1. I found that a little confusing. To me, “status-adjusted welfare” sounds notably less agnostic than does “moral weight” regarding the hierarchical and unitarian approaches.
As you note, “Unitarians assign all creatures with moral standing the same moral status, so for the unitarian, status-adjusted welfare just collapses to welfare.” So if we’re choosing to use the term “status-adjusted welfare”, I think we sound like we’re endorsing the hierarchical view—even if in reality we want to be open to saying “It turns out moral status is equal between animals, so there’s no need for status-adjustment.”
Whereas if we’re choosing to use the term “moral weight”, I think we sound like we’re open to the hierarchical view, but we at least avoid making it sound like we’re actually planning to adjust things by moral weight.
Perhaps the reason you see “status-adjusted welfare” as sounding more agnostic is because you’re imagining the adjustment as potentially being a multiplication by 0, for beings that have no moral status, rather than by a number between 0 and 1? That didn’t come to mind intuitively for me, because then I think I’d just want to say the being has no welfare. But maybe that’s me deviating from how philosophers would usually think/talk about these matters.
2. The prior point may be related to the fact that, as best I can tell, moral weight and status-adjusted welfare aren’t really different terms for the same thing (which seemed to me to be what the first sentence of that quote was implying). At least based on how I’ve seen the term used (mainly by Muehlhauser), “moral weight” seems to mean pretty much just the “moral status” component—just the term we multiply welfare by in order to get the number we really care about, rather than that final number.
So it seems like the synonym for “status-adjusted welfare” would be not “moral weight” but “moral-weight-adjusted welfare”. And that, unlike just “moral weight”, does sound to me like it’s endorsing the hierarchical view.
3. Somewhat separate point, which I’m uncertain about: I’m not sure status-adjusted welfare really “captures” capacity for welfare. Given your description, it seems status-adjusted welfare is just about multiplying the welfare the being is actually at (or a given change in welfare or something like that) by the moral status of the being—without the being’s capacity for welfare playing a role.
Did you mean that status-adjusted welfare “captures” capacity for welfare to the extent that a lower or higher capacity for welfare will tend to reduce or increase the amount of welfare that is being experienced or changed?
I think what you’re saying makes sense to me, but I’m confused by the fact you say “I wrote some thoughts related to moral status (not specifically welfare capacity) and personal identity here”, but then the passage appears to be about moral agency, rather than about moral status/patienthood.
And then occasionally the passage appears to use moral agency as if it means moral status/patienthood. E.g., “Perhaps people are moral agents most of the time, but wouldn’t your account mean their suffering matters less in itself while they aren’t moral agents, even as normally developed adults”. Although perhaps that reflects the particular arguments that that passage of yours was responding to.
Could you clarify which concept you were talking about in that passage?
(It looks to me like essentially the same argument you make could hold in relation to moral status anyway, so I’m not saying this undermines your points.)
Just want to say that I’ve found this exchange quite interesting, and would be keen to read an adversarial collaboration between you two on this sort of thing. Seems like that would be a good addition to the set of discussions there’ve been about key cruxes related to AI safety/alignment.
(ETA: Actually, I’ve gone ahead and linked to this comment thread in that list as well, for now, as it was already quite interesting.)
Good point—added! (I’m actually surprised I didn’t think to check if they had a channel myself, given that I used one of their great videos in a couple presentations last year.)
Two more examples of how these sorts of findings can be applied to matters of interest to EAs:
Seth Baum has written a paper entitled Countering Superintelligence Misinformation drawing on this body of research. (I stumbled upon this recently and haven’t yet had a chance to read beyond the abstract and citations.)
In a comment, Jonas Vollmer applied ideas from this body of research to the matter of how best to handle interactions about EA with journalists
A bunch of exciting activity, as always!
Also cool to see two new channels for EA videos pop up (from Animal Ethics and WANBAM). I’ve now added those to my list of where to find EA-related videos.