Academic philosopher, co-editor of utilitarianism.net, writes goodthoughts.blog
Richard Y Chappellđ¸
Karma: 7,344
Quick clarification: My target here is not so much people with radically different empirical beliefs (such that they regard vaccines as net-negative), but rather the particular form of status quo bias that I discuss in the original post.
My guess is that for relatively elite audiences (like those who read philosophy blogs), theyâre unlikely to feel attached to this status quo bias as part of their identity, but their default patterns of thought may lead them to (accidentally, as it were) give it more weight than it deserves. So a bit of heated rhetoric and stigmatization of the thought-pattern in question may help to better inoculate them against it.
(Just a guess though â I could be wrong!)
I think if some people are importantly right about something big, and others (esp. with more power) are importantly wrong, itâs worth cheerleading getting things right even if it happens to correlate with your in-group!
Interesting post! Re: âhow spotlight sizes should be chosenâ, I think a natural approach is to think about the relative priorities of representatives in a moral parliament. Take the meat eater problem, for example. Suppose you have some mental representatives of human interests, and some representatives of factory farmed animal interests. Then we can ask each representative: âHow high a priority is it for you to get your way on whether or not to prevent this child from dying of malaria?â The human representatives will naturally see this as a very high priorityâwe donât have many better options for saving human lives. But the animal representatives, even if they arenât thrilled by retaining another omnivore, have more pressing priorities than trying to help animals by eliminating meat-eaters one by one. Given how incredibly cost-effective animal-focused charities can be, it will make sense for them to make the moral trade: âOK, save this life, but then letâs donate more to the Animal Welfare Fund.â
Of course, for spotlighting to work out well for all representatives, itâs going to be important to actually follow through on supporting the (otherwise unopposed) top priorities of neglected representatives (like those for wild animal welfare). But I think the basic approach here does a decent job of capturing why it isnât intuitively appropriate to take animal interests into account when deciding whether to save a personâs life. In short: insofar as we want to take animal interests into account, there are better ways to do it, that donât require creating conflict with another representativeâs top priorities. Avoiding such suboptimal conflict, and instead being open to moral trade, seems an important part of being a âgood moral colleagueâ.
Funnily enough, the main example that springs to mind is the excessive self-flagellation post-FTX. Many distanced themselves from the community and its optimizing norms/âmindsetâfor understandable reasons, but ones more closely tied to âexpressingâ (and personal reputation management) than to actually âhelpingâ, IMO.
Iâd be curious to hear if others think of further candidate examples.
EA Infrastructure Fund or Giving What We Can? For the latter, âour best-guess giving multiplier for [2023-24] was approximately 6xâ.
I think itâs more like he disagrees with you about the relative strengths of the objections and responses. (fwiw, Iâm inclined to agree with him, and I donât have any personal stake in the matter.)
Any intellectual community will have (at least implicit) norms surrounding which assumptions /â approaches are regarded as:
(i) presumptively correct or eligible to treat as a starting premise for further argument; this is the community âorthodoxyâ.
(ii) most plausibly mistaken, but reasonable enough to be worth further consideration (i.e. valued critiques, welcomed âheterodoxyâ)
(iii) too misguided to be worth serious engagement.
It would obviously be a problem for an intellectual community if class (ii) were too narrow. Claims like âdissent isnât welcomeâ imply that (ii) is non-existent: your impression is that the only categories within EA culture are (i) and (iii). If that were true, I agree it would be bad. But reasoning from the mere existence of class (iii) to negative conclusions about community epistemics is far too hasty. Any intellectual community will have some things they regard as not worth engaging with. (Classic examples include, e.g., biologistsâ attitudes towards theistic alternatives to Darwinian evolution, or historiansâ attitudes towards various conspiracy theories.)
People with different views will naturally dispute which of these three categories any given contribution ideally ought to fall into. People donât tend to regard their own contributions as lacking intellectual worth, so if they experience a lack of engagement itâs very tempting to leap to the conclusion that others must be dogmatically dismissing them. Sometimes theyâre right! But not always. So itâs worth being aware of the âoutside viewâ that (a) some contributions may be reasonably ignored, and (b) anyone on the receiving end of this will subjectively experience it just as the OP describes, as seeming like dogmatic/âunreasonable dismissal.
Given the unreliability of personal subjective impressions on this issue, itâs an interesting question what more-reliable evidence one could look for to try to determine whether any given instance of non-engagement (and/âor wider community patterns of dis/âengagement) is objectively reasonable or not. Seems like quite a tricky issue in social epistemology!
Thanks for looking into it more!
Iâm not seeing the barrier to Person Aâs thinking thereâs a 1/â1000 chance, conditional on reaching the 50th century, of going extinct in that century. We could easily expect to survive 50 centuries at that rate, and then have the risk consistently decay (halving each century, or something like that) beyond that point, right?
If you instead mean to invoke, say, the 50 millionth century, then Iâd think itâs crazy on its face to suddenly expect a 1/â1000 chance of extinction after surviving so long. That would no longer âseem, on the face of it, credibleâ.
Am I missing something?
Thanks, yeah, I like your point there that âfalse negatives are costlier than false positives in this caseâ, and so even <50% credence can warrant significant action. (I wouldnât literally say we should âact as if 3H is trueâ in all respectsâas per Nunoâs comment, uncertainty may justify some compounding âpatient philanthropyâ, which could have high stakes if the hinge comes later. But thatâs a minor quibble: I take myself to be broadly in agreement with your larger gist.)
My main puzzlement there is how you could think that you ought to perform an act that you simultaneously ought to hope that you fail to perform, subsequently (and predictably) regret performing, etc. (I assume here that all-things-considered preferences are not cognitively isolated, but have implications for other attitudes like hope and regret.) It seems like thereâs a kind of incoherence in that combination of attitudes, that undermines the normative authority of the original âoughtâ claim. We should expect genuinely authoritative oughts to be more wholeheartedly endorsable.
Right, so one crucial clarification is that weâre talking about act-inclusive states of affairs, not mere âoutcomesâ considered in abstraction from how they were brought about. Deontologists certainly donât think that we can get far merely thinking about the latter, but if they assess an action positively then it seems natural enough to take them to be committed to the actionâs actually being performed (all things considered, including what follows from it). Iâve written about this more in Deontology and Preferability. A key passage:
If you think that other things besides impartial value (e.g. deontic constraints) truly matter, then you presumably think that moral agents ought to care about more than just impartial value, and thus sometimes should prefer a less-valuable outcome over a more-valuable one, on the basis of these further considerations. Deontologists are free to have, and to recommend, deontologically-flavored preferences. The basic concept of preferability is theory-neutral on its face, begging no questions.
Thanks! You might like my post, âAxiology, Deontics, and the Telic Questionâ which suggests a reframing of ethical theory that avoids the common error. (In short: distinguish ideal preferability vs instrumental reasoning /â decision theory rather than axiology vs deontics.)
I wonder if it might also help address Mogensenâs challenge. Full aggregation seems plausibly true of preferability not just axiology. But then given principles of instrumental rationality linking reasons for preference/âdesire to reasons for action, itâs hard to see how full aggregation couldnât also be true with regard to choiceworthiness. (But maybe heâd deny my initial claim about preferability?)
To be clear: Iâd be excited for more people to look into these claims! Seems worth investigating. But itâs not my comparative advantage.
Sorry, I donât think I have relevant expertise to assess such empirical claims (which is why I focus more on hypotheticals). It would certainly be convenient if helping people turned out to be the best way to also reduce non-human suffering! And it could be true (I donât take convenience to be an automatic debunker or anything). I just have no idea.
Thanks for your reply! Working backwards...
On your last point, Iâm fully on board with strictly decoupling intrinsic vs instrumental questions (see, e.g., my post distinguishing telic vs decision-theoretic questions). Rather, it seems we just have very different views about what telic ends or priorities are plausible. I give ~zero credence to pro-annihilationist views on which itâs preferable for the world to end than for any (even broadly utopian) future to obtain that includes severe suffering as a component. Such pro-annihilationist lexicality strikes me as a non-starter, at the most intrinsic/âfundamental/âprincipled levels. By contrast, I could imagine some more complex variable-value/âthreshold approach to lexicality turning out to have at least some credibility (even if Iâm overall more inclined to think that the sorts of intuitions youâre drawing upon are better captured at the âinstrumental heuristicâ level).
On moral uncertainty: I agree that bargaining-style approaches seem better than âmaximizing expected choiceworthinessâ approaches. But then if you have over 50% credence in a pro-annihilationist view, it seems like the majority rule is going to straightforwardly win out when it comes to determining your all-things-considered preference regarding the prospect of annihilation.
Re: uncompensable monster: It isnât true that âorthodox utilitarianism also endorses this in principleâ, because a key part of the case description was âno matter what else happens to anyone elseâ. Orthodox consequentialism allows that any good or bad can be outweighed by what happens to others (assuming strictly finite values). No one person or interest can ever claim to settle what should be done no matter what happens to others. Itâs strictly anti-absolutist in this sense, and I think thatâs a theoretically plausible and desirable property that your view is missing.
Another way to flip the âforceâ issue would be âsuppose a society concludes unanimously including via some extremely deliberative process (that predicts and includes the preferences of potential future people) that annihilation is good and desired. Should some outside observer forcibly prevent them taking action to this end (assume that the observer is interested purely in ethics and doesnât care about their own existence or have valenced experience)?â
I donât think itâs helpful to focus on external agents imposing their will on others, because thatâs going to trigger all kinds of instrumental heuristic norms against that sort of thing. Similarly, one might have some concerns about there being some moral cost to the future not going how humanity collectively wants it to. Better to just consider natural causes, and/âor comparisons of alternative possible societal preferences. Here are some possible futures:
(A) Society unanimously endorses your view and agrees that, even though their future looks positive in traditional utilitarian terms, annihilation would be preferable.
(A1): A quantum-freak black hole then envelops the Earth without anyone suffering (or even noticing).
(A2): After the present generations stop reproducing and go extinct, a freak accident in a biolab creates new human beings who go on to repopulate the Earth (creating a future similar to the positive-but-imperfect one that previous generations had anticipated but rejected).
(B) Society unanimously endorses my view and agrees that, even though existence entails some severe suffering, it is compensable and the future overall looks extremely bright.
(B1): A quantum-freak black hole then envelops the Earth without anyone suffering (or even noticing).
(B2): The broadly-utopian (but imperfect) future unfolds as anticipated.
Intuitively: B2 > A2 > A1 > B1.
I think it would be extremely strange to think that B1 > B2, or that A1 > B2. In fact, I think those verdicts are instantly disqualifying: any view yielding those verdicts deserves near-zero credence.
(I think A1 is broadly similar to, though admittedly not quite as bad as, a scenario C1 in which everyone decides that they deserve to suffer and should be tortured to death, and then some very painful natural disaster occurs which basically tortures everyone to death. It would be even worse if people didnât want it, but wanting it doesnât make it good.)
Regarding the âworld-destructionâ reductio:
this isnât strong evidence against the underlying truth of suffering-focused views. Consider scenarios where the only options are (1) a thousand people tortured forever with no positive wellbeing whatsoever or (2) painless annihilation of all sentience. Annihilation seems obviously preferable.
I agree that itâs obviously true that annihilation is preferable to some outcomes. I understand the objection as being more specific, targeting claims like:
(Ideal): annihilation is ideally desirable in the sense that itâs better (in expectation) than any other remotely realistic alternative, including <detail broadly utopian vision here>. (After all, continued existence always has some chance of resulting in some uncompensable suffering at some point.)
or
(Uncompensable Monster): one being suffering uncompensable suffering at any point in history suffices to render the entire universe net-negative or undesirable on net, no matter what else happens to anyone else. We must all (when judging from an impartial point of view) regret the totality of existence.
These strike me as extremely incredible claims, and I donât think that most of the proposed âmoderating factorsâ do much to soften the blow.
I grant your âvirtual impossibilityâ point that annihilation is not really an available option (to us, at least; future SAI might be another matter). But the objection is to the plausibility of the in principle verdicts entailed here, much as I would object to an account of the harm of death that implies that it would do no harm to kill me in my sleep (the force of which objection would not be undermined by my actually being invincible).
Moral uncertainty might help if it resulted in the verdict that you all things considered should prefer positive-utilitarian futures (no matter their uncompensable suffering) over annihilation. But Iâm not quite sure how moral uncertainty could deliver that verdict if you really regard the suffering as uncompensable. How could a lower degree of credence in ordinary positive goods rationally outweigh a higher degree of credence in uncompensable bads? It seems like youâd instead need to give enough credence to something even worse: e.g. violating an extreme deontic constraint against annihilation. But thatâs very hard to credit, given the above-quoted case where annihilation is âobviously preferableâ.)
The âirreversibilityâ consideration does seem stronger here, but I think ultimately rests on a much more radical form of moral uncertainty: itâs not just that you should give some (minority) weight to other views, but that you should give significant weight to the possibility that a more ideally rational agent would give almost no weight to such a pro-annihilationist view as this. Some kind of anti-hubris norm along these lines should probably take priority over all of our first-order views. Iâm not sure what the best full development of the idea would look like, though. (It seems pretty different from ordinary treatments of moral uncertainty!) Pointers to related discussion would be welcome!
I think a more promising form of suffering-focused ethics would explore some form of âvariable valueâ approach, which avoids annihilationism in principle by allowing harms to be compensated (by sufficient benefits) when the alternative is no population at all, but introduces variable thresholds for various harms being specifically uncompensable by extra benefits beyond those basic thresholds. Iâm not sure whether a view of this structure could be made to work, but it seems more worth exploring than pro-annihilationist principles.
Isnât the point of the Long Reflection to avoid âlocking inâ irreversible mistakes? Extinction, for example, is irreversible. But large population isnât. So I donât actually see any sense in which present âmin-natalismâ maintains more future âoptionalityâ (or better minimizes moral risks) than pro-natalism. Both leave entirely open what future generations choose to do. They just differ in our present population target. And presently aiming for a âminimal populationâ strikes me as much the worse and riskier of the two options, for both intrinsic moral reasons and instrumental ones like misjudging /â undershooting the minimally sustainable level.
Thanks! Iâd previously found it a bit stressful deciding which posts were relevant enough to share here, so I ended up outsourcing the decision to the good folks on the Forum team (who also take care of the cross-posting). Accordingly, a good share of the appreciation is owed to them! :-)
Are you presupposing that good practical reasoning involves (i) trying to picture the most-likely future, and then (ii) doing what would be best in that event (while ignoring other credible possibilities, no matter their higher stakes)?
It would be interesting to read a post where someone tries to explicitly argue for a general principle of ignoring credible risks in order to slightly improve most-probable outcomes. Seems like such a principle would be pretty disastrous if applied universally (e.g. to aviation safety, nuclear safety, and all kinds of insurance), but maybe thereâs more to be said? But itâs a bit frustrating to read takes where people just seem to presuppose some such anti-precautionary principle in the background.
To be clear: I take the decision-relevant background question here to not be the binary question Is AGI imminent? but rather something more degreed, like Is there a sufficient chance of imminent AGI to warrant precautionary measures? And I donât see how the AI bubble popping would imply that answering âYesâ to the latter was in any way unreasonable. (A bit like how you canât say an election forecaster did a bad job just because their 40% candidate won rather than the one they gave a 60% chance to. Sometimes seeing the actual outcome seems to make people worse at evaluating othersâ forecasts.)
Some supporters of AI Safety may overestimate the imminence of AGI. Itâs not clear to me how much of a problem that is? (Many people overestimate risks from climate change. That seems important to correct if it leads them to, e.g., anti-natalism, or to misallocate their resources. But if it just leads them to pollute less, then it doesnât seem so bad, and Iâd be inclined to worry more about climate change denialism. Similarly, I think, for AI risk.) There are a lot more people who persist in dismissing AI risk in a way that strikes me as outrageously reckless and unreasonable, and so that seems by far the more important epistemic error to guard against?
That said, Iâd like to see more people with conflicting views about AGI imminence arrange public bets on the topic. (Better calibration efforts are welcome. Iâm just very dubious of the OPâs apparent assumption that losing such a bet ought to trigger deep âsoul-searchingâ. Itâs just not that easy to resolve deep disagreements about what priors /â epistemic practices are reasonable.)