Arguments for moral indefinability

Link post

Epistemic sta­tus: I en­dorse the core in­tu­itions be­hind this post, but am only mod­er­ately con­fi­dent in the spe­cific claims made. Also, while I do have a de­gree in philos­o­phy, I am not a pro­fes­sional ethi­cist, and I’d ap­pre­ci­ate feed­back on how these ideas re­late to ex­ist­ing liter­a­ture.

Mo­ral in­defin­abil­ity is the term I use for the idea that there is no eth­i­cal the­ory which pro­vides ac­cept­able solu­tions to all moral dilem­mas, and which also has the the­o­ret­i­cal virtues (such as sim­plic­ity, pre­ci­sion and non-ar­bi­trari­ness) that we cur­rently de­sire. I think this is an im­por­tant and true per­spec­tive on ethics, and in this post will ex­plain why I hold it, with the caveat that I’m fo­cus­ing more on airing these ideas than con­struct­ing a wa­ter­tight ar­gu­ment.

Here’s an­other way of ex­plain­ing moral in­defin­abil­ity: let’s think of eth­i­cal the­o­ries as pro­ce­dures which, in re­sponse to a moral claim, ei­ther en­dorse it, re­ject it, or do nei­ther. Mo­ral philos­o­phy is an at­tempt to find the the­ory whose an­swers best match our in­tu­itions about what an­swers eth­i­cal the­o­ries should give us (e.g. don’t cause un­nec­es­sary suffer­ing), and whose pro­ce­dure for gen­er­at­ing an­swers best matches our meta-level in­tu­itions about what eth­i­cal the­o­ries should look like (e.g. they should con­sis­tently ap­ply im­par­tial prin­ci­ples rather than us­ing ad-hoc, self­ish or ran­dom crite­ria). None of these desider­ata are fixed in stone, though—in par­tic­u­lar, we some­times change our in­tu­itions when it’s clear that the only the­o­ries which match those in­tu­itions vi­o­late our meta-level in­tu­itions. My claim is that even­tu­ally we will also need to change our meta-level in­tu­itions in im­por­tant ways, be­cause it will be­come clear that the only the­o­ries which match them vi­o­late key ob­ject-level in­tu­itions. In par­tic­u­lar, this might lead us to ac­cept the­o­ries which oc­ca­sion­ally evince prop­er­ties such as:

  • In­com­plete­ness: for some claim A, the the­ory nei­ther en­dorses nor re­jects ei­ther A or ~A, even though we be­lieve that the choice be­tween A and ~A is morally im­por­tant.

  • Vague­ness: the the­ory en­dorses an im­pre­cise claim A, but re­jects ev­ery way of mak­ing it pre­cise.

  • Con­tra­dic­tion: the the­ory en­dorses both A and ~A (note that this is a some­what provoca­tive way of fram­ing this prop­erty, since we can always add ar­bi­trary ad-hoc ex­cep­tions to re­move the con­tra­dic­tions. So per­haps a bet­ter term is ar­bi­trari­ness of scope: when we have both a strong ar­gu­ment for A and a strong ar­gu­ment for ~A, the the­ory can spec­ify in which situ­a­tions each con­clu­sion should ap­ply, based on crite­ria which we would con­sider ar­bi­trary and un­prin­ci­pled. Ex­am­ple: when there are fewer than N lives at stake, use one set of prin­ci­ples; oth­er­wise use a differ­ent set).

Why take moral in­defin­abil­ity se­ri­ously? The main rea­son is that ethics evolved to help us co­or­di­nate in our an­ces­tral en­vi­ron­ment, and did so not by giv­ing us a com­plete de­ci­sion pro­ce­dure to im­ple­ment, but rather by in­grain­ing in­tu­itive re­sponses to cer­tain types of events and situ­a­tions. There were many differ­ent and some­times con­tra­dic­tory se­lec­tion pres­sures driv­ing the for­ma­tion of these in­tu­itions—and so, when we con­struct gen­er­al­is­able prin­ci­ples based on our in­tu­itions, we shouldn’t ex­pect those prin­ci­ples to au­to­mat­i­cally give use­ful or even con­sis­tent an­swers to very novel prob­lems. Un­for­tu­nately, the moral dilem­mas which we grap­ple with to­day have in fact “scaled up” dras­ti­cally in at least two ways. Some are much greater in scope than any prob­lems hu­mans have dealt with un­til very re­cently. And some fea­ture much more ex­treme trade­offs than ever come up in our nor­mal lives, e.g. be­cause they have been con­structed as thought ex­per­i­ments to probe the edges of our prin­ci­ples.

Of course, we’re able to ad­just our prin­ci­ples so that we are more satis­fied with their perfor­mance on novel moral dilem­mas. But I claim that in some cases this comes at the cost of those prin­ci­ples con­flict­ing with the in­tu­itions which make sense on the scales of our nor­mal lives. And even when it’s pos­si­ble to avoid that, there may be many ways to make such ad­just­ments whose rel­a­tive mer­its are so di­vorced from our stan­dard moral in­tu­itions that we have no good rea­son to favour one over the other. I’ll give some ex­am­ples shortly.

A sec­ond rea­son to be­lieve in moral in­defin­abil­ity is the fact that hu­man con­cepts tend to be open tex­ture: there is of­ten no unique “cor­rect” way to rigor­ously define them. For ex­am­ple, we all know roughly what a table is, but it doesn’t seem like there’s an ob­jec­tive defi­ni­tion which gives us a sharp cut­off be­tween ta­bles and desks and benches and a chair that you eat off and a big flat rock on stilts. A less triv­ial ex­am­ple is our in­abil­ity to rigor­ously define what en­tities qual­ify as be­ing “al­ive”: edge cases in­clude viruses, fires, AIs and em­bryos. So when moral in­tu­itions are based on these sorts of con­cepts, try­ing to come up with an ex­act defi­ni­tion is prob­a­bly fu­tile. This is par­tic­u­larly true when it comes to very com­pli­cated sys­tems in which tiny de­tails mat­ter a lot to us—like hu­man brains and minds. It seems im­plau­si­ble that we’ll ever dis­cover pre­cise crite­ria for when some­one is ex­pe­rienc­ing con­tent­ment, or bore­dom, or many of the other ex­pe­riences that we find morally sig­nifi­cant.

I would guess that many anti-re­al­ists are sym­pa­thetic to the ar­gu­ments I’ve made above, but still be­lieve that we can make moral­ity pre­cise with­out chang­ing our meta-level in­tu­itions much—for ex­am­ple, by ground­ing our eth­i­cal be­liefs in what ideal­ised ver­sions of our­selves would agree with, af­ter long re­flec­tion. My main ob­jec­tion to this view is, broadly speak­ing, that there is no canon­i­cal “ideal­ised ver­sion” of a per­son, and differ­ent in­ter­pre­ta­tions of that term could lead to a very wide range of eth­i­cal be­liefs. I ex­plore this ob­jec­tion in much more de­tail in this post. (In fact, the more gen­eral idea that hu­mans aren’t re­ally “util­ity max­imisers”, even ap­prox­i­mately, is an­other good ar­gu­ment for moral in­defin­abil­ity.) And even if ideal­ised re­flec­tion is a co­her­ent con­cept, it sim­ply passes the buck to your ideal­ised self, who might then be­lieve my ar­gu­ments and de­cide to change their meta-level in­tu­itions.

So what are some pairs of moral in­tu­itions which might not be si­mul­ta­neously satis­fi­able un­der our cur­rent meta-level in­tu­itions? Here’s a non-ex­haus­tive list—the gen­eral pat­tern be­ing clashes be­tween small-scale per­spec­tives, large-scale per­spec­tives, and the meta-level in­tu­ition that they should be de­ter­mined by the same prin­ci­ples:

  • Per­son-af­fect­ing views ver­sus non-per­son-af­fect­ing views. Small-scale views: kil­ling chil­dren is ter­rible, but not hav­ing chil­dren is fine, even when those two op­tions lead to roughly the same out­come. Large-scale view: ex­tinc­tion is ter­rible, re­gard­less of whether it comes about from peo­ple dy­ing or peo­ple not be­ing born.

  • The mere ad­di­tion para­dox, aka the re­pug­nant con­clu­sion. Small-scale views: adding happy peo­ple and mak­ing peo­ple more equal can’t make things worse. Large-scale view: a world con­sist­ing only of peo­ple whose lives are barely worth liv­ing is deeply sub­op­ti­mal. (Note also Ar­rhe­nius’ im­pos­si­bil­ity the­o­rems, which show that you can’t avoid the re­pug­nant con­clu­sion with­out mak­ing even greater con­ces­sions).

  • Weigh­ing the­o­ries un­der moral un­cer­tainty. I per­son­ally find OpenPhil’s work on cause pri­ori­ti­sa­tion un­der moral un­cer­tainty very cool, and the fun­da­men­tal in­tu­itions be­hind it seem rea­son­able, but some of it (e.g. var­i­ance nor­mal­i­sa­tion) has reached a level of ab­strac­tion where I feel al­most no moral force from their ar­gu­ments, and aside from an in­stinct to­wards defin­abil­ity I’m not sure why I should care.

  • In­finite and rel­a­tivis­tic ethics. Same as above. See also this LessWrong post ar­gu­ing against ap­ply­ing the “lin­ear util­ity hy­poth­e­sis” at vast scales.

  • Whether we should force fu­ture gen­er­a­tions to have our val­ues. On one hand, we should be very glad that past gen­er­a­tions couldn’t do this. But on the other, the fu­ture will prob­a­bly dis­gust us, like our pre­sent would dis­gust our an­ces­tors. And along with “moral progress” there’ll also be value drift in ar­bi­trary ways—in fact, I don’t think there’s any clear dis­tinc­tion be­tween the two.

I sus­pect that many read­ers share my sense that it’ll be very difficult to re­solve all of the dilem­mas above in a satis­fac­tory way, but also have a meta-level in­tu­ition that they need to be re­solved some­how, be­cause it’s im­por­tant for moral the­o­ries to be defin­able. But per­haps at some point it’s this very urge to­wards defin­abil­ity which will turn out to be the weak­est link. I do take se­ri­ously Parfit’s idea that sec­u­lar ethics is still young, and there’s much progress yet to be made, but I don’t see any prin­ci­pled rea­son why we should be able to com­plete ethics, ex­cept by rais­ing fu­ture gen­er­a­tions with­out whichever moral in­tu­itions are stand­ing in the way of its com­ple­tion (and isn’t that a hor­rify­ing thought?). From an anti-re­al­ist per­spec­tive, I claim that per­pet­ual in­defin­abil­ity would be bet­ter. That may be a lit­tle more difficult to swal­low from a re­al­ist per­spec­tive, of course. My guess is that the core dis­agree­ment is whether moral claims are more like facts, or more like prefer­ences or tastes—if the lat­ter, moral in­defin­abil­ity would be analo­gous to the claim that there’s no (prin­ci­pled, sim­ple, etc) the­ory which speci­fies ex­actly which foods I en­joy.

There are two more plau­si­ble can­di­dates for moral in­defin­abil­ity which were the origi­nal in­spira­tion for this post, and which I think are some of the most im­por­tant ex­am­ples:

  • Whether to define welfare in terms of prefer­ence satis­fac­tion or he­do­nic states.

  • The prob­lem of “max­imi­sa­tion” in util­i­tar­i­anism.

I’ve been torn for some time over the first ques­tion, slowly shift­ing to­wards he­do­nic util­i­tar­i­anism as prob­lems with for­mal­is­ing prefer­ences piled up. While this isn’t the right place to enu­mer­ate those prob­lems (see here for a pre­vi­ous rele­vant post), I’ve now be­come per­suaded that any pre­cise defi­ni­tion of which prefer­ences it is morally good to satisfy will lead to con­clu­sions which I find un­ac­cept­able. After mak­ing this up­date, I can ei­ther re­ject a prefer­ence-based ac­count of welfare en­tirely (in favour of a he­do­nic ac­count), or else en­dorse a “vague” ver­sion of it which I think will never be speci­fied pre­cisely.

The former may seem the ob­vi­ous choice, un­til we take into ac­count the prob­lem of max­imi­sa­tion. Con­sider that a true (non-per­son-af­fect­ing) he­do­nic util­i­tar­ian would kill ev­ery­one who wasn’t max­i­mally happy if they could re­place them with peo­ple who were (see here for a com­pre­hen­sive dis­cus­sion of this ar­gu­ment). And that for any pre­cise defi­ni­tion of welfare, they would search for edge cases where they could push it to ex­treme val­ues. In fact, rea­son­ing about a “true util­i­tar­ian” feels re­mark­ably like rea­son­ing about an un­safe AGI. I don’t think that’s a co­in­ci­dence: psy­cholog­i­cally, hu­mans just aren’t built to be max­imisers, and so a true max­imiser would be fun­da­men­tally ad­ver­sar­ial. And yet many of us also have strong in­tu­itions that there are some good things, and it’s always bet­ter for there to be more good things, and it’s best if there are most good things.

How to rec­on­cile these prob­lems? My an­swer is that util­i­tar­i­anism is point­ing in the right di­rec­tion, which is “lots of good things”, and in gen­eral we can move in that di­rec­tion with­out mov­ing max­i­mally in that di­rec­tion. What are those good things? I use a vague con­cep­tion of welfare that bal­ances prefer­ences and he­do­nic ex­pe­riences and some of my own parochial crite­ria—im­por­tantly, with­out feel­ing like it’s nec­es­sary to find a perfect solu­tion (al­though of course there will be ways in which my cur­rent po­si­tion can be im­proved). In gen­eral, I think that we can of­ten do well enough with­out solv­ing fun­da­men­tal moral is­sues—see, for ex­am­ple, this LessWrong post ar­gu­ing that we’re un­likely to ever face the true re­pug­nant dilemma, be­cause of em­piri­cal facts about psy­chol­ogy.

To be clear, this still means that al­most ev­ery­one should fo­cus much more on util­i­tar­ian ideas, like the enor­mous value of the far fu­ture, be­cause in or­der to re­ject those ideas it seems like we’d need to sac­ri­fice im­por­tant ob­ject- or meta-level moral in­tu­itions to a much greater ex­tent than I ad­vo­cate above. We sim­ply shouldn’t rely on the idea that such value is pre­cisely defin­able, nor that we can ever iden­tify an eth­i­cal the­ory which meets all the crite­ria we care about.