As far as I can tell, we don’t know of any principles that satisfy both (1) they guide our actions in all or at least most situations and (2) when taken seriously, they don’t lead to crazy town. So our options seem to be (a) don’t use principles to guide actions, (b) go to crazy town, or (c) use principles to guide actions but be willing to abandon them when the implications get too crazy.
I wish this post had less of a focus on utilitarianism and more on whether we should be doing (a), (b) or (c).
(I am probably not going to respond to comments about how specific principles satisfy (1) and (2) unless it actually seems plausible to me that they might satisfy (1) and (2).)
It’s not clear to me that (a), (b), and (c) are the only options—or rather, there are a bunch of different variants (c’), (c″), (c‴). Sure, you can say “use principles to guide action untilthey get too crazy’, but you can also say ‘use a multiplicity of principles to guide action until they conflict’, or ‘use a single principle to guide action until you run into cases where it is difficult to judge how the principle applies’, or so on. There are lots of different rules of thumb to tell you when and how principles run out, none of them entirely satisfactory.
And, importantly, there is no meta-principle that tells you when and how to apply object-level principles. Intuitively there’s an infinite regress argument for this claim, but since infinite regress arguments are notoriously unreliable I also tried to explain in more detail why it’s true in this post: any meta-principle would be vulnerable in the same way to the train to crazy town. And so, if you have the worries about the train to crazy town that are expressed in this post, you have to move away from the realm of pure rationalist moral philosophy and begin to use principles as context-specific guides, exercising particular judgments to figure out when they are relevant.
And so, in order to figure out when to use principles, you have to examine some specific principles in specific context, rather than trying very abstract philosophising to get the meta-principle. That was why I focussed so much of utilitarianism, and the specific context within which it is a useful way to think about ethics: I think this kind of specific analysis is the only way to figure out the limits of our principles. Your comment seems to assume that I could do some abstract philosophising to decide between (a), (b), (c), (c’), (c″), etc.; but for the very reasons discussed in this post, I don’t think that’s an option.
Why is criterion (1) something we want? Isn’t it enough to find principles that would guide us in some or many situations? Or even just in “many of the currently relevant-to-EA situations”?
I think there plausibly are principles that achieve (1) and (2), but they’ll give up either transitivity or the independence of irrelevant alternatives, and if used to guide actions locally without anticipating your own future decisions and without the ability to make precommitments, lead to plausibly irrational behaviour (and more than usual than just with known hard problems like Newcomb’s and Parfit’s hitchhiker). I don’t think those count as “crazy towns”, but they’re things people find undesirable or see as “inconsistent”. Also, they might require more arbitrariness than usual, e.g. picking thresholds, or nonlinear monotonically increasing functions.
Principles I have in mind (although they need to be extended or combined with others to achieve 1):
A non-fanatical approach to normative uncertainty or similar, but (possibly) instead of weighing different theories by subjective credences, you think of it as weighing principles or reasons by all-things-considered intuitive weights within a single theory. This interpretation makes most sense if you’re either a moral realist who treats ethics as fundamentally pluralistic or vague, or you’re a moral antirealist. However, again, this will probably lead to violations of transitivity or the independence of irrelevant alternatives to avoid fanaticism, unless you assume reasons are always bounded in strength, which rules out risk-neutral EV-maximizing total utilitarianism.
Also, I normally think of crazy town as requiring an act, rather than an omission, so distinguishing the two could help, although that might be unfair, and I suppose you can imagine forced choices.
As far as I can tell, we don’t know of any principles that satisfy both (1) they guide our actions in all or at least most situations and (2) when taken seriously, they don’t lead to crazy town. So our options seem to be (a) don’t use principles to guide actions, (b) go to crazy town, or (c) use principles to guide actions but be willing to abandon them when the implications get too crazy.
I wish this post had less of a focus on utilitarianism and more on whether we should be doing (a), (b) or (c).
(I am probably not going to respond to comments about how specific principles satisfy (1) and (2) unless it actually seems plausible to me that they might satisfy (1) and (2).)
Thanks for your comment.
It’s not clear to me that (a), (b), and (c) are the only options—or rather, there are a bunch of different variants (c’), (c″), (c‴). Sure, you can say “use principles to guide action until they get too crazy’, but you can also say ‘use a multiplicity of principles to guide action until they conflict’, or ‘use a single principle to guide action until you run into cases where it is difficult to judge how the principle applies’, or so on. There are lots of different rules of thumb to tell you when and how principles run out, none of them entirely satisfactory.
And, importantly, there is no meta-principle that tells you when and how to apply object-level principles. Intuitively there’s an infinite regress argument for this claim, but since infinite regress arguments are notoriously unreliable I also tried to explain in more detail why it’s true in this post: any meta-principle would be vulnerable in the same way to the train to crazy town. And so, if you have the worries about the train to crazy town that are expressed in this post, you have to move away from the realm of pure rationalist moral philosophy and begin to use principles as context-specific guides, exercising particular judgments to figure out when they are relevant.
And so, in order to figure out when to use principles, you have to examine some specific principles in specific context, rather than trying very abstract philosophising to get the meta-principle. That was why I focussed so much of utilitarianism, and the specific context within which it is a useful way to think about ethics: I think this kind of specific analysis is the only way to figure out the limits of our principles. Your comment seems to assume that I could do some abstract philosophising to decide between (a), (b), (c), (c’), (c″), etc.; but for the very reasons discussed in this post, I don’t think that’s an option.
Why is criterion (1) something we want? Isn’t it enough to find principles that would guide us in some or many situations? Or even just in “many of the currently relevant-to-EA situations”?
I think there plausibly are principles that achieve (1) and (2), but they’ll give up either transitivity or the independence of irrelevant alternatives, and if used to guide actions locally without anticipating your own future decisions and without the ability to make precommitments, lead to plausibly irrational behaviour (and more than usual than just with known hard problems like Newcomb’s and Parfit’s hitchhiker). I don’t think those count as “crazy towns”, but they’re things people find undesirable or see as “inconsistent”. Also, they might require more arbitrariness than usual, e.g. picking thresholds, or nonlinear monotonically increasing functions.
Principles I have in mind (although they need to be extended or combined with others to achieve 1):
Partial/limited aggregation, although I don’t know if they’re very well-developed, especially to handle uncertainty (and some extensions may have horrific crazy town counterexamples, like https://forum.effectivealtruism.org/posts/smgFKszHPLfoBEqmf/partial-aggregation-s-utility-monster). The Repugnant Conclusion and extensions (https://link.springer.com/article/10.1007/s00355-021-01321-2 ) can be avoided this way, and I think limiting and totally forbidding aggregation are basically the only ways to do so, but totally forbidding aggregation probably leads to crazy towns.
Difference-making risk aversion, to prevent fanaticism for an otherwise unbounded theory (objections of stochastic dominance and, if universalized, collective defeat here https://globalprioritiesinstitute.org/on-the-desire-to-make-a-difference-hilary-greaves-william-macaskill-andreas-mogensen-and-teruji-thomas-global-priorities-institute-university-of-oxford/ , https://www.youtube.com/watch?v=HT2w5jGCWG4 and https://forum.effectivealtruism.org/posts/QZujaLgPateuiHXDT/concerns-with-difference-making-risk-aversion , and some other objections and responses here https://forum.effectivealtruism.org/posts/sEnkD8sHP6pZztFc2/fanatical-eas-should-support-very-weird-projects?commentId=YqNWwzdpvmFbbXyoe#YqNWwzdpvmFbbXyoe ). There’s also a modification that never locally violates stochastic dominance, by replacing the outcome distributions with their quantile functions (i.e. sorting each outcome distribution statewise), but it’s also not implausible to me that we should just give up stochastic dominance and focus on statewise dominance, because stochastic dominance doesn’t guarantee conditional worseness, difference-making may be fundamentally a statewise concern, and stochastic dominance is fanatical with the right background noise when (deterministic) value is additive (https://www.journals.uchicago.edu/doi/10.1086/705555).
A non-fanatical approach to normative uncertainty or similar, but (possibly) instead of weighing different theories by subjective credences, you think of it as weighing principles or reasons by all-things-considered intuitive weights within a single theory. This interpretation makes most sense if you’re either a moral realist who treats ethics as fundamentally pluralistic or vague, or you’re a moral antirealist. However, again, this will probably lead to violations of transitivity or the independence of irrelevant alternatives to avoid fanaticism, unless you assume reasons are always bounded in strength, which rules out risk-neutral EV-maximizing total utilitarianism.
Also, I normally think of crazy town as requiring an act, rather than an omission, so distinguishing the two could help, although that might be unfair, and I suppose you can imagine forced choices.