You mention this already, but I want to emphasize how much different framings actually matter. This surprised me the most when working on this paper. I’d thus caution anyone against making strong inferences from just one such study.
For example, we conducted the following pilot study (n = 101) where participants were randomly assigned to two different conditions: i) create a new happy person, and ii) create a new unhappy person. See the vignette below:
Imagine there was a magical machine. This machine can create a new adult person. This new person’s life, however, would definitely [not] be worth living. They would be very unhappy [happy] and live a life full of suffering and misery [bliss and joy].
You can push a button that would create this new person.
Morally speaking, how good or bad would it be to push that button?
The response scale ranged from 1 = Extremely bad to 7 = Extremely good.
Creating a happy person was rated as only marginally better than neutral (mean = 4.4), whereas creating an unhappy person was rated as extremely bad (mean = 1.4). So this would lead one to believe that there is strong popular support for the asymmetry. [1]
However, those results were most likely due to the magical machine framing and/or the “push-a-button” framing. Even though these framings clearly “shouldn’t” make such a huge difference.
All in all, we tested many different framings, too many to discuss here. Occasionally, there were significant differences between framings that shouldn’t matter (though we also observed many regularities). For example, we had one pilot with the “multiplier framing”:
Suppose the world contains 1,000 people in total. How many times bigger would the number of extremely happy people have to be than the number of extremely unhappy people for you to think that this world is overall positive rather than negative (i.e., so that it would be better for the world to exist rather than not exist)?
Here, the median trade ratio was 8.5 compared to the median trade ratio of 3-4 that we find in our default framing. It’s clear that the multiplier framing shouldn’t make any difference from a philosophical perspective.
So seemingly irrelevant or unimportant changes in framings (unimportant at least from a consequentialist perspective) sometimes could lead to substantial changes in median trade ratios.
However, changes in the intensity of the experienced happiness and suffering—which is arguably the most important aspect of the whole thought experiment—affected the trade ratios considerably less than the above mentioned multiplier framing.
To see this, it’s worth looking closely at the results of study 1b. Participants were first presented with the following scale:
Let’s assume a happiness scale ranging from −100 (extreme unhappiness) to 0 (neutral) to +100 (extreme happiness). Someone on level 0 is in a neutral state that feels neither good nor bad. Someone on level −1 experiences a very mild form of unhappiness, only slightly worse than being in a neutral state. Someone on level +1 experiences a very mild form of happiness, only slightly better than being in a neutral state. Someone on level −100 experiences the absolute worst form of suffering imaginable. Someone on level +100 experiences the absolute best form of bliss imaginable.
[Editor’s note: From now on, the text is becoming more, um, expressive.]
Note that “worst form of suffering imaginable” is pretty darn bad. Being brutally tortured while kept alive by nano bots is more like −90 on this scale. Likewise, “absolute best form of bliss imaginable” is pretty far out there. Feeling, all your life, like you just created friendly AGI and found your soulmate, while being high on ecstasy would still not be +100.
(Note that we also conducted a pilot study where we used more concrete and explicit descriptions such as “torture”, “falling in love”, “mild headaches”, and “good meal” to describe the feelings of mild or extreme [un]happiness. The results were similar.)
Afterwards, participants were asked:
Given this information, what percentage of extremely [mildly] happy people vs. extremely [mildly] unhappy people would there have to be for you to think that this world is overall positive rather than negative (i.e., so that it would be better for the world to exist rather than not exist)? In my view, the percentage [...] would need to be as follows: X% extremely [mildly] happy people; Y% extremely [mildly] unhappy people.
So how do the MTurkers approach these awe-inspiring intensities?
First, extreme happiness vs. extreme unhappiness. MTurkers think that there need to exist at least 72% people experiencing the absolute best form of bliss imaginable in order to outweigh the suffering of 28% of people experiencing the worst form of suffering imaginable.
Toby Ord and the classical utilitarians rejoice, that’s not bad! That’s like a 3:1 trade ratio, pretty close to a 1:1 trade ratio! “And don’t forget that people’s imagination is likely biased towards negativity for evolutionary reasons!”, Carl Shulman says. “In humans, the pleasure of orgasm may be less than the pain of deadly injury, since death is a much larger loss of reproductive success than a single sex act is a gain.” Everyone nods in agreement with the Shulmaster.
How about extreme happiness vs. mild unhappiness? MTurkers say that there need to exist at least 62% of people experiencing the absolute best form of bliss imaginable in order to outweigh the extremely mild suffering of unhappy people (e.g., people who are stubbing their toes a bit too often for their liking). Brian Tomasik and the suffering-focused crowd rejoice, a 1.5 : 1 trade ratio for practically hedonium to mild suffering?! There is no way the expected value of the future is that good. Reducing s-risks is common sense after all!
How about mild happiness vs. extreme unhappiness? The MTurkers have spoken: A world in which 82% of people experience extremely mild happiness—i.e., eating particularly bland potatoes and listening to muzak without one’s hearing aids on—and 18% of people are brutally tortured while being kept alive by nano bots, is… net positive.
“Wait, that’s a trade ratio of 4.5:1 !” Toby says. “How on Earth is this compatible with a trade ratio of 3:1 for practically hedonium vs. highly optimized suffering, let alone a trade ratio of 1.5:1 for practically hedonium vs. stubbing your toes occasionally!” Carl screams. He looks at Brian but Brian has already fainted.
Toby, Carl and Brian meet the next day, still looking very pale. They shake hands and agree to not do so much descriptive ethics anymore.
Years later, all three still cannot stop wincing with pain when “the Long Reflection” is mentioned.
We also had two conditions about preventing the creation of a happy [unhappy] person. Preventing a happy person from being created (mean = 3.1) was rated as somewhat bad. Preventing an unhappy person (mean = 5.5) from being created was rated as fairly good.
Toby, Carl and Brian meet the next day, still looking very pale. They shake hands and agree to not do so much descriptive ethics anymore.
Garbage answers to verbal elicitations on such questions (and real life decisions that require such explicit reasoning without feedback/experience, like retirement savings) are actually quite central to my views. In particular, my reliance on situations where it is easier for individuals to experience things multiple times in easy-to-process fashion and then form a behavioral response. I would be much less sanguine about error theories regarding such utterances if we didn’t also see people in surveys saying they would rather take $1000 than a 15% chance of $1M, or $100 now rather than $140 a year later, i.e. utterances that are clearly mistakes.
Looking at the literature on antiaggregationist views, and the complete conflict of those moral intuitions with personal choices and self-concerned practice (e.g. driving cars or walking outside) is also important to my thinking. No-tradeoffs views are much more appealing outside our own domains of rich experience in talk.
situations where it is easier for individuals to experience things multiple times in easy-to-process fashion and then form a behavioral response
It’s not obvious to me that our ethical evaluation should match with the way our brains add up good and bad past experiences at the moment of deciding whether to do more of something. For example, imagine that someone loves to do extreme sports. One day, he has a severe accident and feels so much pain that he, in the moment, wishes he had never done extreme sports or maybe even wishes he had never been born. After a few months in recovery, the severity of those agonizing memories fades, and the temptation to do the sports returns, so he starts doing extreme sports again. At that future point in time, his brain has implicitly made a decision that the enjoyment outweighs the risk of severe suffering. But our ethical evaluation doesn’t have to match how the evolved emotional brain adds things up at that moment in time. We might think that, ethically, the version of the person who was in extreme pain isn’t compensated by other moments of the same person having fun.
Even if we think enjoyment can outweigh severe suffering within a life, many people object to extending such tradeoffs across lives, when one person is severely harmed for the benefit of others. The examples in David’s comment were about interpersonal tradeoffs, rather than intrapersonal ones. It’s true that people impose small risks of extreme suffering on some for the happiness of others all the time, like in the case of driving purely for leisure, but that still begs the question of whether we should do that. Most people in the West also eat chickens, but they shouldn’t. (Cases like driving are also complicated by instrumental considerations, as Magnus would likely point out. Also, not driving for leisure might itself cause some people nontrivial levels of suffering, such as by worsening mental-health problems.)
I agree that preferences at different times and different subsystems can conflict. In particular, high discounting of the future can lead to forgoing a ton of positive reward or accepting lots of negative reward in the future in exchange for some short-term change. This is one reason to pay extra attention to cases of near-simultaneous comparisons, or at least to look at different arrangements of temporal ordering. But still the tradeoffs people make for themselves with a lot of experience under good conditions look better than what they tend to impose on others casually. [Also we can better trust people’s self-benevolence than their benevolence towards others, e.g. factory farming as you mention.]
And the brain machinery for processing stimuli into decisions and preferences does seem very relevant to me at least, since that’s a primary source of intuitive assessments of these psychological states as having value, and for comparisons where we can make them. Strong rejection of interpersonal comparisons is also used to argue that relieving one or more pains can’t compensate for losses to another individual.
I agree the hardest cases for making any kind of interpersonal comparison will be for minds with different architectural setups and conflicting univocal viewpoints, e.g. 2 minds with equally passionate complete enthusiasm (with no contrary psychological processes or internal currencies to provide reference points) respectively for and against their own experience, or gratitude and anger for their birth (past or future). They can respectively consider a world with and without their existences completely unbearable and beyond compensation. But if we’re in the business of helping others for their own sakes rather than ours, I don’t see the case for excluding either one’s concern from our moral circle.
Now, one can take take a more nihilistic/personal aesthetics view of morality, and say that one doesn’t personally care about the gratitude of minds happy to exist. I take it this is more your meta-ethical stance around these things? There are good arguments for moral irrealism and nihilism, but it seems to me that going too far down this route can lose a lot of the point of the altruistic project. If it’s not mainly about others and their perspectives, why care so much about shaping (some of) their lives and attending to (some of) their concerns?
David Pearce sometimes uses the Holocaust to argue for negative utilitarianism, to say that no amount of good could offset the pain people suffered there. But this view dismisses (or accidentally valorizes) most of the evil of the Holocaust. The death camps centrally were destroying lives and attempting to destroy future generations of peoples, and the people inside them wanted to live free, and being killed sooner was not a close substitute. Killing them (or willfully letting them die when it would be easy to prevent) if they would otherwise escape with a delay would not be helping them for their own sakes, but choosing to be their enemy by only selectively attending to their concerns. And even though some did choose death. Likewise to genocide by sterilization (in my Jewish household growing up the Holocaust was cited as a reason to have children).
Future generations, whether they enthusiastically endorse or oppose their existence, don’t have an immediate voice (or conventional power) here and now their existence isn’t counterfactually robust. But when I’m in a mindset of trying to do impartial good I don’t see the appeal of ignoring those who would desperately, passionately want to exist, and their gratitude in worlds where they do.
I see demandingness and contractarian/game theory/cooperation reasons that bound sacrifice to realize impartial uncompensated help to others, and inevitable moral dilemmas (almost all beings that could exist in a particular location won’t, wild animals are desperately poor and might on average wish they didn’t exist, people have conflicting desires, advanced civilizations I expect will have far more profoundly self-endorsing good lives than unbearably bad lives but on average across the cosmos will have many of the latter by sheer scope). But being an enemy of all the countless beings that would like to exist, or do exist and would like to exist more (or more of something), even if they’re the vast supermajority, seems at odds to me with my idea of impartial benevolence, which I would identify more with trying to be a friend to all, or at least as much as you can given conflicts.
e.g. 2 minds with equally passionate complete enthusiasm (with no contrary psychological processes or internal currencies to provide reference points) respectively for and against their own experience, or gratitude and anger for their birth (past or future). They can respectively consider a world with and without their existences completely unbearable and beyond compensation. But if we’re in the business of helping others for their own sakes rather than ours, I don’t see the case for excluding either one’s concern from our moral circle.
…
But when I’m in a mindset of trying to do impartial good I don’t see the appeal of ignoring those who would desperately, passionately want to exist, and their gratitude in worlds where they do.
I don’t really see the motivation for this perspective. In what sense, or to whom, is a world without the existence of the very happy/fulfilled/whatever person “completely unbearable”? Who is “desperate” to exist? (Concern for reducing the suffering of beings who actually feel desperation is, clearly, consistent with pure NU, but by hypothesis this is set aside.) Obviously not themselves. They wouldn’t exist in that counterfactual.
To me, the clear case for excluding intrinsic concern for those happy moments is:
“Gratitude” just doesn’t seem like compelling evidence in itself that the grateful individual has been made better off. You have to compare to the counterfactual. In daily cases with existing people, gratitude is relevant as far as the grateful person would have otherwise been dissatisfied with their state of deprivation. But that doesn’t apply to people who wouldn’t feel any deprivation in the counterfactual, because they wouldn’t exist.
I take it that the thrust of your argument is, “Ethics should be about applying the same standards we apply across people as we do for intrapersonal prudence.” I agree. And I also find the arguments for empty individualism convincing. Therefore, I don’t see a reason to trust as ~infallible the judgment of a person at time T that the bundle of experiences of happiness and suffering they underwent in times T-n, …, T-1 was overall worth it. They’re making an “interpersonal” value judgment, which, despite being informed by clear memories of the experiences, still isn’t incorrigible. Their positive evaluation of that bundle can be debunked by, say, this insight from my previous bullet point that the happy moments wouldn’t have felt any deprivation had they not existed.
In any case, I find upon reflection that I don’t endorse tradeoffs of contentment for packages of happiness and suffering for myself. I find I’m generally more satisfied with my life when I don’t have the “fear of missing out” that a symmetric axiology often implies. Quoting myself:
Another takeaway is that the fear of missing out seems kind of silly. I don’t know how common this is, but I’ve sometimes felt a weird sense that I have to make the most of some opportunity to have a lot of fun (or something similar), otherwise I’m failing in some way. This is probably largely attributable to the effect of wanting to justify the “price of admission” (I highly recommend the talk in this link) after the fact. No one wants to feel like a sucker who makes bad decisions, so we try to make something we’ve already invested in worth it, or at least feel worth it. But even for opportunities I don’t pay for, monetarily or otherwise, the pressure to squeeze as much happiness from them as possible can be exhausting. When you no longer consider it rational to do so, this pressure lightens up a bit. You don’t have a duty to be really happy. It’s not as if there’s a great video game scoreboard in the sky that punishes you for squandering a sacred gift.
“Gratitude” just doesn’t seem like compelling evidence in itself that the grateful individual has been made better off
What if the individual says that after thinking very deeply about it, they believe their existence genuinely is much better than not having existed? If we’re trying to be altruistic toward their own values, presumably we should also value their existence as better than nothingness (unless we think they’re mistaken)?
One could say that if they don’t currently exist, then their nonexistence isn’t a problem. It’s true that their nonexistence doesn’t cause suffering, but it does make impartial-altruistic total value lower than otherwise if we would consider their existence to be positive.
This is one reason to pay extra attention to cases of near-simultaneous comparisons
In cases of extreme suffering (and maybe also extreme pleasure), it seems to me there’s an empathy gap: when things are going well, you don’t truly understand how bad extreme suffering is, and when you’re in severe pain, you can’t properly care about large volumes of future pleasure. When the suffering is bad enough, it’s as if a different brain takes over that can’t see things from the other perspective, and vice versa for the pleasure-seeking brain. This seems closer to the case of “univocal viewpoints” that you mention.
I can see how for moderate pains and pleasures, a person could experience them in succession and make tradeoffs while still being in roughly the same kind of mental state without too much of an empathy gap. But the fact of those experiences being moderate and exchangeable is the reason I don’t think the suffering in such cases is that morally noteworthy.
we can better trust people’s self-benevolence than their benevolence towards others
Good point. :) OTOH, we might think it’s morally right to have a more cautious approach to imposing suffering on others for the sake of positive goods than we would use for ourselves. In other words, we might favor a moral view that’s different from MacAskill’s proposal to imagine yourself living through every being’s experience in succession.
Strong rejection of interpersonal comparisons is also used to argue that relieving one or more pains can’t compensate for losses to another individual.
Yeah. I support doing interpersonal comparisons, but there’s inherent arbitrariness in how to weigh conflicting preferences across individuals (or sufficiently different mental states of the same individual), and I favor giving more weight to the extreme-suffering preferences.
But if we’re in the business of helping others for their own sakes rather than ours, I don’t see the case for excluding either one’s concern from our moral circle.
That’s fair. :) In my opinion, there’s just an ethical asymmetry between creating a mind that desperately wishes not to exist versus failing to create a mind that desperately would be glad to exist. The first one is horrifying, while the second one is at most mildly unfortunate. I can see how some people would consider this a failure to impartially consider the preferences of others for their own sakes, and if my view makes me less “altruistic” in that sense, then I’m ok with that (as you suspected). My intuition that it’s wrong to allow creating lots of extra torture is stronger than my intuition that I should be an impartial altruist.
If it’s not mainly about others and their perspectives, why care so much about shaping (some of) their lives and attending to (some of) their concerns?
The extreme-suffering concerns are the ones that speak to me most strongly.
seems at odds to me with my idea of impartial benevolence, which I would identify more with trying to be a friend to all
Makes sense. While raw numbers count, it also matters to me what the content of the preference is. If 99% of individuals passionately wanted to create paperclips, while 1% wanted to avoid suffering, I would mostly side with those wanting to avoid suffering, because that just seems more important to me.
“I would be much less sanguine about error theories regarding such utterances if we didn’t also see people in surveys saying they would rather take $1000 than a 15% chance of $1M, or $100 now rather than $140 a year later, i.e. utterances that are clearly mistakes.”
These could be reasonable due to asymmetric information and a potentially adversarial situation, so respondents don’t really trust that the chance of $1M is that high, or that they’ll actually get the $140 a year from now. I would actually expect most people to pick the $100 now over $140 in a year with real money, and I wouldn’t be too surprised if many would pick $1000 over a 15% chance of a million with real money. People are often ambiguity-averse. Of course, they may not really accept the premises of the hypotheticals.
You do see a bunch of crazy financial behavior in the world, but it decreases as people get more experience individually and especially socially (and with better cognitive understanding).
People do engage in rounding to zero in a lot of cases, but with lots of experience will also take on pain and injury with high cumulative or instantaneous probability (e.g. electric shocks to get rewards, labor pains, war, jobs that involve daily frequencies of choking fumes or injury.
Re lexical views that still make probabilistic tradeoffs, I don’t really see the appeal of contorting lexical views that will still be crazy with respect to real world cases so that one can say they assign infinitesimal value to good things in impossible hypotheticals (but effectively 0 in real life). Real world cases like labor pain and risking severe injury doing stuff aren’t about infinitesimal value too small for us to even perceive, but macroscopic value that we are motivated by. Is there a parameterization you would suggest as plausible and addressing that?
Yes, but they might not really be able to entertain the assumptions of the hypotheticals because they’re too abstract and removed from the real world cases they would plausibly face.
with lots of experience will also take on pain and injury with high cumulative or instantaneous probability (e.g. electric shocks to get rewards, labor pains, war, jobs that involve daily frequencies of choking fumes or injury.
(...)
Real world cases like labor pain and risking severe injury doing stuff aren’t about infinitesimal value too small for us to even perceive, but macroscopic value that we are motivated by. Is there a parameterization you would suggest as plausible and addressing that?
Very plausibly none of these possibilities would meet the lexical threshold, except with very very low probability. These people almost never beg to be killed, so the probability of unbearable suffering seems very low for any individual. The lexical threshold could be set based on bearableness or consent or something similar (e.g. Tomasik, Vinding). Coming up with a particular parameterization seems like a bit of work, though, and I’d need more time to think about that, but it’s worth noting that the same practical problem applies to very large aggregates of finite goods/bads, e.g. Heaven or Hell, very long lives, or huge numbers of mind uploads.
There’s also a question of whether a life of unrelenting but less intense suffering can be lexically negative even if no particular experience meets some intensity threshold that would be lexically negative in all lives. Some might think of Omelas this way, and Mogensen’s “The weight of suffering” is inclusive of this view (and also allows experiential lexical thresholds), although I don’t think he discusses any particular parameterization.
Very plausibly none of these possibilities would meet the lexical threshold, except with very very low probability.
I’m confused. :) War has a rather high probability of extreme suffering. Perhaps ~10% of Russian soldiers in Ukraine have been killed as of July 2022. Some fraction of fighters in tanks die by burning to death:
The kinetic energy and friction from modern rounds causes molten metal to splash everywhere in the crew compartment and ignites the air into a fireball. You would die by melting.
You’ll hear of a tank cooking off as it’s ammunition explodes. That doesn’t happen right away. There’s lots to burn inside a tank other that the tank rounds. Often, the tank will burn for quite awhile before the tank rounds explode.
It is sometimes a slow horrific death if one can’t get out in time or a very quick one. We had side arms and all agreed that if our tank was burning and we were caught inside and couldn’t get out. We would use a round on ourselves. That’s how bad it was.
Some workplace accidents also produce extremely painful injuries.
I don’t know what fraction of people in labor wish they were dead, but probably it’s not negligible: “I remember repeatedly saying I wanted to die.”
These people almost never beg to be killed
It may not make sense to beg to be killed, because the doctors wouldn’t grant that wish.
I don’t expect most war deaths to be nearly as painful as burning to death, but I was too quick to dismiss the frequency of very very bad deaths. I had capture and torture in mind as whatever passes the lexical threshold, and so very rare.
Also fair about labor. I don’t think it really gives us an estimate of the frequency of unbearable suffering, although it seems like trauma is common and women aren’t getting as much pain relief as they’d like in the UK.
if we didn’t also see people in surveys saying they would rather take $1000 than a 15% chance of $1M, or $100 now rather than $140 a year later, i.e. utterances that are clearly mistakes.
I agree there are some objectively stupid answers that have been given to surveys, but I’m surprised these were the best examples you could come up with.
Taking $1000 over a 15% chance of $1M can follow from risk aversion which can follow from diminishing marginal utility of money. And let’s face it—money does have diminishing marginal utility.
Wanting $100 now rather than $140 a year later can follow from the time value of money. You could invest the money, either financially or otherwise. Also, even though it’s a hypothetical, people may imagine in the real scenario that they are less likely to get something promised in a year’s time and therefore that they should accept what is really a similar-ish pot of money now.
They’re wildly quantitatively off. Straight 40% returns are way beyond equities, let alone the risk-free rate. And it’s inconsistent with all sorts of normal planning, e.g. it would be against any savings in available investments, much concern for long-term health, building a house, not borrowing everything you could on credit cards, etc.
Similarly the risk aversion for rejecting a 15% of $1M for $1000 would require a bizarre situation (like if you needed just $500 more to avoid short term death), and would prevent dealing with normal uncertainty integral to life, like going on dates with new people, trying to sell products to multiple customers with occasional big hits, etc.
This page says: “The APRs for unsecured credit cards designed for consumers with bad credit are typically in the range of about 25% to 36%.” That’s not too far from 40%. If you have almost no money and would otherwise need such a loan, taking $100 now may be reasonable.
There are claims that “Some 56% of Americans are unable to cover an unexpected $1,000 bill with savings”, which suggests that a lot of people are indeed pretty close to financial emergency, though I don’t know how true that is. Most people don’t have many non-401k investments, and they roughly live paycheck to paycheck.
I also think people aren’t pure money maximizers. They respond differently in different situations based on social norms and how things are perceived. If you get $100 that seems like a random bonus, it’s socially acceptable to just take it now rather than waiting for $140 next year. But it doesn’t look good to take out big credit-card loans that you’ll have trouble repaying. It’s normal to contribute to a retirement account. And so on. People may value being normal and not just how much money they actually have.
That said, most people probably don’t think through these issues at all and do what’s normal on autopilot. So I agree that the most likely explanation is lack of reflectiveness, which was your original point.
Thanks Magnus for your more comprehensive summary of our population ethics study.
You mention this already, but I want to emphasize how much different framings actually matter. This surprised me the most when working on this paper. I’d thus caution anyone against making strong inferences from just one such study.
For example, we conducted the following pilot study (n = 101) where participants were randomly assigned to two different conditions: i) create a new happy person, and ii) create a new unhappy person. See the vignette below:
The response scale ranged from 1 = Extremely bad to 7 = Extremely good.
Creating a happy person was rated as only marginally better than neutral (mean = 4.4), whereas creating an unhappy person was rated as extremely bad (mean = 1.4). So this would lead one to believe that there is strong popular support for the asymmetry. [1]
However, those results were most likely due to the magical machine framing and/or the “push-a-button” framing. Even though these framings clearly “shouldn’t” make such a huge difference.
All in all, we tested many different framings, too many to discuss here. Occasionally, there were significant differences between framings that shouldn’t matter (though we also observed many regularities). For example, we had one pilot with the “multiplier framing”:
Here, the median trade ratio was 8.5 compared to the median trade ratio of 3-4 that we find in our default framing. It’s clear that the multiplier framing shouldn’t make any difference from a philosophical perspective.
So seemingly irrelevant or unimportant changes in framings (unimportant at least from a consequentialist perspective) sometimes could lead to substantial changes in median trade ratios.
However, changes in the intensity of the experienced happiness and suffering—which is arguably the most important aspect of the whole thought experiment—affected the trade ratios considerably less than the above mentioned multiplier framing.
To see this, it’s worth looking closely at the results of study 1b. Participants were first presented with the following scale:
[Editor’s note: From now on, the text is becoming more, um, expressive.]
Note that “worst form of suffering imaginable” is pretty darn bad. Being brutally tortured while kept alive by nano bots is more like −90 on this scale. Likewise, “absolute best form of bliss imaginable” is pretty far out there. Feeling, all your life, like you just created friendly AGI and found your soulmate, while being high on ecstasy would still not be +100.
(Note that we also conducted a pilot study where we used more concrete and explicit descriptions such as “torture”, “falling in love”, “mild headaches”, and “good meal” to describe the feelings of mild or extreme [un]happiness. The results were similar.)
Afterwards, participants were asked:
So how do the MTurkers approach these awe-inspiring intensities?
First, extreme happiness vs. extreme unhappiness. MTurkers think that there need to exist at least 72% people experiencing the absolute best form of bliss imaginable in order to outweigh the suffering of 28% of people experiencing the worst form of suffering imaginable.
Toby Ord and the classical utilitarians rejoice, that’s not bad! That’s like a 3:1 trade ratio, pretty close to a 1:1 trade ratio! “And don’t forget that people’s imagination is likely biased towards negativity for evolutionary reasons!”, Carl Shulman says. “In humans, the pleasure of orgasm may be less than the pain of deadly injury, since death is a much larger loss of reproductive success than a single sex act is a gain.” Everyone nods in agreement with the Shulmaster.
How about extreme happiness vs. mild unhappiness? MTurkers say that there need to exist at least 62% of people experiencing the absolute best form of bliss imaginable in order to outweigh the extremely mild suffering of unhappy people (e.g., people who are stubbing their toes a bit too often for their liking). Brian Tomasik and the suffering-focused crowd rejoice, a 1.5 : 1 trade ratio for practically hedonium to mild suffering?! There is no way the expected value of the future is that good. Reducing s-risks is common sense after all!
How about mild happiness vs. extreme unhappiness? The MTurkers have spoken: A world in which 82% of people experience extremely mild happiness—i.e., eating particularly bland potatoes and listening to muzak without one’s hearing aids on—and 18% of people are brutally tortured while being kept alive by nano bots, is… net positive.
“Wait, that’s a trade ratio of 4.5:1 !” Toby says. “How on Earth is this compatible with a trade ratio of 3:1 for practically hedonium vs. highly optimized suffering, let alone a trade ratio of 1.5:1 for practically hedonium vs. stubbing your toes occasionally!” Carl screams. He looks at Brian but Brian has already fainted.
Toby, Carl and Brian meet the next day, still looking very pale. They shake hands and agree to not do so much descriptive ethics anymore.
Years later, all three still cannot stop wincing with pain when “the Long Reflection” is mentioned.
We also had two conditions about preventing the creation of a happy [unhappy] person. Preventing a happy person from being created (mean = 3.1) was rated as somewhat bad. Preventing an unhappy person (mean = 5.5) from being created was rated as fairly good.
Garbage answers to verbal elicitations on such questions (and real life decisions that require such explicit reasoning without feedback/experience, like retirement savings) are actually quite central to my views. In particular, my reliance on situations where it is easier for individuals to experience things multiple times in easy-to-process fashion and then form a behavioral response. I would be much less sanguine about error theories regarding such utterances if we didn’t also see people in surveys saying they would rather take $1000 than a 15% chance of $1M, or $100 now rather than $140 a year later, i.e. utterances that are clearly mistakes.
Looking at the literature on antiaggregationist views, and the complete conflict of those moral intuitions with personal choices and self-concerned practice (e.g. driving cars or walking outside) is also important to my thinking. No-tradeoffs views are much more appealing outside our own domains of rich experience in talk.
Good points!
It’s not obvious to me that our ethical evaluation should match with the way our brains add up good and bad past experiences at the moment of deciding whether to do more of something. For example, imagine that someone loves to do extreme sports. One day, he has a severe accident and feels so much pain that he, in the moment, wishes he had never done extreme sports or maybe even wishes he had never been born. After a few months in recovery, the severity of those agonizing memories fades, and the temptation to do the sports returns, so he starts doing extreme sports again. At that future point in time, his brain has implicitly made a decision that the enjoyment outweighs the risk of severe suffering. But our ethical evaluation doesn’t have to match how the evolved emotional brain adds things up at that moment in time. We might think that, ethically, the version of the person who was in extreme pain isn’t compensated by other moments of the same person having fun.
Even if we think enjoyment can outweigh severe suffering within a life, many people object to extending such tradeoffs across lives, when one person is severely harmed for the benefit of others. The examples in David’s comment were about interpersonal tradeoffs, rather than intrapersonal ones. It’s true that people impose small risks of extreme suffering on some for the happiness of others all the time, like in the case of driving purely for leisure, but that still begs the question of whether we should do that. Most people in the West also eat chickens, but they shouldn’t. (Cases like driving are also complicated by instrumental considerations, as Magnus would likely point out. Also, not driving for leisure might itself cause some people nontrivial levels of suffering, such as by worsening mental-health problems.)
Hi Brian,
I agree that preferences at different times and different subsystems can conflict. In particular, high discounting of the future can lead to forgoing a ton of positive reward or accepting lots of negative reward in the future in exchange for some short-term change. This is one reason to pay extra attention to cases of near-simultaneous comparisons, or at least to look at different arrangements of temporal ordering. But still the tradeoffs people make for themselves with a lot of experience under good conditions look better than what they tend to impose on others casually. [Also we can better trust people’s self-benevolence than their benevolence towards others, e.g. factory farming as you mention.]
And the brain machinery for processing stimuli into decisions and preferences does seem very relevant to me at least, since that’s a primary source of intuitive assessments of these psychological states as having value, and for comparisons where we can make them. Strong rejection of interpersonal comparisons is also used to argue that relieving one or more pains can’t compensate for losses to another individual.
I agree the hardest cases for making any kind of interpersonal comparison will be for minds with different architectural setups and conflicting univocal viewpoints, e.g. 2 minds with equally passionate complete enthusiasm (with no contrary psychological processes or internal currencies to provide reference points) respectively for and against their own experience, or gratitude and anger for their birth (past or future). They can respectively consider a world with and without their existences completely unbearable and beyond compensation. But if we’re in the business of helping others for their own sakes rather than ours, I don’t see the case for excluding either one’s concern from our moral circle.
Now, one can take take a more nihilistic/personal aesthetics view of morality, and say that one doesn’t personally care about the gratitude of minds happy to exist. I take it this is more your meta-ethical stance around these things? There are good arguments for moral irrealism and nihilism, but it seems to me that going too far down this route can lose a lot of the point of the altruistic project. If it’s not mainly about others and their perspectives, why care so much about shaping (some of) their lives and attending to (some of) their concerns?
David Pearce sometimes uses the Holocaust to argue for negative utilitarianism, to say that no amount of good could offset the pain people suffered there. But this view dismisses (or accidentally valorizes) most of the evil of the Holocaust. The death camps centrally were destroying lives and attempting to destroy future generations of peoples, and the people inside them wanted to live free, and being killed sooner was not a close substitute. Killing them (or willfully letting them die when it would be easy to prevent) if they would otherwise escape with a delay would not be helping them for their own sakes, but choosing to be their enemy by only selectively attending to their concerns. And even though some did choose death. Likewise to genocide by sterilization (in my Jewish household growing up the Holocaust was cited as a reason to have children).
Future generations, whether they enthusiastically endorse or oppose their existence, don’t have an immediate voice (or conventional power) here and now their existence isn’t counterfactually robust. But when I’m in a mindset of trying to do impartial good I don’t see the appeal of ignoring those who would desperately, passionately want to exist, and their gratitude in worlds where they do.
I see demandingness and contractarian/game theory/cooperation reasons that bound sacrifice to realize impartial uncompensated help to others, and inevitable moral dilemmas (almost all beings that could exist in a particular location won’t, wild animals are desperately poor and might on average wish they didn’t exist, people have conflicting desires, advanced civilizations I expect will have far more profoundly self-endorsing good lives than unbearably bad lives but on average across the cosmos will have many of the latter by sheer scope). But being an enemy of all the countless beings that would like to exist, or do exist and would like to exist more (or more of something), even if they’re the vast supermajority, seems at odds to me with my idea of impartial benevolence, which I would identify more with trying to be a friend to all, or at least as much as you can given conflicts.
I don’t really see the motivation for this perspective. In what sense, or to whom, is a world without the existence of the very happy/fulfilled/whatever person “completely unbearable”? Who is “desperate” to exist? (Concern for reducing the suffering of beings who actually feel desperation is, clearly, consistent with pure NU, but by hypothesis this is set aside.) Obviously not themselves. They wouldn’t exist in that counterfactual.
To me, the clear case for excluding intrinsic concern for those happy moments is:
“Gratitude” just doesn’t seem like compelling evidence in itself that the grateful individual has been made better off. You have to compare to the counterfactual. In daily cases with existing people, gratitude is relevant as far as the grateful person would have otherwise been dissatisfied with their state of deprivation. But that doesn’t apply to people who wouldn’t feel any deprivation in the counterfactual, because they wouldn’t exist.
I take it that the thrust of your argument is, “Ethics should be about applying the same standards we apply across people as we do for intrapersonal prudence.” I agree. And I also find the arguments for empty individualism convincing. Therefore, I don’t see a reason to trust as ~infallible the judgment of a person at time T that the bundle of experiences of happiness and suffering they underwent in times T-n, …, T-1 was overall worth it. They’re making an “interpersonal” value judgment, which, despite being informed by clear memories of the experiences, still isn’t incorrigible. Their positive evaluation of that bundle can be debunked by, say, this insight from my previous bullet point that the happy moments wouldn’t have felt any deprivation had they not existed.
In any case, I find upon reflection that I don’t endorse tradeoffs of contentment for packages of happiness and suffering for myself. I find I’m generally more satisfied with my life when I don’t have the “fear of missing out” that a symmetric axiology often implies. Quoting myself:
What if the individual says that after thinking very deeply about it, they believe their existence genuinely is much better than not having existed? If we’re trying to be altruistic toward their own values, presumably we should also value their existence as better than nothingness (unless we think they’re mistaken)?
One could say that if they don’t currently exist, then their nonexistence isn’t a problem. It’s true that their nonexistence doesn’t cause suffering, but it does make impartial-altruistic total value lower than otherwise if we would consider their existence to be positive.
Your reply is an eloquent case for your view. :)
In cases of extreme suffering (and maybe also extreme pleasure), it seems to me there’s an empathy gap: when things are going well, you don’t truly understand how bad extreme suffering is, and when you’re in severe pain, you can’t properly care about large volumes of future pleasure. When the suffering is bad enough, it’s as if a different brain takes over that can’t see things from the other perspective, and vice versa for the pleasure-seeking brain. This seems closer to the case of “univocal viewpoints” that you mention.
I can see how for moderate pains and pleasures, a person could experience them in succession and make tradeoffs while still being in roughly the same kind of mental state without too much of an empathy gap. But the fact of those experiences being moderate and exchangeable is the reason I don’t think the suffering in such cases is that morally noteworthy.
Good point. :) OTOH, we might think it’s morally right to have a more cautious approach to imposing suffering on others for the sake of positive goods than we would use for ourselves. In other words, we might favor a moral view that’s different from MacAskill’s proposal to imagine yourself living through every being’s experience in succession.
Yeah. I support doing interpersonal comparisons, but there’s inherent arbitrariness in how to weigh conflicting preferences across individuals (or sufficiently different mental states of the same individual), and I favor giving more weight to the extreme-suffering preferences.
That’s fair. :) In my opinion, there’s just an ethical asymmetry between creating a mind that desperately wishes not to exist versus failing to create a mind that desperately would be glad to exist. The first one is horrifying, while the second one is at most mildly unfortunate. I can see how some people would consider this a failure to impartially consider the preferences of others for their own sakes, and if my view makes me less “altruistic” in that sense, then I’m ok with that (as you suspected). My intuition that it’s wrong to allow creating lots of extra torture is stronger than my intuition that I should be an impartial altruist.
The extreme-suffering concerns are the ones that speak to me most strongly.
Makes sense. While raw numbers count, it also matters to me what the content of the preference is. If 99% of individuals passionately wanted to create paperclips, while 1% wanted to avoid suffering, I would mostly side with those wanting to avoid suffering, because that just seems more important to me.
“I would be much less sanguine about error theories regarding such utterances if we didn’t also see people in surveys saying they would rather take $1000 than a 15% chance of $1M, or $100 now rather than $140 a year later, i.e. utterances that are clearly mistakes.”
These could be reasonable due to asymmetric information and a potentially adversarial situation, so respondents don’t really trust that the chance of $1M is that high, or that they’ll actually get the $140 a year from now. I would actually expect most people to pick the $100 now over $140 in a year with real money, and I wouldn’t be too surprised if many would pick $1000 over a 15% chance of a million with real money. People are often ambiguity-averse. Of course, they may not really accept the premises of the hypotheticals.
With respect to antiaggregationist views, people could just be ignoring small enough probabilities regardless of the severity of the risk. There are also utility functions where any definite amount of A outweighs any definite amount of B, but probabilistic tradeoffs between them are still possible: https://forum.effectivealtruism.org/posts/GK7Qq4kww5D8ndckR/michaelstjules-s-shortform?commentId=4Bvbtkq83CPWZPNLB
In the surveys they know it’s all hypothetical.
You do see a bunch of crazy financial behavior in the world, but it decreases as people get more experience individually and especially socially (and with better cognitive understanding).
People do engage in rounding to zero in a lot of cases, but with lots of experience will also take on pain and injury with high cumulative or instantaneous probability (e.g. electric shocks to get rewards, labor pains, war, jobs that involve daily frequencies of choking fumes or injury.
Re lexical views that still make probabilistic tradeoffs, I don’t really see the appeal of contorting lexical views that will still be crazy with respect to real world cases so that one can say they assign infinitesimal value to good things in impossible hypotheticals (but effectively 0 in real life). Real world cases like labor pain and risking severe injury doing stuff aren’t about infinitesimal value too small for us to even perceive, but macroscopic value that we are motivated by. Is there a parameterization you would suggest as plausible and addressing that?
Yes, but they might not really be able to entertain the assumptions of the hypotheticals because they’re too abstract and removed from the real world cases they would plausibly face.
Very plausibly none of these possibilities would meet the lexical threshold, except with very very low probability. These people almost never beg to be killed, so the probability of unbearable suffering seems very low for any individual. The lexical threshold could be set based on bearableness or consent or something similar (e.g. Tomasik, Vinding). Coming up with a particular parameterization seems like a bit of work, though, and I’d need more time to think about that, but it’s worth noting that the same practical problem applies to very large aggregates of finite goods/bads, e.g. Heaven or Hell, very long lives, or huge numbers of mind uploads.
There’s also a question of whether a life of unrelenting but less intense suffering can be lexically negative even if no particular experience meets some intensity threshold that would be lexically negative in all lives. Some might think of Omelas this way, and Mogensen’s “The weight of suffering” is inclusive of this view (and also allows experiential lexical thresholds), although I don’t think he discusses any particular parameterization.
I’m confused. :) War has a rather high probability of extreme suffering. Perhaps ~10% of Russian soldiers in Ukraine have been killed as of July 2022. Some fraction of fighters in tanks die by burning to death:
Some workplace accidents also produce extremely painful injuries.
I don’t know what fraction of people in labor wish they were dead, but probably it’s not negligible: “I remember repeatedly saying I wanted to die.”
It may not make sense to beg to be killed, because the doctors wouldn’t grant that wish.
Good points.
I don’t expect most war deaths to be nearly as painful as burning to death, but I was too quick to dismiss the frequency of very very bad deaths. I had capture and torture in mind as whatever passes the lexical threshold, and so very rare.
Also fair about labor. I don’t think it really gives us an estimate of the frequency of unbearable suffering, although it seems like trauma is common and women aren’t getting as much pain relief as they’d like in the UK.
On workplace injuries, in the US in 2020, the highest rate by occupation seems to be around 200 nonfatal injuries and illnesses per 100,000 workers, and 20 deaths per 100,000 workers, but they could be even higher in more specific roles: https://injuryfacts.nsc.org/work/industry-incidence-rates/most-dangerous-industries/
I assume these are estimates of the number of injuries in 2020 only, too, so the lifetime risk is several times higher in such occupations. Maybe the death rate is similar to the rate of unbearable pain, around 1 out of 5,000 per year, which seems non-tiny when added up over a lifetime (around 0.4% over 20 years assuming a geometric distribution https://www.wolframalpha.com/input?i=1-(1-1%2F5000)^20), but also similar in probability to the kinds of risks we do mitigate without eliminating (https://forum.effectivealtruism.org/posts/5y3vzEAXhGskBhtAD/most-small-probabilities-aren-t-pascalian?commentId=jY9o6XviumXfaxNQw).
I agree there are some objectively stupid answers that have been given to surveys, but I’m surprised these were the best examples you could come up with.
Taking $1000 over a 15% chance of $1M can follow from risk aversion which can follow from diminishing marginal utility of money. And let’s face it—money does have diminishing marginal utility.
Wanting $100 now rather than $140 a year later can follow from the time value of money. You could invest the money, either financially or otherwise. Also, even though it’s a hypothetical, people may imagine in the real scenario that they are less likely to get something promised in a year’s time and therefore that they should accept what is really a similar-ish pot of money now.
They’re wildly quantitatively off. Straight 40% returns are way beyond equities, let alone the risk-free rate. And it’s inconsistent with all sorts of normal planning, e.g. it would be against any savings in available investments, much concern for long-term health, building a house, not borrowing everything you could on credit cards, etc.
Similarly the risk aversion for rejecting a 15% of $1M for $1000 would require a bizarre situation (like if you needed just $500 more to avoid short term death), and would prevent dealing with normal uncertainty integral to life, like going on dates with new people, trying to sell products to multiple customers with occasional big hits, etc.
This page says: “The APRs for unsecured credit cards designed for consumers with bad credit are typically in the range of about 25% to 36%.” That’s not too far from 40%. If you have almost no money and would otherwise need such a loan, taking $100 now may be reasonable.
There are claims that “Some 56% of Americans are unable to cover an unexpected $1,000 bill with savings”, which suggests that a lot of people are indeed pretty close to financial emergency, though I don’t know how true that is. Most people don’t have many non-401k investments, and they roughly live paycheck to paycheck.
I also think people aren’t pure money maximizers. They respond differently in different situations based on social norms and how things are perceived. If you get $100 that seems like a random bonus, it’s socially acceptable to just take it now rather than waiting for $140 next year. But it doesn’t look good to take out big credit-card loans that you’ll have trouble repaying. It’s normal to contribute to a retirement account. And so on. People may value being normal and not just how much money they actually have.
That said, most people probably don’t think through these issues at all and do what’s normal on autopilot. So I agree that the most likely explanation is lack of reflectiveness, which was your original point.