Isaacs, Beckstead & Thomas, and Wilkinson point out how weird it would be to adopt a complete and consistent decision theory that wasn’t fanatical. It would involve making arbitrary distinctions between minute differences of the probability of different wagers or evaluating packages of wagers differently then one evaluates the sum of the wagers individually. Offered enough wagers, non-fanatics must make some distinctions that they will be very hard-pressed to justify.
Idk, bounded utility functions seem pretty justifiable to me.* Just slap diminishing returns on everything. Yes, more happy lives are good, but if you already have a googleplex of them, it’s not so morally important to make more. Etc. As for infinities, well, I think we need a measure over infinities anyway, so let’s say that our utility function is bounded by 1 and −1, with 1 being the case where literally everything that happens across infinite spacetime is as good as possible—the best possible world—and −1 is the opposite, and in between we have various cases in which good things happen with some measure and bad things happen with some measure.
*I totally feel the awkwardness/counterintuitiveness in certain cases, as the papers you link point out. E.g. when it’s about suffering. But it feels much less bad than the problems with unbounded utility functions. As you say, it seems like people with unbounded utility functions should be fanatical (or paralyzed, I’d add) and fanatics… well, no one I know is willing to bite the bullet and actually start doing absurdist research in earnest. People might claim, therefore, to have unbounded utility functions, but I doubt their claims.
I’m curious how you think about bounded utility function. Its not something I’ve thought about much. The following sort of case seems problematic.
Walking home one night from a lecture on astrophysics where you learned about the latest research establishing the massive size of the universe, you come across a child drowning in a pond. The kid is kicking and screaming trying to stay above the water. You can see the terror in his eyes and you know that it’s going to get painful when the water starts filling his lungs. You see is mother, off in the distance, screaming and running. Something just tells you she’ll never get over this. It will wreck her marriage and her career. There’s a life preserver in easy reach. You could save the child without much fuss. But you recall your lecture the oodles and oodles of people living on other planets and figure that we must be very near the bound of total value for the universe, so the kid’s death can’t be of more than the remotest significance. And there’s a real small chance that solipsism is true, in which case your whims matter much more (we’re not near the bounds) and satisfying them will make a much bigger difference to total value. The altruistic thing to do is to not make the effort, which could be mildly unpleasant, even though it very likely means the kid will die an agonizing death and his mother will mourn for decades.
That seems really wrong. Much more so than thinking that fanaticism is unreasonable.
Ooof, yeah, I hadn’t thought about the solipsism possibility before. If the math checks out then I’ll keep my bounded utility function but also maybe add in some nonconsequentialist-ish stuff to cover this case and cases like it. (or, you can think of it as just specifying that the utility function should assign significant negative utility to you doing unvirtuous acts like this.)
That said, I’m skeptical that the math works out for this example. Just because the universe is very big doesn’t mean we are very near the bound. We’d only be very near the bound if the universe was both very big and very perfect, i.e. suffering, injustice, etc. all practically nonexistent as a fraction of things happening.
So we are probably nowhere near either end of the bound, and the question is how much difference saving one child makes in a very big universe.
For reasons related to noncausal decision theory, the answer is “a small but non-negligible fraction of all the things that happen in this universe depend on what you do in this case. If you save the child, people similar to you in similar situations all across the multiverse will choose to save similar children (or alien children, or whatever).”
The question is whether that small but non-negligible positive impact is outweighed by the maybe-solipsism-is-true-and-me-enjoying-this-ice-cream-is-thus-somewhat-important possibility.
Intuitively it feels like the answer is “hell no” but it would be good to see a full accounting. I agree that if the full accounting says the answer is “yes” then that’s a reductio.
Note that the best possible solipsistic world is still vastly worse than the best possible big world.
(Oops, didn’t realize you were the same person that talked to me about the sequence, shoulda put two and two together, sorry!)
Your utility function can instead be bounded wrt the difference you make relative to some fixed default distribution of outcomes (“doing nothing”, or “business as usual”) or in each pairwise comparison (although I’m not sure this will be well-behaved). For example, take all the differences in welfare between the two random variable outcomes corresponding to two options, apply some bounded function of all of these differences, and finally take the expected value.
Your utility function can instead be bounded wrt the difference you make relative to some fixed default distribution of outcomes (“doing nothing”, or “business as usual”) or in each pairwise comparison (although I’m not sure this will be well-behaved). For example, take all the differences in welfare between the two random variable outcomes corresponding to two options, apply some bounded function of all of these differences, and finally take the expected value.
Consider the following amended thought experiment: (changes in bold)
Walking home one night from a lecture on astrophysics where you learned about the latest research establishing the massive size of the universe, you come across a child drowning in a pond. The kid is kicking and screaming trying to stay above the water. You can see the terror in his eyes and you know that it’s going to get painful when the water starts filling his lungs. You see is mother, off in the distance, screaming and running. Something just tells you she’ll never get over this. It will wreck her marriage and her career. There’s two buttons near you. Pressing either will trigger an event that adds 101000 really good lives to the universe. (The buttons will create the exact same lives and only function once.) The second also causes a life preserver to be tossed to the child. The second button is slightly further from you, and you’d have to strain to reach it. And there’s a real small chance that solipsism is true, in which case your whims matter much more (we’re not near the bounds) and satisfying them will make a much bigger difference to total value. The altruistic thing to do is to not make the additional effort to react the further button, which could be mildly unpleasant, even though it very likely means the kid will die an agonizing death and his mother will mourn for decades.
Good example! At least this isn’t solipsistic egoism, but I agree the results seem too egoistic.
What you could do is rearrange the two probability distributions of aggregate welfares statewise in non-decreasing order (or in a way that minimizes some distance between the two distributions), take the difference between the two resulting random variables, apply a bounded monotonically increasing function to the difference, and then take the expected value.
Unfortunately, I suspect this pairwise comparison approach won’t even be transitive.
Given an intransitive relation over options (distributions over outcomes), you can use voting methods like beatpath to define a similar transitive relation or choose among options even when there’s intransitivity in a choice set. Using beatpath on the specific actual option sets you face in particular will mean violating the independence of irrelevant alternatives, which I’m pretty okay with giving up, personally.
You could apply beatpath to the set of all conceivable options, even those not actually available to you in a given choice situation, but I imagine you’ll get too much indifference or incomparability.
Re. non-consequentialist stuff, I notice that I expect societies to go better if people have some degree of extra duty towards (or caring towards) those closer to them. That could be enough here?
(i.e. Boundedly rational agents shouldn’t try to directly approximate their best guess about the global utility function.)
Just because the universe is very big doesn’t mean we are very near the bound. We’d only be very near the bound if the universe was both very big and very perfect, i.e. suffering, injustice, etc. all practically nonexistent as a fraction of things happening.
My thought was that you’d need a large universe consisting of people like us to be very near the bound, otherwise you couldn’t use boundedness to get out of assigning a high expected value to the example projects I proposed. There might be ways of finessing the dimensions of boundedness to avoid this sort of concern, but I’m skeptical (though I haven’t thought about it much).
I also find it methodologically dubious to adjust your value function to fit what actions you think you should do. It feels to me like your value function should be your value function, and you should adjust your decision rules if they produce a bad verdict. If your value function is bounded, so be it. But don’t cut it off to make expected value maximization more palatable.
If the math checks out then I’ll keep my bounded utility function but also maybe add in some nonconsequentialist-ish stuff to cover this case and cases like it.
I can see why you might do this, but it feels strange to me. The reason to save the child isn’t because its a good thing for the child not to drown, but because there’s some rule that you’re supposed to follow that tells you to save the kid? Do these rules happen to require you to act in ways that basically align with what a total utilitarian would do, or do they have the sort of oddities that afflict deontological views (e.g. don’t lie to the murderer at the door)?
My own thoughts on this subject.
Also relevant: Impossibility results for unbounded utility functions.
You say:
Idk, bounded utility functions seem pretty justifiable to me.* Just slap diminishing returns on everything. Yes, more happy lives are good, but if you already have a googleplex of them, it’s not so morally important to make more. Etc. As for infinities, well, I think we need a measure over infinities anyway, so let’s say that our utility function is bounded by 1 and −1, with 1 being the case where literally everything that happens across infinite spacetime is as good as possible—the best possible world—and −1 is the opposite, and in between we have various cases in which good things happen with some measure and bad things happen with some measure.
*I totally feel the awkwardness/counterintuitiveness in certain cases, as the papers you link point out. E.g. when it’s about suffering. But it feels much less bad than the problems with unbounded utility functions. As you say, it seems like people with unbounded utility functions should be fanatical (or paralyzed, I’d add) and fanatics… well, no one I know is willing to bite the bullet and actually start doing absurdist research in earnest. People might claim, therefore, to have unbounded utility functions, but I doubt their claims.
Big fan of your sequence!
I’m curious how you think about bounded utility function. Its not something I’ve thought about much. The following sort of case seems problematic.
That seems really wrong. Much more so than thinking that fanaticism is unreasonable.
Ooof, yeah, I hadn’t thought about the solipsism possibility before. If the math checks out then I’ll keep my bounded utility function but also maybe add in some nonconsequentialist-ish stuff to cover this case and cases like it. (or, you can think of it as just specifying that the utility function should assign significant negative utility to you doing unvirtuous acts like this.)
That said, I’m skeptical that the math works out for this example. Just because the universe is very big doesn’t mean we are very near the bound. We’d only be very near the bound if the universe was both very big and very perfect, i.e. suffering, injustice, etc. all practically nonexistent as a fraction of things happening.
So we are probably nowhere near either end of the bound, and the question is how much difference saving one child makes in a very big universe.
For reasons related to noncausal decision theory, the answer is “a small but non-negligible fraction of all the things that happen in this universe depend on what you do in this case. If you save the child, people similar to you in similar situations all across the multiverse will choose to save similar children (or alien children, or whatever).”
The question is whether that small but non-negligible positive impact is outweighed by the maybe-solipsism-is-true-and-me-enjoying-this-ice-cream-is-thus-somewhat-important possibility.
Intuitively it feels like the answer is “hell no” but it would be good to see a full accounting. I agree that if the full accounting says the answer is “yes” then that’s a reductio.
Note that the best possible solipsistic world is still vastly worse than the best possible big world.
(Oops, didn’t realize you were the same person that talked to me about the sequence, shoulda put two and two together, sorry!)
There’s a paper by Tarsney on solipsistic swamping for some specific social welfare functions, like average utilitarianism, just considering moral patients on our Earth so far: https://www.tandfonline.com/doi/full/10.1080/00048402.2021.1962375
Your utility function can instead be bounded wrt the difference you make relative to some fixed default distribution of outcomes (“doing nothing”, or “business as usual”) or in each pairwise comparison (although I’m not sure this will be well-behaved). For example, take all the differences in welfare between the two random variable outcomes corresponding to two options, apply some bounded function of all of these differences, and finally take the expected value.
Consider the following amended thought experiment: (changes in bold)
Good example! At least this isn’t solipsistic egoism, but I agree the results seem too egoistic.
What you could do is rearrange the two probability distributions of aggregate welfares statewise in non-decreasing order (or in a way that minimizes some distance between the two distributions), take the difference between the two resulting random variables, apply a bounded monotonically increasing function to the difference, and then take the expected value.
Unfortunately, I suspect this pairwise comparison approach won’t even be transitive.
Given an intransitive relation over options (distributions over outcomes), you can use voting methods like beatpath to define a similar transitive relation or choose among options even when there’s intransitivity in a choice set. Using beatpath on the specific actual option sets you face in particular will mean violating the independence of irrelevant alternatives, which I’m pretty okay with giving up, personally.
This is done in this paper:
https://globalprioritiesinstitute.org/teruji-thomas-the-asymmetry-uncertainty-and-the-long-term/
You could apply beatpath to the set of all conceivable options, even those not actually available to you in a given choice situation, but I imagine you’ll get too much indifference or incomparability.
Re. non-consequentialist stuff, I notice that I expect societies to go better if people have some degree of extra duty towards (or caring towards) those closer to them. That could be enough here?
(i.e. Boundedly rational agents shouldn’t try to directly approximate their best guess about the global utility function.)
My thought was that you’d need a large universe consisting of people like us to be very near the bound, otherwise you couldn’t use boundedness to get out of assigning a high expected value to the example projects I proposed. There might be ways of finessing the dimensions of boundedness to avoid this sort of concern, but I’m skeptical (though I haven’t thought about it much).
I also find it methodologically dubious to adjust your value function to fit what actions you think you should do. It feels to me like your value function should be your value function, and you should adjust your decision rules if they produce a bad verdict. If your value function is bounded, so be it. But don’t cut it off to make expected value maximization more palatable.
I can see why you might do this, but it feels strange to me. The reason to save the child isn’t because its a good thing for the child not to drown, but because there’s some rule that you’re supposed to follow that tells you to save the kid? Do these rules happen to require you to act in ways that basically align with what a total utilitarian would do, or do they have the sort of oddities that afflict deontological views (e.g. don’t lie to the murderer at the door)?