Satisfying the Countable Sure-Thing Principle (CSTP, which sounds a lot like the principle of reflection) and updating your credences about outcomes properly as a Bayesian and looking ahead as necessary should save you here. Expected utility maximization with a bounded utility function satisfies the CSTP so it should be safe. See Russell and Isaacs, 2021 for the definition of the CSTP and a theorem, but it should be quick to check that expected utility maximization with a bounded utility function satisfies the CSTP.
You can also preserve any preorder over outcomes from an unbounded real-valued utility function with a bounded utility function (e.g. apply arctan) and avoid these problems. So to me it does seem to be a problem with the attitudes towards risk involved with unbounded utility functions, and it seems appropriate to consider implications for decision theory.
Maybe it is also an epistemic issue, too, though. Like it means having somehow (dynamically?) inconsistent or epistemically irrational joint beliefs.
Are there other violations of the principle of reflection that aren’t avoidable? I’m not familiar with it.
Are there other violations of the principle of reflection that aren’t avoidable? I’m not familiar with it
The case reminded me of one you get without countable additivity. Suppose you have two integers drawn with a fair chancy process that is as likely to result in any integer. What’s the probability the second is greater than the first? 50 50. Now what if you find out the first is 2? Or 2 trillion? Or any finite number? You should then think the second is greater.
Ya, that is similar, but I think the implications are very different.
The uniform measure over the integers can’t be normalized to a probability distribution with total measure 1. So it isn’t a real (or proper) probability distribution. Your options are, assuming you want to address the problem:
It’s not a valid set of credences to hold.
The order on the integers (outcomes) is the problem and we have to give it up (at least for this distribution).
2 gives up a lot more than 1, and there’s no total order we can replace it with that will avoid the problem. Giving up the order also means giving up arithmetical statements about the outcomes of the distribution, because the order is definable from addition or the successor function.
If you give up the total order entirely (not just for the distribution or distributions in general), then you can’t even form the standard set of natural numbers, because the total order is definable from addition or the successor function. So, you’re forced to give up 1 (and the Axiom of Infinity from ZF) along with it, anyway. You also lose lots of proofs in measure theory.
OTOH, the distribution of outcomes in a St Petersburg prospect isn’t improper. The probabilities sum to 1. It’s the combination with your preferences and attitudes to risk that generate the problem. Still, you can respond nearly the same two ways:
It’s not a valid set of credences (over outcomes) to hold.
Your preferences over prospects are the problem and we have to give them up.
However, 2 seems to give up less than 1 here, because:
There’s little independent argument for 1.
You can hold such credences over outcomes without logical contradiction. You can still have non-trivial complete preferences and avoid the problem, e.g. with a bounded utility function.
Your preferences aren’t necessary to make sense of things like the total order on the integers is.
The other unlisted option (here) is that we just accept that infinities are weird and can generate counter-intuitive results and that we shouldn’t take too much from them because it is easier to blame them then all of the other things wrapped up with them. I think the ordering on integers is weird, but it’s not a metaphysical problem. The weird fact is that every integer is unusually small. But that’s just a fact, not a problem to solve.
Infinities generate paradoxes. There are plenty of examples. In decision theory, there is also stuff like Satan’s apple and the expanding sphere of suffering / pleasure. Blaming them all on the weirdness of infinities just seems tidier than coming up with separate ad hoc resolutions.
I think there’s something to this. I argue in Sacrifice or weaken utilitarian principles that it’s better to satisfy the principles you find intuitive more than less (i.e. satisfy weaker versions, which could include the finitary or deterministic case versions, or approximate versions). So, it’s kind of a matter of degree. Still, I think we should have some nuance about infinities rather than treat them all the same and paint their consequences as all easily dismissable. (I gather that this is compatible with your responses so far.)
In general, I take actual infinities (infinities in outcomes or infinitely many decisions or options) as more problematic for basically everyone (although perhaps with additional problems for those with impartial aggregative views) and so their problems easier to dismiss and blame on infinities. Problems from probability distributions with infinitely many outcomes seem to apply much more narrowly and so harder to dismiss or blame on infinities.
(The rest of this comment goes through examples.)
And I don’t think the resolutions are in general ad hoc. Arguments for the Sure-Thing Principle are arguments for bounded utility (well, something more general), and we can characterize the ways that avoid the problem as such (given other EUT axioms, e.g. Russell and Isaacs, 2021). Dutch book arguments for probabilism are arguments that your credences should satisfy certain properties not satisfied by improper distributions. And improper distributions are poorly behaved in other ways that make them implausible for use as credences. For example, how do you define expectations, medians and other quantiles over them — or even the expected value of a nonzero constant functions or two-valued step function over improper distributions — in a way that makes sense? Improper distributions just do very little of what credences are supposed to do.
There are also representation theorems in infinite ethics, specifically giving discounting and limit functions under some conditions in Asheim, 2010 (discussed in West, 2015), and average utilitarianism under others in Pivato (2021, and further discussed in 2022 and 2023).
Satan’s apple would be a problem for basically everyone, and it results from an actual infinity, i.e. infinitely many actual decisions made. (I think how you should handle it in practice is to precommit to taking at most a specific finite number of pieces of the apple, or use a probability distribution, possibly one with infinite expected value but finite with certainty.)
Similarly, when you have infinitely many options to choose from, there may not be any undominated option. As long as you respect statewise dominance and have two outcomes A and B, with one strictly worse than the other, then there’s no undominated option among the set pA + (1-p)B, for all p strictly between 0 and 1 (with p=1/n or 1-1/n for each n). These are cases where the argument for dismissal is strong, because “solving” these problems would mean giving up the most basic requirements of our theories. (And this fits well with scalar utilitarianism.)
My inclination for the expanding sphere of suffering/pleasure is that there are principled solutions:
If you can argue for the separateness of persons, then you should sum over each person’s life before summing across lives. Or, people’s utility values are in fact utility functions, just preferences about how things go, then there may be nothing to aggregate within the person. There’s no temporal aggregation over each person in Harsanyi’s theorem.
If we have to pick an order to sum in or take a value density over, there are more or less natural ones, e.g. using a sequence of nested compact convex sets whose union is the whole space. If we can’t pick one, we can pick multiple or them all, either allowing incompleteness with a multi-utility representation (Shapley and Baucells, 1998, Dubra, Maccheroni, and Ok, 2004, McCarthy et al., 2017, McCarthy et al., 2021), or having normative uncertainty between them.
I think Parfit’s Hitchhiker poses a similar problem for everyone, though.
You’re outside town and hitchhiking with your debit card but no cash, and a driver offers to drive you if you pay him when you get to town. The driver can also tell if you’ll pay (he’s good at reading people), and will refuse to drive if he predicts that you won’t. Assuming you’d rather keep your money than pay conditional on getting into town, it would be irrational to pay then (you wouldn’t be following your own preferences). So, you predict that you won’t pay. And then the driver refuses to drive you, and you lose.
So, the thing to do here is to somehow commit to paying and actually pay, despite it violating your later preference to not pay when you get into town.
And we might respond the same way for A vs B-$100 in the money pump in the post: just commit to sticking with A (at least for high enough value outcomes) and actually do it, even though you know you’ll regret it when you find out the value of A.
So maybe (some) money pump arguments prove too much? Still, it seems better to avoid having these dilemmas when you can, and unbounded utility functions face more of them.
On the other hand, you can change your preferences so that you actually prefer to pay when you get into town. Doing the same for the post’s money pump would mean actually not preferring B-$100 over some finite outcome of A. If you do this in response to all possible money pumps, you’ll end up with a bounded utility function (possibly lexicographic, possibly multi-utility representation). Or, this could be extremely situation-specific preferences. You don’t have to prefer to pay drivers all the time, just in Parfit’s Hitchhiker situations. In general, you can just have preferences specific to every decision situation to avoid money pumps. This violates the Independence of Irrelevant Alternatives, at least in spirit.
Satisfying the Countable Sure-Thing Principle (CSTP, which sounds a lot like the principle of reflection) and updating your credences about outcomes properly as a Bayesian and looking ahead as necessary should save you here. Expected utility maximization with a bounded utility function satisfies the CSTP so it should be safe. See Russell and Isaacs, 2021 for the definition of the CSTP and a theorem, but it should be quick to check that expected utility maximization with a bounded utility function satisfies the CSTP.
You can also preserve any preorder over outcomes from an unbounded real-valued utility function with a bounded utility function (e.g. apply arctan) and avoid these problems. So to me it does seem to be a problem with the attitudes towards risk involved with unbounded utility functions, and it seems appropriate to consider implications for decision theory.
Maybe it is also an epistemic issue, too, though. Like it means having somehow (dynamically?) inconsistent or epistemically irrational joint beliefs.
Are there other violations of the principle of reflection that aren’t avoidable? I’m not familiar with it.
The case reminded me of one you get without countable additivity. Suppose you have two integers drawn with a fair chancy process that is as likely to result in any integer. What’s the probability the second is greater than the first? 50 50. Now what if you find out the first is 2? Or 2 trillion? Or any finite number? You should then think the second is greater.
Ya, that is similar, but I think the implications are very different.
The uniform measure over the integers can’t be normalized to a probability distribution with total measure 1. So it isn’t a real (or proper) probability distribution. Your options are, assuming you want to address the problem:
It’s not a valid set of credences to hold.
The order on the integers (outcomes) is the problem and we have to give it up (at least for this distribution).
2 gives up a lot more than 1, and there’s no total order we can replace it with that will avoid the problem. Giving up the order also means giving up arithmetical statements about the outcomes of the distribution, because the order is definable from addition or the successor function.
If you give up the total order entirely (not just for the distribution or distributions in general), then you can’t even form the standard set of natural numbers, because the total order is definable from addition or the successor function. So, you’re forced to give up 1 (and the Axiom of Infinity from ZF) along with it, anyway. You also lose lots of proofs in measure theory.
OTOH, the distribution of outcomes in a St Petersburg prospect isn’t improper. The probabilities sum to 1. It’s the combination with your preferences and attitudes to risk that generate the problem. Still, you can respond nearly the same two ways:
It’s not a valid set of credences (over outcomes) to hold.
Your preferences over prospects are the problem and we have to give them up.
However, 2 seems to give up less than 1 here, because:
There’s little independent argument for 1.
You can hold such credences over outcomes without logical contradiction. You can still have non-trivial complete preferences and avoid the problem, e.g. with a bounded utility function.
Your preferences aren’t necessary to make sense of things like the total order on the integers is.
The other unlisted option (here) is that we just accept that infinities are weird and can generate counter-intuitive results and that we shouldn’t take too much from them because it is easier to blame them then all of the other things wrapped up with them. I think the ordering on integers is weird, but it’s not a metaphysical problem. The weird fact is that every integer is unusually small. But that’s just a fact, not a problem to solve.
Infinities generate paradoxes. There are plenty of examples. In decision theory, there is also stuff like Satan’s apple and the expanding sphere of suffering / pleasure. Blaming them all on the weirdness of infinities just seems tidier than coming up with separate ad hoc resolutions.
I think there’s something to this. I argue in Sacrifice or weaken utilitarian principles that it’s better to satisfy the principles you find intuitive more than less (i.e. satisfy weaker versions, which could include the finitary or deterministic case versions, or approximate versions). So, it’s kind of a matter of degree. Still, I think we should have some nuance about infinities rather than treat them all the same and paint their consequences as all easily dismissable. (I gather that this is compatible with your responses so far.)
In general, I take actual infinities (infinities in outcomes or infinitely many decisions or options) as more problematic for basically everyone (although perhaps with additional problems for those with impartial
aggregativeviews) and so their problems easier to dismiss and blame on infinities. Problems from probability distributions with infinitely many outcomes seem to apply much more narrowly and so harder to dismiss or blame on infinities.(The rest of this comment goes through examples.)
And I don’t think the resolutions are in general ad hoc. Arguments for the Sure-Thing Principle are arguments for bounded utility (well, something more general), and we can characterize the ways that avoid the problem as such (given other EUT axioms, e.g. Russell and Isaacs, 2021). Dutch book arguments for probabilism are arguments that your credences should satisfy certain properties not satisfied by improper distributions. And improper distributions are poorly behaved in other ways that make them implausible for use as credences. For example, how do you define expectations, medians and other quantiles over them — or even the expected value of a nonzero constant functions or two-valued step function over improper distributions — in a way that makes sense? Improper distributions just do very little of what credences are supposed to do.
There are also representation theorems in infinite ethics, specifically giving discounting and limit functions under some conditions in Asheim, 2010 (discussed in West, 2015), and average utilitarianism under others in Pivato (2021, and further discussed in 2022 and 2023).
Satan’s apple would be a problem for basically everyone, and it results from an actual infinity, i.e. infinitely many actual decisions made. (I think how you should handle it in practice is to precommit to taking at most a specific finite number of pieces of the apple, or use a probability distribution, possibly one with infinite expected value but finite with certainty.)
Similarly, when you have infinitely many options to choose from, there may not be any undominated option. As long as you respect statewise dominance and have two outcomes A and B, with one strictly worse than the other, then there’s no undominated option among the set pA + (1-p)B, for all p strictly between 0 and 1 (with p=1/n or 1-1/n for each n). These are cases where the argument for dismissal is strong, because “solving” these problems would mean giving up the most basic requirements of our theories. (And this fits well with scalar utilitarianism.)
My inclination for the expanding sphere of suffering/pleasure is that there are principled solutions:
If you can argue for the separateness of persons, then you should sum over each person’s life before summing across lives. Or, people’s utility values are in fact utility functions, just preferences about how things go, then there may be nothing to aggregate within the person. There’s no temporal aggregation over each person in Harsanyi’s theorem.
If we have to pick an order to sum in or take a value density over, there are more or less natural ones, e.g. using a sequence of nested compact convex sets whose union is the whole space. If we can’t pick one, we can pick multiple or them all, either allowing incompleteness with a multi-utility representation (Shapley and Baucells, 1998, Dubra, Maccheroni, and Ok, 2004, McCarthy et al., 2017, McCarthy et al., 2021), or having normative uncertainty between them.
I think Parfit’s Hitchhiker poses a similar problem for everyone, though.
You’re outside town and hitchhiking with your debit card but no cash, and a driver offers to drive you if you pay him when you get to town. The driver can also tell if you’ll pay (he’s good at reading people), and will refuse to drive if he predicts that you won’t. Assuming you’d rather keep your money than pay conditional on getting into town, it would be irrational to pay then (you wouldn’t be following your own preferences). So, you predict that you won’t pay. And then the driver refuses to drive you, and you lose.
So, the thing to do here is to somehow commit to paying and actually pay, despite it violating your later preference to not pay when you get into town.
And we might respond the same way for A vs B-$100 in the money pump in the post: just commit to sticking with A (at least for high enough value outcomes) and actually do it, even though you know you’ll regret it when you find out the value of A.
So maybe (some) money pump arguments prove too much? Still, it seems better to avoid having these dilemmas when you can, and unbounded utility functions face more of them.
On the other hand, you can change your preferences so that you actually prefer to pay when you get into town. Doing the same for the post’s money pump would mean actually not preferring B-$100 over some finite outcome of A. If you do this in response to all possible money pumps, you’ll end up with a bounded utility function (possibly lexicographic, possibly multi-utility representation). Or, this could be extremely situation-specific preferences. You don’t have to prefer to pay drivers all the time, just in Parfit’s Hitchhiker situations. In general, you can just have preferences specific to every decision situation to avoid money pumps. This violates the Independence of Irrelevant Alternatives, at least in spirit.
See also https://www.lesswrong.com/tag/parfits-hitchhiker