Making decisions when both morally and empirically uncertain
Cross-posted to LessWrong. For an epistemic status statement and an outline of the purpose of the series of posts this is part of, please see the top of my prior post. There are also some explanations and caveats in that post which I wonât repeatâor will repeat only brieflyâin this post.
Purpose of this post
In my prior post, I wrote:
We are often forced to make decisions under conditions of uncertainty. This uncertainty can be empirical (e.g., what is the likelihood that nuclear war would cause human extinction?) or moral (e.g., does the wellbeing of future generations matter morally?). The issue of making decisions under empirical uncertainty has been well-studied, and expected utility theory has emerged as the typical account of how a rational agent should proceed in these situations. The issue of making decisions under moral uncertainty appears to have received less attention (though see this list of relevant papers), despite also being of clear importance.
I then went on to describe three prominent approaches for dealing with moral uncertainty (based on Will MacAskillâs 2014 thesis):
Maximising Expected Choice-worthiness (MEC), if all theories under consideration by the decision-maker are cardinal and intertheoretically comparable.[1]
Variance Voting (VV), a form of what Iâll call âNormalised MECâ, if all theories under consideration are cardinal but not intertheoretically comparable.[2]
The Borda Rule (BR), if all theories under consideration are ordinal.
But I was surprised to discover that I couldnât find any very explicit write-up of how to handle moral and empirical uncertainty at the same time. I assume this is because most people writing on relevant topics consider the approach I will propose in this post to be quite obvious (at least when using MEC with cardinal, intertheoretically comparable, consequentialist theories). Indeed, many existing models from EAs/ârationalists (and likely from other communities) already effectively use something very much like the first approach I discuss here (âMEC-Eâ; explained below), just without explicitly noting that this is an integration of approaches for dealing with moral and empirical uncertainty.[3]
But it still seemed worth explicitly spelling out the approach I propose, which is, in a nutshell, using exactly the regular approaches to moral uncertainty mentioned above, but on outcomes rather than on actions, and combining that with consideration of the likelihood of each action leading to each outcome. My aim for this post is both to make this approach âobviousâ to a broader set of people and to explore how it can work with non-comparable, ordinal, and/âor non-consequentialist theories (which may be less obvious).
(Additionally, as a side-benefit, readers who are wondering what on earth all this âmodellingâ business some EAs love talking about is, or who are only somewhat familiar with modelling, may find this post to provide useful examples and explanations.)
Iâd be interested in any comments or feedback you might have on anything I discuss here!
MEC under empirical uncertainty
To briefly review regular MEC: MacAskill argues that, when all moral theories under consideration are cardinal and intertheoretically comparable, a decision-maker should choose the âoptionâ that has the highest expected choice-worthiness. Expected choice-worthiness is given by the following formula:
In this formula, C(Ti) represents the decision-makerâs credence (belief) in Ti (some particular moral theory), while CWi(A) represents the âchoice-worthinessâ (CW) of A (an âoptionâ or action that the decision-maker can choose) according to Ti. In my prior post, I illustrated how this works with this example:
Suppose Devon assigns a 25% probability to T1, a version of hedonistic utilitarianism in which human âhedonsâ (a hypothetical unit of pleasure) are worth 10 times more than fish hedons. He also assigns a 75% probability to T2, a different version of hedonistic utilitarianism, which values human hedons just as much as T1 does, but doesnât value fish hedons at all (i.e., it sees fish experiences as having no moral significance). Suppose also that Devon is choosing whether to buy a fish curry or a tofu curry, and that heâd enjoy the fish curry about twice as much. (Finally, letâs go out on a limb and assume Devonâs humanity.)
According to T1, the choice-worthiness (roughly speaking, the rightness or wrongness of an action) of buying the fish curry is â90 (because itâs assumed to cause 1,000 negative fish hedons, valued as â100, but also 10 human hedons due to Devonâs enjoyment).[5] In contrast, according to T2, the choice-worthiness of buying the fish curry is 10 (because this theory values Devonâs joy as much as T1 does, but doesnât care about the fishâs experiences). Meanwhile, the choice-worthiness of the tofu curry is 5 according to both theories (because it causes no harm to fish, and Devon would enjoy it half as much as heâd enjoy the fish curry).
[...] Using MEC in this situation, the expected choice-worthiness of buying the fish curry is 0.25 * â90 + 0.75 * 10 = â15, and the expected choice-worthiness of buying the tofu curry is 0.25 * 5 + 0.75 * 5 = 5. Thus, Devon should buy the tofu curry.
But can Devon really be sure that buying the fish curry will lead to that much fish suffering? What if this demand signal doesnât lead to increased fish farming/âcapture? What if the additional fish farming/âcapture is more humane than expected? What if fish canât suffer because they arenât actually conscious (empirically, rather than as a result of what sorts of consciousness our moral theory considers relevant)? We could likewise question Devonâs apparent certainty that buying the tofu curry definitely wonât have any unintended consequences for fish suffering, and his apparent certainty regarding precisely how much heâd enjoy each meal.
These are all empirical rather than moral questions, but they still seem very important for Devonâs ultimate decision. This is because T1 and T2 donât âintrinsically careâ about whether someone buys fish curry or buys tofu curry; these theories assign no terminal value to which curry is bought. Instead, these theories âcareâ about some of the outcomes which those actions may or may not cause.[4]
More generally, I expect that, in all realistic decision situations, weâll have both moral and empirical uncertainty, and that itâll often be important to explicitly consider both types of uncertainties. For example, GiveWellâs models consider both how likely insecticide-treated bednets are to save the life of a child, and how that outcome would compare to doubling the income of someone in extreme poverty. However, typical discussions of MEC seem to assume that we already know for sure what the outcomes of our actions will be, just as typical discussions of expected value reasoning seem to assume that we already know for sure how valuable a given outcome is.
Luckily, it seems to me that MEC and traditional (empirical) expected value reasoning can be very easily and neatly integrated in a way that resolves those issues. (This is perhaps partly due to that fact that, if I understand MacAskillâs thesis correctly, MEC was very consciously developed by analogy to expected value reasoning.) Here is my formula for this integration, which Iâll call Maximising Expected Choice-worthiness, accounting for Empirical uncertainty (MEC-E), and which Iâll explain and provide an example for below:
Here, all symbols mean the same things they did in the earlier formula from MacAskillâs thesis, with two exceptions:
Iâve added Oj, to refer to each âoutcomeâ: each consequence that an action may lead to, which at least one moral theory under consideration intrinsically values/âdisvalues. (E.g., a fish suffering; a person being made happy; rights being violated.)
Related to that, Iâd like to be more explicit that A refers only to the âactionsâ that the decision-maker can directly choose (e.g., purchasing a fish meal, imprisoning someone), rather than the outcomes of those actions.[5]
(I also re-ordered the choice-worthiness term and the credence term, which makes no actual difference to any results, and was just because I think this ordering is slightly more intuitive.)
Stated verbally (and slightly imprecisely[6]), MEC-E claims that:
One should choose the action which maximises expected choice-worthiness, accounting for empirical uncertainty. To calculate the expected choice-worthiness of each action, you first, for each potential outcome of the action and each moral theory under consideration, find the product of 1) the probability of that outcome given that that action is taken, 2) the choice-worthiness of that outcome according to that theory, and 3) the credence given to that theory. Second, for each action, you sum together all of those products.
To illustrate, I have modelled in Guesstimate an extension of the example of Devon deciding what meal to buy to also incorporate empirical uncertainty.[7] In the text here, I will only state the information that was not in the earlier version of the example, and the resulting calculations, rather than walking through all the details.
Suppose Devon believes thereâs an 80% chance that buying a fish curry will lead to âfish being harmedâ (modelled as 1000 negative fish hedons, with a choice-worthiness of â100 according to T1 and 0 according to T2), and a 10% chance that buying a tofu curry will lead to that same outcome. He also believes thereâs a 95% chance that buying a fish curry will lead to âDevon enjoying a meal a lotâ (modelled as 10 human hedons), and a 50% chance that buying a tofu curry will lead to that.
The expected choice-worthiness of buying a fish curry would therefore be:
(0.8 * â100 * 0.25) + (0.8 * 0 * 0.75) + (0.95 * 10 * 0.25) + (0.95 * 10 * 0.75) = â10.5
Meanwhile, the expected choice-worthiness of buying a tofu curry would be:
(0.1 * â100 * 0.25) + (0.1 * 0 * 0.75) + (0.5 * 10 * 0.25) + (0.5 * 10 * 0.75) = 2.5
As before, the tofu curry appears the better choice, despite seeming somewhat worse according to the theory (T2) assigned higher credence, because the other theory (T1) sees the tofu curry as much better.
In the final section of this post, I discuss potential extensions of these approaches, such as how it can handle probability distributions (rather than point estimates) and non-consequentialist theories.
The last thing Iâll note about MEC-E in this section is that MEC-E can be used as a heuristic, without involving actual numbers, in exactly the same way MEC or traditional expected value reasoning can. For example, without knowing or estimating any actual numbers, Devon might reason that, compared to buying the tofu curry, buying the fish curry is âmuchâ more likely to lead to fish suffering and only âsomewhatâ more likely to lead to him enjoying his meal a lot. He may further reason that, in the âunlikely but plausibleâ event that fish experiences do matter, the badness of a large amount of fish suffering is âmuchâ greater than the goodness of him enjoying a meal. He may thus ultimately decide to purchase the tofu curry.
(Indeed, my impression is that many effective altruists have arrived at vegetarianism/âveganism through reasoning very much like that, without any actual numbers being required.)
Normalised MEC under empirical uncertainty
(From here onwards, Iâve had to go a bit further beyond whatâs clearly implied by existing academic work, so the odds Iâll make some mistakes go up a bit. Please let me know if you spot any errors.)
To briefly review regular Normalised MEC: Sometimes, despite being cardinal, the moral theories we have credence in are not intertheoretically comparable (basically meaning that thereâs no consistent, non-arbitrary âexchange rateâ between the theoriesâ âunits of choice-worthinessâ). MacAskill argues that, in such situations, one must first ânormaliseâ the theories in some way (i.e., â[adjust] values measured on different scales to a notionally common scaleâ), and then apply MEC to the new, normalised choice-worthiness scores. He recommends Variance Voting, in which the normalisation is by variance (rather than, e.g., by range), meaning that we:
â[treat] the average of the squared differences in choice-worthiness from the mean choice-worthiness as the same across all theories. Intuitively, the variance is a measure of how spread out choice-worthiness is over different options; normalising at variance is the same as normalising at the difference between the mean choice-worthiness and one standard deviation from the mean choice-worthiness.â
(I provide a worked example here, based on an extension of the scenario with Devon deciding what meal to buy, but itâs possible Iâve made mistakes.)
My proposal for Normalised MEC, accounting for Empirical Uncertainty (Normalised MEC-E) is just to combine the ideas of non-empirical Normalised MEC and non-normalised MEC-E in a fairly intuitive way. The steps involved (which may be worth reading alongside this worked example and/âor the earlier explanations of Normalised MEC and MEC-E) are as follows:
-
Work out expected choice-worthiness just as with regular MEC, except that here one is working out the expected choice-worthiness of outcomes, not actions. I.e., for each outcome, multiply that outcomeâs choice-worthiness according to each theory by your credence in that theory, and then add up the resulting products.
You could also think of this as using the MEC-E formula, except with âProbability of outcome given actionâ removed for now.
-
Normalise these expected choice-worthiness scores by variance, just as MacAskill advises in the quote above.
-
Find the âexpected valueâ of each action in the traditional way, with these normalised expected choice-worthiness scores serving as the âvalueâ for each potential outcome. I.e., for each action, multiply the probability it leads to each outcome by the normalised expected choice-worthiness of that outcome (from step 2), and then add up the resulting products.
You could think of this as bringing âProbability of outcome given actionâ back into the MEC-E formula.
-
Choose the action with the maximum score from step 3 (which we could call normalised expected choice-worthiness, accounting for empirical uncertainty, or expected value, accounting for normalised moral uncertainty).[8]
BR under empirical uncertainty
The final approach MacAskill recommends in his thesis is the Borda Rule (BR; also known as Borda counting). This is used when the moral theories we have credence in are merely ordinal (i.e., they donât say âhow muchâ more choice-worthy one option is compared to another). In my prior post, I provided the following quote of MacAskillâs formal explanation of BR (here with âoptionsâ replaced by âactionsâ):
âAn [action] Aâs Borda Score, for any theory Ti, is equal to the number of [actions] within the [action]-set that are less choice-worthy than A according to theory Tiâs choice-worthiness function, minus the number of [actions] within the [action]-set that are more choice-worthy than A according to Tiâs choice-worthiness function.
An [action] Aâs Credence-Weighted Borda Score is the sum, for all theories Ti, of the Borda Score of A according to theory Ti multiplied by the credence that the decision-maker has in theory Ti.
[The Borda Rule states that an action] A is more appropriate than an [action] B iff [if and only if] A has a higher Credence-Weighted Borda Score than B; A is equally as appropriate as B iff A and B have an equal Credence-Weighted Borda Score.â
To apply BR when one is also empirically uncertain, I propose just explicitly considering/âmodelling oneâs empirical uncertainties, and then figuring out each actionâs Borda Score with those empirical uncertainties in mind. (That is, we donât change the method at all on a mathematical level; we just make sure each moral theoryâs preference rankings over actionsâwhich is used as input into the Borda Ruleâtakes into account our empirical uncertainty about what outcomes each action may lead to.)
Iâll illustrate how this works with reference to the same example from MacAskillâs thesis that I quoted in my prior post, but now with slight modifications (shown in bold).
âJulia is a judge who is about to pass a verdict on whether Smith is guilty for murder. She is very confident that Smith is innocent. There is a crowd outside, who are desperate to see Smith convicted. Julia has three options:
[G]: Pass a verdict of âguiltyâ.
[R]: Call for a retrial.
[I]: Pass a verdict of âinnocentâ.
She thinks thereâs a 0% chance of M if she passes a verdict of guilty, a 30% chance if she calls for a retrial (there may mayhem due to the lack of a guilty verdict, or later due to a later innocent verdict), and a 70% chance if she passes a verdict of innocent.
Thereâs obviously a 100% chance of C if she passes a verdict of guilty and a 0% chance if she passes a verdict of innocent. She thinks thereâs also a 20% chance of C happening later if she calls for a retrial.
Julia believes the crowd is very likely (~90% chance) to riot if Smith is found innocent, causing mayhem on the streets and the deaths of several people. If she calls for a retrial, she believes itâs almost certain (~95% chance) that he will be found innocent at a later date, and that it is much less likely (only ~30% chance) that the crowd will riot at that later date if he is found innocent then. If she declares Smith guilty, the crowd will certainly (~100%) be appeased and go home peacefully. She has credence in three moral theories**, which, when taking the preceding probabilities into account, provide the following choice-worthiness orderings**:
35% credence in a variant of utilitarianism, according to which [Gâ»Iâ»R].
34% credence in a variant of common sense, according to which [I>Râ»G].
31% credence in a deontological theory, according to which [Iâ»Râ»G].â
This leads to the Borda Scores and Credence-Weighted Borda Scores shown in the table below, and thus to the recommendation that Julia declare Smith innocent.
(More info on how that was worked out can be found in the following footnote, along with the corresponding table based on the moral theoriesâ preference orderings in my prior post, when empirical uncertainty wasnât taken into account.[9])
In the original example, both the utilitarian theory and the common sense theory preferred a retrial to a verdict of innocent (in order to avoid a riot), which resulted in calling for a retrial having the highest Credence-Weighted Borda Score.
However, Iâm now imagining that Julia is no longer assuming each action 100% guarantees a certain outcome will occur, and paying attention to her empirical uncertainty has changed her conclusions.
In particular, Iâm imagining that she realises sheâd initially been essentially ârounding upâ (to 100%) the likelihood of a riot if she provides a verdict of innocent, and ârounding downâ (to 0%) the likelihood of the crowd rioting at a later date. However, with more realistic probabilities in mind, utilitarianism and common sense would both actually prefer an innocent verdict to a retrial (because the innocent verdict seems less risky, and the retrial more risky, than sheâd initially thought, while an innocent verdict still frees this innocent person sooner and with more certainty). This changes each actionâs Borda Score, and gives the result that she should declare Smith innocent.[10]
Potential extensions of these approaches
Does this approach presume/âprivilege consequentialism?
A central idea of this post has been making a clear distinction between âactionsâ (which one can directly choose to take) and their âoutcomesâ (which are often what moral theories âintrinsically care aboutâ). This clearly makes sense when the moral theories one has credence in are consequentialist. However, other moral theories may âintrinsically careâ about actions themselves. For example, many deontological theories would consider lying to be wrong in and of itself, regardless of what it leads to. Can the approaches Iâve proposed handle such theories?
Yesâand very simply! For example, suppose I wish to use MEC-E (or Normalised MEC-E), and I have credence in a (cardinal) deontological theory that assigns very low choice-worthiness to lying (regardless of outcomes that action leads to). We can still calculate expected choice-worthiness using the formulas shown above; in this case, we find the product of (multiply) âprobability me lying leads to me having liedâ (which weâd set to 1), âchoice-worthiness of me having lied, according to this deontological theoryâ, and âcredence in this deontological theoryâ.
Thus, cases where a theory cares intrinsically about the action and not its consequences can be seen as a âspecial caseâ in which the approaches discussed in this post just collapse back to the corresponding approaches discussed in MacAskillâs thesis (which these approaches are the âgeneralisedâ versions of). This is because thereâs effectively no empirical uncertainty in these cases; we can be sure that taking an action would lead to us having taken that action. Thus, in these and other cases of no relevant empirical uncertainty, accounting for empirical uncertainty is unnecessary, but creates no problems.[11][12]
Iâd therefore argue that a policy of using the generalised approaches by default is likely wise. This is especially the case because:
One will typically have at least some credence in consequentialist theories.
My impression is that even most ânon-consequentialistâ theories still do care at least somewhat about consequences. For example, theyâd likely say lying is in fact ârightâ if the negative consequences of not doing so are âlarge enoughâ (and one should often be empirically uncertain about whether they would be).
Factoring things out further
In this post, I modified examples (from my prior post) in which we had only one moral uncertainty into examples in which we had one moral and one empirical uncertainty. We could think of this as âfactoring outâ what originally appeared to be only moral uncertainty into its âfactorsâ: empirical uncertainty about whether an action will lead to an outcome, and moral uncertainty about the value of that outcome. By doing this, weâre more closely approximating (modelling) our actual understandings and uncertainties about the situation at hand.
But weâre still far from a full approximation of our understandings and uncertainties. For example, in the case of Julia and the innocent Smith, Julia may also be uncertain how big the riot would be, how many people would die, whether these people would be rioters or uninvolved bystanders, whether thereâs a moral difference between a rioter vs a bystanders dying from the riot (and if so, how big this difference is), etc.[13]
A benefit of the approaches shown here is that they can very simply be extended, with typical modelling methods, to incorporate additional uncertainties like these. You simply disaggregate the relevant variables into the âfactorsâ you believe theyâre composed of, assign them numbers, and multiply them as appropriate.[14][15]
Need to determine whether uncertainties are moral or empirical?
In the examples given just above, you may have wondered whether I was considering certain variables to represent moral uncertainties or empirical ones. I suspect this ambiguity will be common in practice (and I plan to discuss it further in a later post). Is this an issue for the approaches Iâve suggested?
Iâm a bit unsure about this, but I think the answer is essentially ânoâ. I donât think thereâs any need to treat moral and empirical uncertainty in fundamentally different ways for the sake of models/âcalculations using these approaches. Instead, I think that, ultimately, the important thing is just to âfactor outâ variables in the way that makes the most sense, given the situation and what the moral theories under consideration âintrinsically care aboutâ. (An example of the sort of thing I mean can be found in footnote 14, in a case where the uncertainty is actually empirical but has different moral implications for different theories.)
Probability distributions instead of point estimates
You may have also thought that a lot of variables in the examples Iâve given should be represented by probability distributions (e.g., representing 90% confidence intervals), rather than point estimates. For example, why would Devon estimate the probability of âfish being harmedâ, as if itâs a binary variable whose moral significance switches suddenly from 0 to â100 (according to T1) when a certain level of harm is reached? Wouldnât it make more sense for him to estimate the amount of harm to fish that is likely, given that that better aligns both with his understanding of reality and with what T1 cares about?
If you were thinking this, I wholeheartedly agree! Further, I canât see any reason why the approaches Iâve discussed couldnât use probability distributions and model variables as continuous rather than binary (the only reason I havenât modelled things in that way so far was to keep explanations and examples simple). For readers interested in an illustration of how this can be done, Iâve provided a modified model of the Devon example in this Guesstimate model. (Existing models like this one also take essentially this approach.)
Closing remarks
I hope youâve found this post useful, whether to inform your heuristic use of moral uncertainty and expected value reasoning, to help you build actual models taking into account both moral and empirical uncertainty, or to give you a bit more clarity on âmodellingâ in general.
In the next post, Iâll discuss how we can combine the approaches discussed in this and my prior post with sensitivity analysis and value of information analysis, to work out what specific moral or empirical learning would be most decision-relevant and when we should vs shouldnât postpone decisions until weâve done such learning.
- â©ïž
What âchoice-worthinessâ, âcardinalâ (vs âordinalâ), and âintertheoretically comparableâ mean is explained in the previous post. To quickly review, roughly speaking:
Choice-worthiness is the rightness or wrongness of an action, according to a particular moral theory.
A moral theory is ordinal if it tells you only which options are better than which other options, whereas a theory is cardinal if it tells you how big a difference in choice-worthiness there is between each option.
A pair of moral theories can be cardinal and yet still not intertheoretically comparable if we cannot meaningfully compare the sizes of the âdifferences in choice-worthinessâ between the theories; basically, if thereâs no consistent, non-arbitrary âexchange rateâ between different theoriesâ âunits of choice-worthinessâ.
- â©ïž
MacAskill also discusses a âHybridâ procedure, if the theories under consideration differ in whether theyâre cardinal or ordinal and/âor whether theyâre intertheoretically comparable; readers interested in more information on that can refer to pages 117-122 MacAskillâs thesis. An alternative approach to such situations is Christian Tarsneyâs (pages 187-195) âmulti-stage aggregation procedureâ, which I may write a post about later (please let me know if you think thisâd be valuable).
- â©ïž
Examples of models that effectively use something like the âMEC-Eâ approach include GiveWellâs cost-effectiveness models and this model of the cost effectiveness of âalternative foodsâ.
And some of the academic moral uncertainty work Iâve read seemed to indicate the authors may be perceiving as obvious something like the approaches I propose in this post.
But I think the closest thing I found to an explicit write-up of this sort of way of considering moral and empirical uncertainty at the same time (expressed in those terms) was this post from 2010, which states: âUnder Robinâs approach to value uncertainty, we would (I presume) combine these two utility functions into one linearly, by weighing each with its probability, so we get EU(x) = 0.99 EU1(x) + 0.01 EU2(x)â.
- â©ïž
Some readers may be thinking the âempiricalâ uncertainty about fish consciousness is inextricable from moral uncertainties, and/âor that the above paragraph implicitly presumes/âprivileges consequentialism. If youâre one of those readers, 10 points to you for being extra switched-on! However, I believe these are not really issues for the approaches outlined in this post, for reasons outlined in the final section.
- â©ïž
Note that my usage of âactionsâ can include âdoing nothingâ, or failing to do some specific thing; I donât mean âactionsâ to be distinct from âomissionsâ in this context. MacAskill and other writers sometimes refer to âoptionsâ to mean what I mean by âactionsâ. I chose the term âactionsâ both to make it more obvious what the A and O terms in the formula stand for, and because it seems to me that the distinction between âoptionsâ and âoutcomesâ would be less immediately obvious.
- â©ïž
My university education wasnât highly quantitative, so itâs very possible Iâll phrase certain things like this in clunky or unusual ways. If you notice such issues and/âor have better phrasing ideas, please let me know.
- â©ïž
In that link, the model using MEC-E follows a similar model using regular MEC (and thus considering only moral uncertainty) and another similar model using more traditional expected value reasoning (and thus considering only empirical uncertainty); readers can compare these against the MEC-E model.
- â©ïž
Before I tried to actually model an example, I came up with a slightly different proposal for integrating the ideas of MEC-E and Normalised MEC. Then I realised the proposal outlined above might make more sense, and it does seem to work (though Iâm not 100% certain), so I didnât further pursue my original proposal. I therefore donât know for sure whether my original proposal would work or not (and, if it does work, whether itâs somehow better than what I proposed above). My original proposal was as follows:
Work out expected choice-worthiness just as with regular MEC-E; i.e., follow the formula from above to incorporate consideration of the probabilities of each action leading to each outcome, the choice-worthiness of each outcome according to each moral theory, and the credence one has in each theory. (But donât yet pick the action with the maximum expected choice-worthiness score.)
Normalise these expected choice-worthiness scores by variance, just as MacAskill advises in the quote above. (The fact that these scores incorporate consideration of empirical uncertainty has no impact on how to normalise by variance.)
Now pick the action with the maximum normalised expected choice-worthiness score.
- â©ïž
G (for example) has a Borda Scoreof 2 â 0 = 2 according to utilitarianism because that theory views two options as less choice-worthy than G, and 0 options as more choice-worthy than G.
To fill in the final column, you take a credence-weighted average of the relevant actionâs Borda Scores.
What follows is the corresponding table based on the moral theoriesâ preference orderings in my prior post, when empirical uncertainty wasnât taken into account:
- â©ïž
Itâs also entirely possible for paying attention to empirical uncertainty to not change any moral theoryâs preference orderings in a particular situation, or for some preference orderings to change without this affecting which action ends up with the highest Credence-Weighted Borda Score. This is a feature, not a bug.
Another perk is that paying attention to both moral and empirical uncertainty also provides more clarity on what the decision-maker should think or learn more about. This will be the subject of my next post. For now, a quick example is that Julia may realise that a lot hangs on what each moral theoryâs preference ordering should actually be, or on how likely the crowd actually is to riot if she passes a verdict or innocent or calls for a retrial, and it may be worth postponing her decision in order to learn more about these things.
- â©ïž
Arguably, the additional complexity in the model is a cost in itself. But this is only a problem only in the same way this is a problem for any time one decides to model something in more detail or with more accuracy at the cost of adding complexity and computations. Sometimes itâll be worth doing so, while other times itâll be worth keeping things simpler (whether by considering only moral uncertainty, by considering only empirical uncertainty, or by considering only certain parts of oneâs moral/âempirical uncertainties).
- â©ïž
The approaches discussed in this post can also deal with theories that âintrinsically careâ about other things, like a decision-makerâs intentions or motivations. You can simply add in a factor for âprobability that, if I take X, itâd be due to motivation Y rather than motivation Zâ (or something along those lines). It may often be reasonable to round this to 1 or 0, in which case these approaches didnât necessarily âadd valueâ (though they still worked). But often we may genuinely be (empirically) uncertain about our own motivations (e.g., are we just providing high-minded rationalisations for doing something we wanted to do anyway for our own self-interest?), in which case explicitly modelling that empirical uncertainty may be useful.
- â©ïž
For another example, in the case of Devon choosing a meal, he may also be uncertain how many of each type of fish will be killed, the way in which theyâd be killed, whether each type of fish has certain biological and behavioural features thought to indicate consciousness, whether those features do indeed indicate consciousness, whether the consciousness they indicate is morally relevant, whether creatures with consciousness like that deserve the same âmoral weightâ as humans or somewhat lesser weight, etc.
- â©ïž
For example, Devon might replace âProbability that purchasing a fish meal leads to âfish being harmedââ with (âProbability that purchasing a fish meal leads to fish being killedâ * âProbability fish who were killed would be killed in a non-humane wayâ * âProbability any fish killed in these ways would be conscious enough that this can count as âharmingâ themâ). This whole term would then be in calculations used wherever ââProbability that purchasing a fish meal leads to âfish being harmedââ was originally used.
For another example, Julia might replace âProbability the crowd riots if Julia finds Smith innocentâ with âProbability the crowd riots if Julia finds Smith innocentâ * âProbability a riot would lead to at least one deathâ * âProbability that, if at least one death occurs, thereâs at least one death of a bystander (rather than of one of the rioters themselves)â (as shown in this partial Guesstimate model). She can then keep in mind this more specific final outcome, and its more clearly modelled probability, as she tries to work out what choice-worthiness ordering each moral theory she has credence in would give to the actions sheâs considering.
Note that, sometimes, it might make sense to âfactor outâ variables in different ways for the purposes of different moral theoriesâ evaluations, depending on what the moral theories under consideration âintrinsically care aboutâ. In the case of Julia, it definitely seems to me to make sense to replace âProbability the crowd riots if Julia finds Smith innocentâ with âProbability the crowd riots if Julia finds Smith innocentâ * âProbability a riot would lead to at least one deathâ. This is because all moral theories under consideration probably care far more about potential deaths from a riot than about any other consequences of the riot. This can therefore be considered an âempirical uncertaintyâ, because its influence on the ultimate choice-worthiness âflows throughâ the same âmoral outcomeâ (a death) for all moral theories under consideration.
However, it might only make sense to further multiply that term by âProbability that, if at least one death occurs, thereâs at least one death of a bystander (rather than of one of the rioters themselves)â for the sake of the common sense theoryâs evaluation of the choice-worthiness order, not for the utilitarian theoryâs evaluation. This would be the case if the utilitarian theory cared not at all (or at least much less) about the distinction between the death of a rioter and the death of a bystander, while common sense does. (The Guesstimate model should help illustrate what I mean by this.)
- â©ïž
Additionally, the process of factoring things out in this way could by itself provide a clearer understanding of the situation at hand, and what the stakes really are for each moral theory one has credence in. (E.g., Julia may realise that passing a verdict of innocent is much less bad than she thought, as, even if a riot does occur, thereâs only a fairly small chance it leads to the death of a bystander.) It also helps one realise what uncertainties are most worth thinking/âlearning more about (more on this in my next post).
Maybe worth noting that ordinal preferences and a probability distribution over empirical outcomes (the setup in âBR under empirical uncertaintyâ) are used to generate cardinal preferences in the vNM utility theorem.
Interesting. I hadnât explicitly made that connection, but it does seem worth thinking about.
I donât know if this is what you were implying, but that made me wonder about whether what I wrote in this post effectively entails that we could âcardinaliseâ the preferences of the ordinal theories under consideration. My first impression is that we probably still canât/âshouldnât, but Iâm actually not sure about that, so hereâs some long-winded thinking-aloud on the matter.
In MacAskillâs thesis, he discusses a related matter:
And later he adds:
So it seems to me that heâs arguing that we should respect that the theory is really meant to be ordinal, and we shouldnât force cardinality upon it.
Which leaves me with an initial, unclear thought along the lines of:
But to be honest this seems to run into a bit of a roadblock related to me not really understanding how ordinal moral theories are really meant to work. I think thatâs a hard issue to avoid in general when thinking about moral uncertainty. There are these theories that seem like they just canât really be made to give us consistent, coherent preferences or follow axioms of rationality or whatever. But some of these theories are also very popular, including among professional philosophers, so, given epistemic humility, it does seem like itâs worth trying to take them seriouslyâand trying to take them seriously for what they claim themselves to be (i.e., ordinal and arguably irrational).
(Plus thereâs the roadblock of me not having in-depth understanding of how the vNM utility theorem is meant to work.)
Additionally, in any case, itâs also possible that MacAskillâs Borda Rule effectively does implicitly cardinalize the theories. Tarsney seems to argue this, e.g.:
If Iâm interpreting Tarsney correctly, and if heâs right, then that may be why when you poke around and consider empirical uncertainties it starts to look a lot like a typical method for getting cardinal preferences from preference orderings. But I havenât read that section of Tarsneyâs thesis properly, so Iâm not sure.
(I may later write a post about Tarsneyâs suggested approach for making decision under moral uncertainty, which seems to have some advantages, and may also more fully respect ordinal theories ordinality.)
Yeah, I think this is where Iâm at too. It seems inescapable that ordinal preferences have cardinal implications when combined with empirical uncertainty (e.g. if I prefer a 20% chance of A to an 80% chance of B, that implies I like A at least four times as much). The only choice we really have is whether the corresponding cardinal implications are well-formed (e.g. Dutch bookable). The best distinctions I can come up with are:
In a purely deterministic world without lotteries, there wouldnât be an obvious mechanism forcing the cardinalization of ordinal preferences. So their overlap is only a contingent feature of the world we find ourselves in. (Though see A Theory of Experienced Utility and Utilitarianism for an alternate basis for cardinalization.)
Ordinal preferences only specify a unique cardinalization in the limit of an infinite sequence of choices. Since we arenât likely to face an infinite sequence of choices any time soon, theyâre distinct in practice.
P.S. Thanks for the Tarsney link. I have it open in a tab and should get around to reading it at some point.
Not sure if itâll help but I have a short explanation and interactive widget trying to explain it here.
Those are two interesting distinctions. I donât have anything to add on that, but thanks for sharing those thoughts.
Oh, youâre the person who made this value of information widget! I stumbled upon that earlier somehow, and am likely to link to it in a later post on applying VoI ideas to moral uncertainty.
Thanks for sharing the vNM widget; I intend to look at that soon.