Hedging against deep and moral uncertainty
Summary: Like for quantified risk, we can sometimes hedge against deep uncertainty and moral uncertainty: we can sometimes choose a portfolio of interventions which looks good in expectation to all (or more) worldviewsâempirical and ethical beliefsâwe find plausible, even if each component intervention is plausibly harmful or not particularly good in expectation according to some plausible worldview. We can sometimes do better than nothing in expectation when this wasnât possible by choosing a single intervention, and we can often improve the minimum expected value. I think doing so can therefore sometimes reduce complex cluelessness.
My recommendations are the following:
We should, when possible, avoid portfolios (and interventions) which are robustly dominated by any other in expectation - those worse in expectation than another under some plausible worldview and no worse on any plausible worldview (or those ruled out by the maximality rule; EA Forum post on the paper). I think this is rationally required under consequentialism, assuming standard rationality axioms under uncertainty.
I further endorse choosing portfolios among those that are robustly positive in expectationâbetter in expectation than doing nothing under all worldviews we find plausibleâif any are available. However, this is more a personal preference than a (conditional) requirement like 1, although I think something thatâs often implicitly assumed in EA. I think this would lead us to allocate more to work for nonhuman animals and s-risks.
EAs should account for interactions between causes and conflicts in judgements about the sign of the expected value of different interventions according to different worldviews. I think this is being somewhat neglected as many EA organizations or divisions in EA organizations are cause-specific.
Approaches for moral uncertainty and deep uncertainty are better applied to portfolios of interventions than to each intervention in isolation, since portfolios can promote win-wins.
We should not commit to priors arbitrarily. If you donât feel justified in choosing one prior over all others (see the reference class problem), this is what sensitivity analysis and other approaches to decision making under deep uncertainty are for, and sometimes hedging can help, as I hope to illustrate in this post.
EDIT: Itâs not always necessary to hedge against the negative side effects of our interventions we choose at the same time as we work on or fund those interventions. We can sometimes address them later or assume we (or others) will.
Introduction
EAs have often argued against diversification and for funding only the most cost-effective intervention, at least for individual donors where the marginal returns on donations are roughly constant. However, this assumes away a lot of uncertainty we could have; we might not believe any specific intervention is the most cost-effective. Due to deep uncertainty, we might not be willing to commit to a specific single joint probability distribution for the effects of our interventions, since we canât justify any choice over all others. Due to moral uncertainty, we might not be confident in how to ethically value different outcomes or actions. This can result in complex cluelessness, according to which we just donât know whether we should believe a given intervention is better or worse than another in expectation; it could go either way.
Sometimes, using a portfolio of interventions can be robustly better in expectation than doing nothing, while none of the best individual interventions according to some worldview are, since theyâre each plausibly harmful in expectation (whether or not weâre committed to the claim that they definitely are harmful in expectation, since we may have deep or moral uncertainty about that). For example, cost-effective work in one cause might plausibly harm another cause more in expectation, and we donât know how to trade off between the two causes.
We might expect to find such robustly positive portfolios in practice where the individual interventions are not robust, because the interventions most robustly cost-effective in one domain, effect or worldview will not systematically be the most harmful in others, and if they arenât so harmful that they canât be cost-effectively compensated for with interventions optimized for cost-effectiveness in those other domains, effects or worldviews. The aim of this post is to make a more formal and EA-relevant illustration of the following reason for hedging:
We can sometimes choose portfolios which look good to all (or more) worldviews we find plausible, even if each component intervention is plausibly harmful or not particularly good in expectation according to some plausible worldview.
Diversification is of course not new in the EA community; itâs an approach taken by Open Phil, and this post builds upon their âStrong uncertaintyâ factor, although most organizations tend not to consider the effects of interventions on non-target causes/âworldviews, where hedging becomes useful.
An illustrative example
I will assume, for simplicity, constant marginal cost-effectiveness across each domain/âeffect/âworldview, and that the effects of the different interventions are independent of one another. Decreasing marginal cost-effectiveness is also a separate reason for diversification, so by assuming a constant rate (which I expect is also approximately true for small donors), we can consider the uncertainty argument independently. (Thanks to Michael_Wiebe for pointing this out.) EDIT: Iâm also assuming away moral uncertainty (e.g. about relative moral weight of nonhuman animals vs humans, conditional on empirical facts) here; see this article for some discussion.
Suppose you have deep or moral uncertainty about the effects of a given global health and poverty intervention on nonhuman animals, farmed or wild, enough uncertainty that your expected value for the intervention ranges across positive and negative values, where the negative comes from effects on nonhuman animals, due to moral empirical uncertainty related to how to weigh the experiences of nonhuman animals and welfare in the wild, the meat eater problem (it may increase animal product consumption) and deep uncertainty about the effects on wild animals.
You could represent the expected cost-effectiveness across these components as a set of vectors with ranges. Assuming independent effects on each component, you might write this as the following set, a box in 3 dimensions:
Here, the first component, , is the range of expected cost-effectiveness for the humans (living in poverty), the second, , for farmed animals, and the third, , for wild animals. These arenât necessarily in the same comparable utility units across these three components. The point is that two of the components are plausibly negative in expectation, while the first is only positive in expectation, and itâs plausible that the intervention does more harm than good in expectation or does more good than harm in expectation. (Depending on your kind of uncertainty, you might be able to just add each of the components instead, e.g. as , but I will continue to illustrate with separate components, since thatâs more general and can capture deeper uncertainty and worse moral uncertainty.)
You might also have an intervention targeting farmed animals and deep or moral uncertainty about its effects on wild animals. Suppose you represent the expected cost-effectiveness as follows, with the effects on humans first, then on farmed animals and then on wild animals, as before:
You represent the expected effectiveness for a wild animal intervention as follows, with the effects on humans first, then on farmed animals and then on wild animals:
And finally, you have a default âdo nothingâ or âbusiness as usualâ option, e.g. spending selfishly:
I model as all 0s, since Iâm considering the differences in value with doing nothing, not the expected value in the universe.
Now, based on this example, we arenât confident that any of these interventions are better in expectation than , doing nothing, and generally, none of them definitely beat any other in expectation, so on this basis, we might say all of them are permissible according to the maximality rule. However, there are portfolios of these interventions that are better than doing nothing. Assuming a budget of 10 units, one such portfolio (better than ) is the following:
That is, you spend 4 times as much on as , and 5 times as much on as . We can divide by 10 to normalize. Notice that each components is strictly positive, so that this portfolio is good in expectationâbetter than (and ) - for humans, farmed animals and wild animals simultaneously.
According to recommendation 1, , doing nothing, is now ruled out, impermissible. This does not depend on the fact that I took differences with doing nothing, since we can shift them all the same way.
According to recommendation 2, which I only weakly endorse, each individual intervention is now ruled out, since each was plausibly negative (compared to doing nothing) in expectation, and we must choose among the portfolios that are robustly positive in expectation. Similarly, portfolios with only two interventions are also ruled out, since at least one of their components will have negative values in its range.
I constructed this example by thinking about how I could offset each interventionâs harms with anotherâs. The objection that offsetting is suboptimal doesnât apply, since, by construction, I canât decide which of the interventions is best in expectation, although I know itâs not doing nothing.
Note also that the cost-effectiveness values do not depend on when the effects occur. Similarly, we can hedge over time: the plausible negative effects of one intervention can be made up for with positive effects from another that occur far earlier or later in time.
Dependence
We assumed the interventionsâ components were independent of one another and of the other interventionsâ components. With dependence, all the portfolios that were robustly at least as good as doing nothing will still be robustly as good as doing nothing, since the lower bounds under the independent case are lower bounds for the dependent case, but we could have more such portfolios. On the other hand, different portfolios could become dominated by others when modelling dependence that werenât under the assumption of independence.
Lexicality and deontological constraints
Under some deontological ethical theories, rule violations (you commit) canât be compensated for, no matter how small. You could represent rule violations as - or multiples of it, without multiplying through, or using vectors for individual components to capture lexicality. Portfolios that include interventions that violate some rule will generally also violate that rule. However, we should be careful to not force cardinalization on theories that are only meant to be ordinal and do not order risky lotteries according to standard rationality axioms; see some quotes from MacAskillâs thesis here on this, and this section from MichaelAâs post.
Other potential examples
Pairing human life-saving interventions with family planning interventions can potentially minimize externalities due to human population sizes, which we may have deep uncertainty about (although this requires taking a close look at the population effects of each, and it may not work out). These interventions could even target different regions based on particular characteristics, e.g. average quality of life, meat consumption. Counterfactually reducing populations where average welfare is worse (or meat consumption is higher) and increasing it the same amount where itâs better (or meat consumption is lower) increases average and total human welfare (or total farmed animal welfare, assuming net negative lives) without affecting human population size. Of course, this is a careful balancing act, especially under deep uncertainty. Furthermore, there may remain other important externalities.
We might find it plausible that incremental animal welfare reform contributes to complacency and moral licensing and have deep uncertainty about whether this is actually the case in expectation, but we might find more direct advocacy interventions that can compensate for this potential harm so that their combination is robustly positive.
Extinction risk interacts with animal welfare in many ways: extinction would end factory farming, could wipe out all wild animals if complete, could prevent us from addressing wild animal suffering if only humans go extinct, and if we donât go extinct, we could spread animal suffering to other planets. There are other interactions and correlations with s-risks, too, since things that risk extinction could also lead to far worse outcomes (e.g. AI risk, conflict), or could prevent s-risks.
Animal advocacy seems good for reducing s-risks due to moral circle expansion, but there are also plausible effects going in the opposite direction, like correlations with environmentalism or âwrongâ population ethics or near-misses.
In the wild animal welfare space, Iâve been told about pairing interventions that reduce painful causes of death with population control methods to get around uncertainty about the net welfare in the wild. In principle, with a portfolio approach, it may not be necessary to pair these interventions on the same population to ensure a positive outcome in expectation, although applying them to the same population may prevent ecological risks and reduce uncertainty further.
Substitution effects between animal products. We might have moral uncertainty about the sign of the expected value of an intervention raising the price of fish, in case it leads consumers to eat more chicken, and similarly for an intervention raising the price of chicken, in case it leads consumers to eat more fish. Combining both interventions can reduce both chicken and fish consumption. As before, these interventions do not have to even target the same region, as long as the increase in fish consumption in the one region is smaller than the increase in the other (assuming similar welfare, amount of product per animal, etc. or taking these into account), and the same for chicken consumption.
Questions and possible implications
I think recommendation 2 would push us partially away from global health and poverty work and extinction risk work and towards work for nonhuman animals and s-risks, due to the interactions I discuss above.
Should we choose portfolios as individuals or as a community? If as a community, and we endorse recommendation 2 for the community, i.e. we should do robustly better in expectation than doing nothing, individuals may be required to focus on plausible domains/âworldviews/âeffects according to which the community is plausibly doing more harm than good in expectation, if any exists. This could mean many more EAs should focus on work for nonhuman animals and s-risks, since global health and poverty work and extinction risk work, some of the largest parts of the EA portfolio, are plausibly net negative due to interactions with these.
I personally doubt that we have fundamental reasons to decide as a community (coordination and cooperation are instrumental reasons). Either our (moral) reasons are agent-relative or agent-neutral/âuniversal; they are not relative to some specific and fairly arbitrarily defined group like the EA community.
Should we model the difference compared to doing nothing and use doing nothing as a benchmark as I endorse in recommendation 2, or just model the overall outcomes under each intervention (or, more tractably, all pairwise differences, allowing us to ignore whatâs unaffected)? What I endorse seems similar to similar risk-aversion with respect to the difference you make by centering the agent, which Snowden claims is incompatible with impartiality. In this case, rather than risk-aversion, itâs closer to uncertainty/âambiguity aversion. It also seems non-consequentialist, since it treats one option differently from the rest, and consequentialism usually assumes no fundamental difference between acts and omissions (and the concept of omission itself may be shaky).
What other plausible EA-relevant examples are there where hedging can help by compensating for plausible expected harms?
Can we justify stronger rules if we assume more structure to our uncertainty, short of specifying full distributions? What if I think one worldview is more likely than another, but I canât commit to actual probabilities? What if Iâm willing to say something about the difference or ratio of probabilities?
- ParÂtiÂciÂpate in the DonaÂtion ElecÂtion and the first weekly theme (startÂing 7 NovemÂber) by 2 Nov 2023 17:02 UTC; 85 points) (
- Should you go with your best guess?: Against preÂcise BayesiÂanism and reÂlated views by 27 Jan 2025 20:25 UTC; 77 points) (
- Should you go with your best guess?: Against preÂcise BayesiÂanism and reÂlated views by 27 Jan 2025 20:25 UTC; 65 points) (LessWrong;
- âLongterÂmist causesâ is a tricky classification by 29 Aug 2023 17:41 UTC; 63 points) (
- 3 Nov 2020 7:07 UTC; 58 points) 's comment on EvÂiÂdence, clueÂlessÂness, and the long termâHilary Greaves by (
- DonaÂtion ElecÂtion Fund AnÂnounceÂment: MatchÂing, ReÂwards and FAQ. by 1 Nov 2024 19:09 UTC; 48 points) (
- Why does (any parÂticÂuÂlar) AI safety work reÂduce s-risks more than it inÂcreases them? by 3 Oct 2021 16:55 UTC; 48 points) (
- 4. Why exÂistÂing apÂproaches to cause priÂoriÂtiÂzaÂtion are not roÂbust to unawareness by 2 Jun 2025 8:55 UTC; 35 points) (
- 17 Oct 2022 15:52 UTC; 28 points) 's comment on Ask CharÂity EnÂtrepreneurÂship Anything by (
- 15 Jun 2024 5:39 UTC; 27 points) 's comment on What âEffecÂtive AltruÂismâ Means to Me by (
- 4. Why exÂistÂing apÂproaches to cause priÂoriÂtiÂzaÂtion are not roÂbust to unawareness by 13 Jun 2025 8:55 UTC; 26 points) (LessWrong;
- 4 Jun 2025 4:10 UTC; 25 points) 's comment on Cost-effecÂtiveÂness acÂcountÂing for soil neÂmaÂtodes, mites, and springtails by (
- 3 Jun 2025 19:22 UTC; 25 points) 's comment on Cost-effecÂtiveÂness acÂcountÂing for soil neÂmaÂtodes, mites, and springtails by (
- Is this a good way to bet on short timelines? by 28 Nov 2020 14:31 UTC; 16 points) (
- Even AlloÂcaÂtion StratÂegy unÂder High Model Ambiguity by 31 Dec 2020 9:10 UTC; 15 points) (
- 3 Jun 2025 18:15 UTC; 13 points) 's comment on Cost-effecÂtiveÂness acÂcountÂing for soil neÂmaÂtodes, mites, and springtails by (
- Post on maxÂiÂmizÂing EV by diÂverÂsifyÂing w/â deÂclinÂing marginal reÂturns and uncertainty by 11 Aug 2021 4:32 UTC; 12 points) (
- 6 Mar 2023 15:08 UTC; 12 points) 's comment on Model-Based Policy AnalÂyÂsis unÂder Deep Uncertainty by (
- 22 Oct 2022 18:25 UTC; 10 points) 's comment on ConÂseÂquenÂtialÂism and Cluelessness by (
- 14 Feb 2024 22:43 UTC; 9 points) 's comment on SumÂmary: MaxÂiÂmal ClueÂlessÂness (AnÂdreas MoÂgensen) by (
- Iâm inÂterÂviewÂing Oxford philosoÂpher, global priÂoriÂties reÂsearcher and early thinker in EA, AnÂdreas MoÂgensen. What should I ask him? by 10 Jun 2022 14:24 UTC; 9 points) (
- EA toÂwards huÂmans = effecÂtive viÂoÂlence toÂwards farm anÂiÂmals? by 3 Dec 2020 17:22 UTC; 7 points) (
- 16 Feb 2021 19:49 UTC; 4 points) 's comment on ComÂplex clueÂlessÂness as credal fragility by (
- 12 Mar 2021 7:04 UTC; 4 points) 's comment on ComÂplex clueÂlessÂness as credal fragility by (
- 18 Feb 2023 18:21 UTC; 4 points) 's comment on The deathÂprint of reÂplacÂing beef by chicken and inÂsect meat by (
- 23 Oct 2020 20:22 UTC; 4 points) 's comment on Use reÂsilience, inÂstead of imÂpreÂciÂsion, to comÂmuÂniÂcate uncertainty by (
- 3 Nov 2024 20:17 UTC; 4 points) 's comment on Why I No Longer PriÂoriÂtize Wild AnÂiÂmal Welfare by (
- 21 Dec 2022 18:34 UTC; 3 points) 's comment on A Case for VolÂunÂtary AborÂtion Reduction by (
- 27 Dec 2020 9:52 UTC; 2 points) 's comment on Strong LongterÂmism, IrÂrefutabilÂity, and MoÂral Progress by (
- 2 Jan 2023 3:03 UTC; 2 points) 's comment on The propÂerty rights apÂproach to moral uncertainty by (
- 27 Aug 2021 3:44 UTC; 2 points) 's comment on Changes in conÂdiÂtions are a priÂori bad for avÂerÂage anÂiÂmal welfare by (
- 23 Dec 2020 17:55 UTC; 2 points) 's comment on A case against strong longtermism by (
- 6 May 2021 2:22 UTC; 2 points) 's comment on Thoughts on âA case against strong longterÂmismâ (MasÂrani) by (
- 6 May 2021 7:13 UTC; 2 points) 's comment on Thoughts on âA case against strong longterÂmismâ (MasÂrani) by (
- 14 Feb 2024 22:42 UTC; 2 points) 's comment on SumÂmary: MaxÂiÂmal ClueÂlessÂness (AnÂdreas MoÂgensen) by (
- 12 Nov 2023 21:58 UTC; 2 points) 's comment on Zach Stein-PerÂlÂmanâs Quick takes by (
- 15 May 2021 2:41 UTC; 2 points) 's comment on AnÂiÂmal Welfare Fund: Ask us anyÂthing! by (
- 13 Sep 2020 22:07 UTC; 2 points) 's comment on How to think about an unÂcerÂtain fuÂture: lesÂsons from other secÂtors & misÂtakes of longterÂmist EAs by (
Thanks Michael for the post. I happened to be thinking in similar terms recently regarding how to divide donations between saving human lives and increasing welfare of farmed animals (though nothing like as thoroughly and generally). I thought perhaps this could be an interesting real-world example to analyse:
This review estimated that saving a life in a very poor country would result in a reduction in births of 0.33-0.5, hence giving 0.5-0.67 extra lives. Though, the uncertainty in the various studies included indicates to me it could plausibly give 1 extra life.
1 extra human life perhaps gives ~60 years of extra life (not counting extra descendants, so maybe itâs an underestimate).
Then I remember reading estimates that for every typical westerner there are around 5-10 farmed animals alive at a given time to produce the animals products they eat (though I canât remember the source, and Iâm not sure if this includes fish). An extra person in a developing world country isnât going to consume as much as a typical westerner straight away of course, but I suppose the long-term consumption could be this large if they or their descendants reached present western levels of wealth and factory farming was still prevalent then, so it could be a reasonable pessimistic estimate of the effect of saving a human life on the increase in the number of farmed animals.
This 2018 Founders Pledge report gives a mean estimate of the effect of The Humane Leagueâs corporate campaigns alone as â10 hen-years shift from battery cages to aviaries [equivalent] per dollar receivedâ [p.68].
Perhaps shifting a hen from a battery to aviary system could be taken to be a tenth as good as removing an animal from the system (just based on a not-very-informed intuition).
So an estimate of an amount to donate to THL to be likely to offset any negative impact on animal welfare from saving a human life (using pessimistic figures to give the rough upper end of the uncertainty range) is [no. of extra human lives] x [no. of extra human years lived per life] x [no. of farmed animals per person]/â[no. of animal years saved per $] = 1 x 60 x 10/â(10*0.1) = $600.
This is ~20% of the cost of saving one life through Malaria Consortium from GiveWellâs 2020 cost effectiveness analysis. So perhaps this indicates that if you wanted to donate to save lives from malaria but were worried about potential negative impacts on farm animal welfare, splitting donations between MC and THL in a 5:1 ratio would be an option robustly better than doing nothing. (But the THL fraction may need to be higher if impacts on fish, long-term impacts of increasing the human population or other things Iâve not thought of need to be included).
Does this sound reasonable?
(Edited to correct â4:1 ratioâ to â5:1 ratioâ)
I think the overall approach youâve taken is good, and itâs cool to see youâve worked through this. This is also the kind of example I had in mind, although I didnât bother to work with estimates.
I do think it would be better to use some projections for animal product consumption and fertility rates in the regions MC works in (I expect consumption per capita to increase and fertility to decrease) to include effects of descendants and changing consumption habits, since these plausibly could end up dominating the effects of MC, or at least on animals (and you also have to decide on your population ethics: does the happiness of the additional descendants contribute to the good compared to if they were never born?). Then, there are also timelines for alternatives proteins (e.g. here), but these are much more speculative to me.
I also personally worry that cage-free campaigns could be net negative in expectation (at least in the short-term, without further improvements), mostly since on-farm mortality rates are higher in cage-free systems. See some context and further discussion here. I believe that corporate campaigns work, though, so I think we could come up with a target for a corporate campaign that weâd expect to be robustly positive for animals. I think work for more humane slaughter is robustly positive. Family planning interventions might be the most promising, see this new charity incubated by Charity Entrepreneurship and their supporting report, including their estimated cost-effectiveness of:
â$144 per unintended birth avertedâ, and
â377 welfare points gained per dollar spentâ for farmed animals. (I donât know off-hand if theyâre including descendants or projected changes in consumption in this figure.)
However, this new charity doesnât have any track record yet, so itâs in some ways more speculative than GiveWell charities or THL. CE does use success probabilities in their models, but this is a parameter that you might want to do a sensitivity analysis to. (Disclosure: Iâm an animal welfare research intern for Charity Entrepreneurship.)
Finally, Founders Pledge did a direct comparison between THL and AMF, including sensitivity analysis to moral weights, that might be useful.
Thanks for your thoughts and the links. I agree that more consideration of long-term effects and population ethics seems important (also, I would have thought, for the impact of accelerating animal welfare improvements). I donât know anything to go on for quantitative estimates of long-term effects myself, though.
Regarding the possibility of cage-free campaigns as being net negative, I agree this sounds like a risk, so perhaps I was loose in saying donating a certain amount to THL could be ârobustly betterâ. Iâm not sure itâs going to be possible to be 100% sure that any set of interventions wonât have a negative impact, thoughâI was basically going for being able to feel âquite confidentâ that the impact on farmed animals wouldnât be negative (edit: given the assumptions Iâve madeâall things considered Iâm not as confident as that), and havenât been able yet to be precise about what that means.
Thinking about it, in general, it seems to me that the ranges of possible effects of interventions could be unbounded, so then youâd have to accept some chance of having a negative impact in the corresponding cause areas. Perhaps this is something your general framework could be augmented to take into account e.g. could one set a maximum allowed probability of having a negative effect in one cause area, or would it be sufficient to have a positive expected effect in each area?
So, itâs worth distinguishing between
quantified uncertainty, or, risk, when you can put a single probability on something, and
unquantified uncertainty, when you canât decide among multiple probabilities).
If thereâs a quantified risk of negative, but your expected value is positive under all of the worldviews you find plausible enough to consider anyway (e.g. for all cause areas), then youâre still okay under the framework I propose in this post. I am effectively suggesting that itâs sufficient to have a positive expected effect in each area (although there may be important considerations that go beyond cause areas).
However, you might have enough cluelessness that you canât find any portfolio thatâs positive in expected value under all plausible worldviews like this. That would suck, but I would normally accept continuing to look for robustly positive expected value portfolios as a good option (whether or not it is robustly positive).
Thanks for writing this, that was an interesting read!
Whether or not you think you can add separate components seems pretty important for the hedging approach.
Indeed, if a portfolio dominates the default on each individual component, then some interventions in the portfolio must dominate the default overall.[1] So if you can compare interventions based on their total effects, the existence of such portfolios imply that some interventions dominate the default. Intuitively then, you would prefer investing in one of those interventions over hedging? (Although a complication I havenât thought about is that you should compare interventions with one another too, unless you think the default has a privileged status.)
Given the above, a worry I have is that the hedging approach doesnât save us from cluelessness, because we donât have access to an overall-better-than-the-default intervention to begin with.
To put my two questions in more concrete terms:
In your example, how do we become confident that the wild animal intervention is worst-case net-positive compared to the default?
Given that weâre confident about (1), why prefer a portfolio to investing solely in the wild animal intervention?
Sketch of proof: Let wi be ressources allocated to intervention Ai in your portfolio, and let aji be the worst-case effect of intervention Ai on component j. Then
âi,jwiaji=âi(âjaji)wiâ„0and there is an intervention for which worst-case effects are in aggregate non-negative.
Hi Nicolas, thanks for commenting!
Ah, good point. (Youâre assuming the separate components can be added directly (or with fixed weights, say).)
I guess the cases where you canât add directly (or with fixed weights) involve genuine normative uncertainty or incommensurability. Or, maybe some cases of two envelopes problems where itâs too difficult or unjustifiable to set a unique common scale and use the Bayesian solution.
In practice, I may have normative uncertainty about moral weights between species.
If youâre risk neutral, probably. Maybe not if youâre difference-making risk averse. Perhaps helping insects is robustly positive in expectation, but highly likely to have no impact at all. Then you might like a better chance of positive impact, while maintaining 0 (or low) probability of negative impact.
For my illustration, thatâs right.
However, my illustration treats the components as independent, so that you can get the worst case on each of them together. But this need not be the case in practice. You could in principle have interventions A and B, both with ranges of (expected) cost-effectiveness [-1, 2], but whose sum is exactly 1. Let the cost-effectiveness of B be 1-âthe cost-effectiveness of Aâ. Having things cancel out so exactly and ending up with a range thatâs a single value is unrealistic, but I wonder if we could at least get a positive range this way.
Ya, the default doesnât seem privileged if youâre a consequentialist. See this post.
Great post, Michael! The more I have realised how uncertain the world is, the more I have come to appreciate this post.
I think y â 100 should be y â 50.
I think the sum of the 2nd component should be 200 + 50 + 50 = 300 (not 200 + 200 + 50 = 450).
Good point. Personally, I think our reasons are agent-neutral, i.e. that we should think about how to improve the portfolio of the universe, not our own portfolio or that of the EA community.
Thank you for the kind words and the corrections!
I think my framework/âillustration doesnât really handle moral uncertainty well, since it effectively assumes a particular normalization, but I think the general idea can still be useful in those cases, and you should consider compensating moral worldviews that are harmed by an intervention in your portfolio and/âor allocating less to interventions that are harmful on other moral worldviews than interventions that are neutral on other moral worldviews, all else equal, aiming for a portfolio that is robustly positive across these worldviews and not dominated.
I thought these ideas were interesting, but it would be useful to have a less technical and/âor more intuitive explanation.
Does the âOther potential examplesâ section help? Maybe I should have that section before the technical example?
I think this would be easier to explain with a two-sector model: ie, just H and F. Also, would it be easier to just work with algebra? Ie, H=[âa,b]Ă[âc,d].
How does this fit with H+4F+5W? Thatâs 10 units, no?
Itâs worth emphasizing that this assumption rules out the diminishing returns case for diversifying; this is a feature, since we want to isolate the uncertainty-case for diversifying.
I think it would get part of it across slightly more easily, although I donât think the burden is large. I think a 2-sector model might give the false impression that you should try to pair interventions so that each makes up for the negatives of the other, whereas with a good enough example for 3, people might more intuitively grasp that you have far more flexibility.
Iâd have to write down a bunch of inequalities to get a portfolio thatâs better than N and ensure that one even exists, which I think would be much harder to follow (and do, for me). I expect people would get that the general problem is a system of linear inequalities, although itâs not central to the point Iâm making.
Ha. Thanks for pointing this out! Iâll fix this.
Ya, this is a good point. Iâll mention this in the text.
Re algebra, are you defending the numbers you gave as reasonable? Otherwise, if weâre just making up numbers, might as well do the general case.
I think the general case (with the independence and constant marginal cost-effectiveness assumptions) will be harder to follow for some readers (and not easier to follow for anyone), much more work for me (Iâm not sure how I would approach it yet), and not general enough to be very useful. Even more generally, itâs a multi-objective linear program, which we would solve algorithmically, not symbolically for a closed form solution.
In the caculation below it is assumed that, we know at least one intervention for each cause area that has no negatives so that we use our funds to compensate this net positive with negatives from other interventions.
However isnât it hard to assume this? I personally canât think of an intervention that is definitely positive in the long term. It feels like instead of numbers there should be question marks due to complexity of everything. Am I missing something?
Hmm, I might be misunderstanding you, so let me know if this doesnât respond to what you have in mind.
I donât think Iâve assumed any intervention is definitely positive in the long run. I have assumed they are each net positive in expectation for at least one set of targets (or types of effects) we can identify ahead of time (just humans, just farmed animals or just wild animal).
I guess I didnât specify how far these sets of targets go into the future, so you could assume Iâm ignoring far future moral patients, or we can extend the example to include them, with interventions positive in expectation for some sets of far future targets.
The example is just illustrative, so I didnât want to complicate it with more than 3 sets of targets.
I think you didnât misunderstand me.
If you are not assuming the longterm effects, the intervals below are only for shorttermish effects. (this is what I got from your comment, correct please if Itâs wrong)
H=[100,200]Ă[â300,50]Ă[â50,50]
Isnât one of the main problems brought by deep moral uncertinity is about how an intervention itself is benefical for the target group or not?
An intervention effects far future. Far future effect is generally more important than short term effects. So even if our short therm analysis shows the intervention was beneficial it doesnât mean in aggregate it is net positive. Thus we donât know the impact of the intervention.
I believe this was the one of the main problems of moral uncertinity/âcluenessness. What I donât get is how the model can solve moral uncertinity if it does not takes far longterm effects into account.
Is the bolded part even possible? Is there interventions that are highly likely to be positive for the target group in the very far future?
P.S: Thank you for responding my comment this fast even if the post is 5 years old :)