Very interesting, thanks for writing it :) I had a brief chat with Opus 4.6 about your essay and it pointed out that the “robustness across maps” section is probably the most decision-relevant idea particularly under deep moral uncertainty, but also that the literature on robustness is less useful in practice than one might hope, working through cases in global health / AI safety / insect welfare / x-risk mitigation to illustrate. Opus concludes (mods, let me know if this longform quote is low-value AI slop and I’ll remove it):
A “robustness across maps” strategy — favoring actions that look good under many theories — has a systematic directional bias. It favors interventions that are near-term, measurable, target existing beings, and operate through well-understood causal pathways. It disfavors interventions that are speculative, long-term, target merely possible beings, and depend on contested empirical models.
This is because the theories that assign high value to speculative, long-term interventions (total utilitarianism with low discount rates, for example) are precisely the theories that diverge most from other theories in their recommendations. An intervention can only be “robust” if theories with very different structures agree on it, and theories with very different structures are most likely to agree on cases where the action-relevant features (existing beings, measurable outcomes, clear causal pathways) are the ones all theories care about.
In other words: robustness-seeking is implicitly risk-averse in theory-space, and this risk aversion is not neutral — it systematically favors the neartermist portfolio. This isn’t an argument against neartermism, but it is an argument that “robustness” isn’t the neutral, above-the-fray methodology it’s often presented as. It’s a substantive position that deserves to be argued for on the merits rather than smuggled in as though it were mere prudence. …
So what does the existing literature actually suggest you should do?
Honestly? The literature is in a state of productive but genuine confusion. MEC is the most formally developed approach but founders on normalization. The parliamentary model handles normalization differently but introduces bargaining-mechanism sensitivity. Robustness-seeking avoids both problems but has the hidden directional bias I described. “My Favourite Theory” (just go with whatever theory you assign highest credence) avoids aggregation problems entirely but seems to throw away valuable information from your uncertainty.
If I were being maximally honest about what the current state of the art justifies, I’d say something like this: the right approach is probably a hybrid where you use robustness as a first filter (if an action looks good across all plausible theories, just do it), MEC-style reasoning for decisions where robustness doesn’t settle the question (accepting that normalization introduces some arbitrariness), and a precautionary overlay for irreversible decisions (where the option-value argument from the essay is actually doing real work — preserving future decision-space is one of the few principles that survives across most frameworks).
But I want to flag that this hybrid isn’t a clean resolution — it’s a pragmatic kludge that inherits problems from each component. The field needs either a breakthrough in inter-theoretic value comparison or a persuasive argument that the problem is fundamentally insoluble and we should accept some particular principled approximation. Neither has arrived yet.
A suggested tweak to the landscape metaphor: think about robustness as the set of directions that are uphill on most maps simultaneously, because it makes visually obvious that this set shrinks as you include more diverse maps, and it makes the directional bias visible — robust paths tend to point toward nearby, well-surveyed terrain rather than distant, poorly-mapped peaks.
Thanks Mo! I am no expert on moral uncertainty and how to deal with it, so I’m sure there are much more knowledgeable people than myself to judge. That’s also why I don’t want to imply that robustness is the uniquely correct approach. I do like the metaphor of robustness as “directions that are uphill on most maps” and this is the kind of visualisation I hoped the post could spark. I’d be curious to hear more about how different approaches of dealing with moral uncertainty would “aggregate over maps”.
Very interesting, thanks for writing it :) I had a brief chat with Opus 4.6 about your essay and it pointed out that the “robustness across maps” section is probably the most decision-relevant idea particularly under deep moral uncertainty, but also that the literature on robustness is less useful in practice than one might hope, working through cases in global health / AI safety / insect welfare / x-risk mitigation to illustrate. Opus concludes (mods, let me know if this longform quote is low-value AI slop and I’ll remove it):
Thanks Mo! I am no expert on moral uncertainty and how to deal with it, so I’m sure there are much more knowledgeable people than myself to judge. That’s also why I don’t want to imply that robustness is the uniquely correct approach. I do like the metaphor of robustness as “directions that are uphill on most maps” and this is the kind of visualisation I hoped the post could spark. I’d be curious to hear more about how different approaches of dealing with moral uncertainty would “aggregate over maps”.