Researcher at the Center on Long-Term Risk. All opinions my own.
Anthony DiGiovanni
Karma: 1,519
Thanks Magnus — I’m not sure I understand your objection or which specific language choices in the posts seem too strong, yet.
nevertheless holding something weaker, like that some heuristics are slightly better than others in expectation, or that we in general can do meaningfully (or even just marginally) better than nothing
Can you say a bit more about why you think I haven’t adequately responded to this perspective, in the sequence? (“Degrees of imprecision”, “The ‘better than chance’ argument”, and my response to “Meta-extrapolation” explain why I don’t think “X is slightly better than Y in expectation” makes sense here.)
4. Why existing approaches to cause prioritization are not robust to unawareness
3. Why impartial altruists should suspend judgment under unawareness
2. Why intuitive comparisons of large-scale impact are unjustified
1. The challenge of unawareness for impartial altruist action guidance: Introduction
Thanks! I unfortunately don’t have time to engage fully with this thread going forward, but briefly:
To be clear, I don’t share Karnofsky’s overall framework. I’m skeptical of the “regression to normality” criterion myself. (And I don’t find his model of the problem behind Pascal’s mugging probabilities compelling, since he still uses precise estimates.)
In the Pascal’s mugging case, I think people have some fuzzy sense that the mugger’s claim is made-up, which can be more carefully operationalized with imprecise credences. But if we can’t even point to what our “this is absurd” reaction is about, and are instead merely asserting that our pretheoretic sense should dictate our decisions, I’m more skeptical. Especially when we’re embracing an ethical principle most people would consider absurd (impartial altruism).
FWIW, while I didn’t downvote the comment, I can see how folks would consider “Why stop at X?” a lazy “gotcha” argument or appeal to absurdity heuristic, which seems worth discouraging.
I wrote an article recently about insects.
FYI, I assume the link here doesn’t go to the post you intended.
I partly had in mind personal communications, but some public examples (and very brief summaries of my reactions, not fleshed out counterarguments):
In “Sequence thinking vs. cluster thinking”, Holden says, “For example, obeying common-sense morality (“ends don’t justify the means”) heuristics seems often to lead to unexpected good outcomes, and contradicting such morality seems often to lead to unexpected bad outcomes.”
I guess the argument is supposed to be that we have empirical evidence of heuristics working well in this sense. But on its face, this just pushes the question back to why we should expect “how well a strategy works under unknown unknowns” to generalize so cleanly from local scales to longtermist scales. (Related discussion here.)
“Heuristics for clueless agents” claims that “heuristics produce effective decisions without demanding too much of ordinary decision-makers.”
Their arguments seem to be some combination of “in some decision situations, it’s pretheoretically clear which decision procedures are more or less ‘effective’” (Sec. 5) and “heuristics have theoretical justification based on the bias-variance tradeoff” (Sec. 7). But pretheoretic judgments about effectiveness from a longtermist perspective seem extremely unreliable, and appeals to bias-variance tradeoffs are irrelevant when the problem (under UUs) is model misspecification.
Why expect “heuristics” to be robust to unknown unknowns?
I often read/hear claims that, if we’re worried that our evaluations of interventions won’t hold up under unknown unknowns, we should follow (simple) heuristics. But what precisely is the argument for this? This isn’t a rhetorical question — I’m just noting my confusion and want to understand this view better.
Interested to hear more from those who endorse this view!
I worry we’re going to continue to talk past each other. So I don’t plan to engage further. But for other readers’ sake:
I definitely don’t treat broad imprecision as “a privileged default”. In the post I explain the motivation for having more or less severely imprecise credences in different hypotheses. The heart of it is that adding more precision, beyond what the evidence and plausible foundational principles merit, seems arbitrary. And you haven’t explained why your bottom-line intuition — about which decisions are good w.r.t. a moral standard as extremely far-reaching as impartial beneficence[1] — would constitute evidence or a plausible foundational principle. (To me this seems pretty clearly different from the kind of intuition that would justify rejecting radical skepticism.)
I don’t see how this engages with the arguments I cited, or the cited post more generally. Why do you think it’s plausible to form a (non-arbitrary) determinate judgment about these matters? Why think these determinate judgments are our “best” judgment, when we could instead have imprecise credences that don’t narrow things down beyond what we have reason to?
I don’t think this response engages with the argument that judgment calls about our impact on net welfare over the whole cosmos are extraordinary claims, so they should be held to a high epistemic standard. What do you think of my points on this here and in this thread?
I think this is the most honest answer, from an impartial altruistic perspective.
I’ve got a big moral circle (all sentient beings and their descendants), but it does not extend to aliens because of cluelessness.
...
I’m quite confident that if we’re thinking about the moral utility of spacefaring civilisation, we should at least limit our scope to our own civilisation
I agree that the particular guesses we make about aliens will be very speculative/arbitrary. But “we shouldn’t take the action recommended by our precise ‘best guess’ about XYZ” does not imply “we can set the expected contribution of XYZ to the value of our interventions to 0″. I think if you buy cluelessness — in particular, the indeterminate beliefs framing on cluelessness — the lesson you should take from Maxime’s post is that we simply aren’t justified in saying any intervention with effects on x-risk is net-positive or net-negative (w.r.t. total welfare of sentient beings).
This is linked to my discussion with Jim about determinate credences (since I didn’t initially understand this concept well, ChatGPT gave me a useful explanation).
FYI, I don’t think ChatGPT’s answer here is accurate. I’d recommend this post if you’re interested in (in)determinate credences.
Should you go with your best guess?: Against precise Bayesianism and related views
To be clear, “preferential gap” in the linked article just means incomplete preferences. The property in question is insensitivity to mild sweetening.
If one was exactly indifferent between 2 outcomes, I believe any improvement/worsening of one of them must make one prefer one of the outcomes over the other
But that’s exactly the point — incompleteness is not equivalent to indifference, because when you have an incomplete preference between 2 outcomes it’s not the case that a mild improvement/worsening makes you have a strict preference. I don’t understand what you think doesn’t “make sense in principle” about insensitivity to mild sweetening.
I fully endorse expectational total hedonistic utilitarianism (ETHU) in principle
As in you’re 100% certain, and wouldn’t put weight on other considerations even as a tiebreaker? That seems extreme. (If, say, you became convinced all your options were incomparable from an ETHU perspective because of cluelessness, you would presumably still all-things-considered-prefer not to do something that injures yourself for no reason.)
Sorry this wasn’t clear — my arguments throughout this sequence don’t just apply to perfect impartiality. I use “impartiality” loosely, in the sense in the first sentence of the intro: “gives moral weight to all consequences, no matter how distant”. (I couldn’t think of a better, non-clunky term for this.) See also footnote 4 of the first post:
I’m happy to grant that normative neartermism is immune to my arguments. When I said “our reason to work on EA causes”, I meant “the reason to work on such causes that my target audience actually endorses.”