Researcher at the Center on Long-Term Risk (part-time). All opinions my own.
Anthony DiGiovanni
Karma: 2,149
What would the justification standards in wild animal welfare say about uncertainty-laden decisions that involve neither AI nor animals: e.g. as a government, deciding which policies to enact, or as a US citizen, deciding who to vote for President?
Yeah, I think this is a feeling that the folks working on bracketing are trying to capture: that in quotidian decision-making contexts, we generally use the factors we aren’t clueless about (@Anthony DiGiovanni—I think I recall a bracketing piece explicitly making a comparison to day-to-day decision making, but now can’t find it… so correct me if I’m wrong!). So I’m interested to see how that progresses.
I think the vast majority of people making decisions about public policy or who to vote for either aren’t ethically impartial, or they’re “spotlighting”, as you put it. I expect the kind of bracketing I’d endorse upon reflection to look pretty different from such decision-making.
That said, maybe you’re thinking of this point I mentioned to you on a call: I think even if someone is purely self-interested (say), they plausibly should be clueless about their actions’ impact on their expected lifetime welfare, because of strange post-AGI scenarios (or possible afterlives, simulation hypotheses, etc.).[1] See this paper. So it seems like the justification for basic prudential decision-making might have to rely on something like bracketing, as far as I can tell. Even if it’s not the formal theory of bracketing given here. (I have a draft about this on the backburner, happy to share if interested.)
- ^
I used to be skeptical of this claim, for the reasons argued in this comment. I like the “impartial goodness is freaking weird” intuition pump for cluelessness given in the comment. But I’ve come around to thinking “time-impartial goodness, even for a single moral patient who might live into the singularity, is freaking weird”.
- ^
Would you say that what dictates my view on (a)vs(b) is my uncertainty between different epistemic principles
It seems pretty implausible to me that there are distinct normative principles that, combined with the principle of non-arbitrariness I mention in the “Problem 1” section, imply (b). Instead I suspect Vasco is reasoning about the implications of epistemic principles (applied to our evidence) in a way I’d find uncompelling even if I endorsed precise Bayesianism. So I think I’d answer “no” to your question. But I don’t understand Vasco’s view well enough to be confident.
Can you explain more why answering “no” makes metanormatively bracketing in consequentialist bracketing (a bit) arbitrary? My thinking is: Let E be epistemic principles that, among other things, require non-arbitrariness. (So, normative views that involve E might provide strong reasons for choice, all else equal.) If it’s sufficiently implausible that E would imply Vasco’s view, then E will still leave us clueless, because of insensitivity to mild sweetening.
But lots of the interventions in 2. seem to also be helpful for getting things to go better for current farmed and wild animals, e.g. because they are aimed avoiding a takeover of society by forces which don’t care at all about morals
Presumably misaligned AIs are much less likely than humans to want to keep factory farming around, no? (I’d agree the case of wild animals is more complicated, if you’re very uncertain or clueless whether their lives are good or bad.)
Thanks Jo! Yeah, the perspective I defend in that post in a nutshell is:
The “reasons” given by different normative views are qualitatively different.
So, when choosing between A and B, we should look at whether each normative view gives us reason to prefer A over B (or B over A).
If consequentialist views say A and B are incomparable, these views don’t give me a reason to prefer A over B (or B over A).
Therefore, if the other normative views in aggregate say A is preferable, I have more reason to choose A.
(Similarly, the decision theory of “bracketing” might also resolve incomparability within consequentialism, but see here for some challenges.)
Re: the first link, what do you think of Dynamic Strong Maximality, which avoids money pumps while allowing for incomparability?
this happens to break at least the craziest Pascalian wagers, assuming plausible imprecise credences (see DiGiovanni 2024).
FWIW, since writing that post, I’ve come to think it’s still pretty dang intuitively strange if taking the Pascalian wager is permissible on consequentialist grounds, even if not obligatory. Which is what maximality implies. I think you need something like bracketing in particular to avoid that conclusion, if you don’t go with (IMO really ad hoc) bounded value functions or small-probability discounting.
(This section of the bracketing post is appropos.)
This particular claim isn’t empirical, it’s about what follows from compelling epistemic principles.
(As for empirical evidence that would change my mind about imprecision being so severe that we’re clueless, see our earlier exchange. I guess we hit a crux there.)
Hi Vasco —
one will still be better than the other in expectation
My posts argue that this is fundamentally the wrong framework. We don’t have precise “expectations”.
Yep, the whole unawareness sequence, really. Post #3 is most directly relevant.
Sounds great, please DM me! Thanks for the invite. :)
In the meantime, if it helps, for the purposes of this discussion I think the essential sections of the posts I linked are:
(The section I linked to from this other post is more of a quick overview of stuff mostly discussed in the sections above. But it might be harder to follow because it’s in the context of a post about unawareness specifically, hence the “UEV” term etc. — sorry about that! You could skip the first paragraph and replace “UEV” with “imprecise EV”.)
And yet even a very flawed procedure will, on average across worlds, do better than chance
I respond to the “better than chance” claim in the post I linked to (in my reply to Richard). What do you think I’m missing there? (See also here.)
Maybe the people who endorse cluelessness are right and our actions can’t make the future reliably better (though not likely). But are you really willing to bet 10^42 expected life years on that proposition? Are you really willing to gamble all that expected value on a speculative philosophical proposition like moral cluelessness?
I’m sorry if I’m a bit of a broken record, but this argument doesn’t engage with the strongest case for cluelessness. See my comment here.
I’d be pretty interested to hear why those who disagree-voted disagree.
For my part, I simply didn’t know the series existed until seeing this post, since this is the only post in the series on EAF. :)
Moreover, I take it that there is very little credibility to the opposite view, that we should regard the inverse of the above claims as disproportionately likely by default. So if you give some (higher-order) credence to views or models implying cluelessness, and some to views on which we can often reasonably expect commonsensically good things to be long-term good, then it seems the positive expectations could trivially win out
I don’t think this works, at least at the level of our empirical credences, for reasons I argue here. (I think the crux here is the “insensitivity to mild sweetening” of imprecise expectations / incomplete preferences; more on that here.)
I do think what you say might work as a response to the precise Bayesian epistemic diffusion model, to be clear, but that’s not the strongest case for cluelessness.
Right, but the same point applies to other scope-restricted views, no? We need some non-arbitrary answer as to why we limit the scope to some set of consequences rather than a larger or smaller set. (I do think bracketing is a relatively promising direction for such a non-arbitrary answer, to be clear.)
Sorry this wasn’t clear! I wasn’t thinking about the choice between fully eliminating factory farming vs. the status quo. I had in mind marginal decreased demand for animal products leading to marginal decreased land use (in expectation), which I do think we have a fairly simple and well-evidenced mechanism for.
I also didn’t mean to say the wild animal effects dominate, just that they’re large enough to be competitive with the farmed animal effects. I agree the tradeoffs between e.g. cow or chicken suffering vs. wild insect suffering seem ambiguous. (And yep, from a non-suffering-focused perspective, it would also plausibly be ambiguous whether increased wild insect populations are bad.)
(I think when I wrote the above comment, I was thinking of pretty coarse-grained buckets of “robustness” vs “speculativeness”.)
I have mixed feelings about this. So, there are basically two reasons why bracketing isn’t orthodox impartial consequentialism:
My choice between A and B isn’t exactly determined by whether I think A is “better” than B. See Jesse’s discussion in this part of the appendix.
Even if we could interpret bracketing as a betterness ranking, the notion of “betterness” here requires assigning a weight of zero to consequences that I don’t think are precisely equally good under A vs. B.
I do think both of these are reasons to give less weight to bracketing in my decision-making than I give to standard non-consequentialist views.[1]
However:
It’s still clearly consequentialist in the sense that, well, we’re making our choice based only on the consequences, and in a scope-sensitive manner. I don’t think standard non-consequentialist views get you the conclusion that you should donate to AMF rather than MAWF, unless they’re defined such that they suffer from cluelessness too.
There’s an impartial reason why we “ignore” the consequences at some locations of value in our decision-making, namely, that those consequences don’t favor one action over the other. (I think the same is true if we don’t use the “locations of value” framework, but instead something more like what Jesse sketches here, though that’s harder to make precise.)
- ^
E.g. compare (i) “A reduces x more units of disutility than B within the maximal bracket-set I’, but I’m clueless about A vs. B when looking outside the maximal bracket-set”, with (ii) “A reduces x more units of disutility than B within I’, and A and B are equally good in expectation when looking outside the maximal bracket-set.” I find (i) to be a somewhat compelling reason to do A, but it doesn’t feel like as overwhelming a moral duty as the kind of reason given by (ii).
Rejecting premise 1, completeness is essentially a nonstarter in the context of morality, where the whole project is premised on figuring out which worlds, actions, beliefs, rules, etc., are better than or equivalent to others. You can deny this your heart of hearts—I won’t say that you literally cannot believe that two things are fundamentally incomparable—but I will say that the world never accommodates your sincerely held belief or conscientious objector petition when it confronts you with the choice to take option A, option B, or perhaps coin flip between them.
This isn’t central to your post, but I’m commenting on it because it’s a very common defense of completeness in EA and I think rejecting completeness has very important implications:
I don’t buy this argument. If you’re forced to choose between A and B, and you pick A, this isn’t enough to show you think A is “better” with respect to some particular normative view v — e.g., some lexical threshold consequentialism. You might have simply picked arbitrarily, or you might have chosen based on some other normative criteria you put some weight on.[1]
And incomparability differs from indifference in that, if you consider A and B incomparable, you might also consider “A + $1” and B incomparable. To me this property seems pretty intuitive in many cases, like this one from Schoenfield (in the context of relative probabilities, not betterness):
You are a confused detective trying to figure out whether Smith or Jones committed the crime. You have an enormous body of evidence that you need to evaluate. Here is some of it: You know that 68 out of the 103 eyewitnesses claim that Smith did it but Jones’ footprints were found at the crime scene. Smith has an alibi, and Jones doesn’t. But Jones has a clear record while Smith has committed crimes in the past. The gun that killed the victim belonged to Smith. But the lie detector, which is accurate 71% percent of the time, suggests that Jones did it. After you have gotten all of this evidence, you have no idea who committed the crime. You are no more confident that Jones committed the crime than that Smith committed the crime, nor are you more confident that Smith committed the crime than that Jones committed the crime.
[paraphrased:] But now you learn, actually 69 eyewitnesses claim Smith did it, not 68. The proposition that Smith did it has been mildly sweetened. So, should you now think that Smith is more likely to have done it than Jones?
I’d recommend specifically checking out here and here, for why we should expect unintended effects (of ambiguous sign) to dominate any intervention’s impact on total cosmos-wide welfare by default. The whole cosmos is very, very weird. (Heck, ASI takeoff on Earth alone seems liable to be very weird.) I think given the arguments I’ve linked, anyone proposing that a particular intervention is an exception to this default should spell out much more clearly why they think that’s the case.
Gotcha, thanks! Yeah, I think it’s fair to be somewhat suspicious of giving special status to “normative views”. I’m still sympathetic to doing so for the reasons I mention in the post (here). But it would be great to dig into this more.