Econ PhD student at Oxford, about to start a postdoc at the Digital Economy Lab at Stanford, and research affiliate at the Global Priorities Institute. I’m slightly less ignorant about economic theory than about everything else.
trammell
Karma: 1,767
Glad to see you found my post thought-provoking, but let me emphasize that my own understanding is also partial at best, to put it mildly!
Ah wait, did your first comment always say “similar”? No worries if not (I often edit stuff just after posting!) but if so, I must have missed it—apologies for just pointing out that they were different points and not addressing whether they are sufficiently similar.
But they do seem like significantly different hypotheses to me. The reason is that it seems like the arguments presented against many experiences in a single brain can convince me that there is probably (something like) a single, highly “integrative” field of hedonic intensities, just as I don’t doubt that there is a single visual processing system behind my single visual field, and yet leave me fully convinced that both fields can come in different sizes, so that one brain can have higher welfare capacity than another for size reasons.
Thanks for the second comment though! It’s interesting, and to my mind more directly relevant, in that it offers reasons to doubt the idea that hedonic intensities are spread across locations at all. They move me a bit, but I’m still mostly left thinking
- Re 1, we don’t need to appeal to scientific evidence about whether it’s possible to have different amounts of, say, pain in different parts of the phenomenal field. It happens all the time that we feel pain in one hand but not the other. If that’s somehow an illusion, it’s the illusion that needs a lot of scientific evidence to debunk.
- Re 2, it’s not clear why we would have evolved to create valence (or experience) at all in the first place, so in some sense the fact that it would evidently be more efficient to have less of it doesn’t help here. But assuming that valence evolved to motivate us in adaptive ways, it doesn’t seem like such a stretch to me to say that forming the feeling “my hand is on fire and it in particular hurts” shapes our motivations in the right direction more effectively than forming the feeling “my hand is on fire and I’ve just started feeling bad overall for some reason”, and that this is worth whatever costs come with producing a field of valences.
- Re 3, the proposal I call (iii*) and try to defend is “that the welfare of a whole experience is, or at least monotonically incorporates, some monotonic aggregation of the hedonic intensities felt in these different parts of the phenomenal field” (emphasis added). I put in the “incorporates” because I don’t mean to take a stand on whether there are also things that contribute to welfare that don’t correspond to particular locations in the phenomenal field, like perhaps the social pains you mention. I just find it hard to deny from first-hand experience that there are some “location-dependent” pains; and if so, I would think that these can scale with “size”.
Thanks! That’s a really interesting thought. I hadn’t thought of that possibility—I’ve been working on the assumption that they’re not reducible—but now that you mention it, I don’t have very strong intuitions about whether it seems more or less likely than there being two dimensions “at bottom”.
One intuition against is that it seems a bit weirdly discrete to suppose that a “hedonic atom” can just be +1, 0, or −1. But I guess there’s some discreteness at bottom with literal atoms (or perhaps a better analogy would be electrical charge) as well...
Thanks for sharing this. (Thank you very much as well for letting me start exploring a tricky idea like this without assuming this is all just an excuse for discriminating against those with disabilities!) I definitely agree that a risk of trying to account for differences in “experience size”, even if the consideration is warranted, is that it could lead us to quickly dismiss experiences different from our own as smaller even if they aren’t.
I am no expert on deafness or most of the other topics relevant here, but my understanding is that often, if someone loses a sensory faculty or body part but doesn’t suffer damage to the relevant part of the brain, the brain in some sense rewires to give more attention (i.e., I would guess, more hedonic intensity and/or more “size”) to the remaining sensory faculties. This is why, when bringing up the case of an amputee, I only consider the case of someone whose brain has not had time to make this adjustment. I think it could totally be the case that deaf people, at least post-adjustment (or throughout life, if they have been deaf from birth), have such richer experiences on other dimensions that their welfare capacities tend to be greater overall than non-deaf people.
I cited your post (at the end of the 2nd paragraph of “How these implications are revisionary”) as an exploration of a different idea from mine, namely that one brain might have more moral weight than another because it contains more experiences at once. Your excerpt seems to highlight this different idea.
Are you saying your post should be read as also exploring the idea that one brain might have more moral weight than another even if they each contain one experience, because one experience is larger than the other? If so, can you point me to the relevant bit?
Ok great!
And ok, I agree that the answer to the first question is probably “yes”, so maybe what I was calling an alternative anthropic principle in my original comment could be framed as SSA with this directly time-centric reference class. If so, instead of saying “that’s not SSA”, I should have said “that’s not SSA with a standard reference class (or a reference class anyone seems to have argued for)”. I agree that Bostrom et al. (2010) don’t seem to argue for such a reference class.
On my reading (and Teru’s, not coincidentally), the core insight Bostrom et al. have (and iterate on) is equivalent to the insight that if you haven’t observed something before, and you assign it a probability per unit of time equal to its past frequency, then you must be underestimating its probability per unit of time. The response isn’t that this is predicated on, or arguing for, any weird view on anthropics, but just that it has nothing to do with anthropics: it’s true, but for the same reason that you’ll underestimate the probability of rain per unit time based on past frequency if it’s never rained (though in the prose they convey their impression that the fact that you wouldn’t exist in the event of a catastrophe is what’s driving the insight). The right thing to do in both cases is to have a prior and update the probability downward as the dry spell lengthens. A nonstandard anthropic principle (or reference class) is just what would be necessary to motivate a fundamental difference from “no rain”.
Good Ventures rather than Effective Ventures, no?
The title of this post might give the impression that Rory Stewart was a founder of GiveDirectly. To clarify, GiveDirectly was founded by other people in 2008; Stewart became its president in 2022.
Interesting, thanks for pointing this out! And just to note, that result doesn’t rely on any sort of suspicious knowledge about whether you’re on the planet labeled “x” or “y”; one could also just say “given that you observe that you’re in period 2, …”.
I don’t think it’s right to describe what’s going on here as anthropic shadow though, for the following reason. Let me know what you think.
To make the math easier, let me do what perhaps I should have done from the beginning and have A be the event that the risk is 50% and B be the event that it’s 0%. So in the one-planet case, there are 3 possible worlds:
A1 (prior probability 25%) -- risk is 50%, lasts one period
A2 (prior probability 25%) -- risk is 50%, lasts two periods
B (prior probability 50%) -- risk is 0%, lasts two periods
At time 1, whereas SIA tells us to put credence of 1⁄2 on A, SSA tells us to put something higher--
(0.25 + 0.25/2) / (0.25 + 0.25/2 + 0.5/2) = 3⁄5
--because a higher fraction of expected observers are at period 1 given A than given B. This is the Doomsday Argument. When we reach period 2, both SSA and SIA then tell us to update our credence in A downward. Both principles tell us fully to update downward for the same reasons that we would update downward on the probability of an event that didn’t change the number of observers: e.g. if A is the event you live in a place where the probability of rain per day is 50% and B is the event that it’s 0%; you start out putting credence 50% [or 60%] on A; and you make it to day 2 without rain (and would live to see day 2 either way). But in the catastrophe case SSA further has you update downward because the Doomsday Argument stops applying in period 2.
One way to put the general lesson is that, as time goes on and you learn how many observers there are, SSA has less room to shift probability mass (relative to SIA) toward the worlds where there are fewer observers.
In the case above, once you make it to period 2, that uncertainty is fully resolved: given A or B, you know you’re in a world with 2 observers. This is enough to motivate such a big update according to SSA that at the end the two principles agree on assigning probability 1⁄3 to A.
In cases where uncertainty about the number of observers is only partially resolved in the move from period 1 to period 2--as in my 3-period example, or in your 2-planet example*--then the principles sustain some disagreement in period 2. This is because
SSA started out in period 1 assigning a higher credence to A than SIA;
both recommend updating on the evidence given by survival as you would update on anything else, like lack of rain;
SSA further updates downward because the Doomsday Argument partially loses force; and
the result is that SSA still assigns a higher credence to A than SIA.
*To verify the Doomsday-driven disagreement in period 1 in the two-planet case explicitly (with the simpler definitions of A and B), there are 5 possible worlds:
A1 (prior probability 12.5%) -- risk is 50% per planet, both last one period
A2 (prior probability 12.5%) -- risk is 50% per planet, only x lasts two periods
A3 (prior probability 12.5%) -- risk is 50% per planet, only y lasts two periods
A4 (prior probability 12.5%) -- risk is 50% per planet, both last two periods
B (prior probability 50%) -- risk is 0% per planet, both last two periods
In period 1, SIA gives credence in A of 1⁄2; SSA gives (0.125 + 0.125*2/3 + 0.125*2/3 + 0.125/2) / (0.125 + 0.125*2/3 + 0.125*2/3 + 0.125/2 + 0.5/2) = 5⁄8.
One could use the term “anthropic shadow” to refer to the following fact: As time goes on, in addition to inferring existential risks are unlikely as we would infer that rain is unlikely, SSA further recommends inferring that existential risks are unlikely by giving up the claim that we’re more likely to be in a world with fewer observers; but this second update is attenuated by the (possible) existence of other planets. I don’t have any objection to using the term that way and I do think it’s an interesting point. But I think the old arguments cited in defense of an “anthropic shadow” effect were pretty clearly arguing for the view that we should update less (or even not at all) toward thinking existential risk per unit time is low as time goes on than we would update about the probabilities per unit time of other non-observed events.
Eli Rose was the one who linked to it, to give credit where it’s due : )
I agree that those are different claims, but I expect the weaker claim is also not true, for whatever that’s worth. The claim in Toby Crisford’s Possible Solution 2, as I understand it, is the same as the claim I was making at the end of my long comment: that one could construct some anthropic principle according to which the anthropic shadow argument would be justified. But that principle would have to be different from SSA and SIA; I’m pretty sure it would have to be something which no one has argued for; and my guess is that on thinking about it further most people would consider any principle that fits the bill to have significantly weirder implications than either SSA or SIA.
Ok great! And sorry the numbers in my example got unwieldy, I just picked some probabilities at the beginning and ran with them, instead of bothering to reverse-engineer something cleaner…
I’m not sure I understand the second question. I would have thought both updates are in the same direction: the fact that we’ve survived on Earth a long time tells us that this is a planet hospitable to life, both in terms of its life-friendly atmosphere/etc and in terms of the rarity of supervolcanoes.
We can say, on anthropic grounds, that it would be confused to think other planets are hospitable on the basis of Earth’s long and growing track record. But as time goes on, we get more evidence that we really are on a life-friendly planet, and haven’t just had a long string of luck on a life-hostile planet.
The anthropic shadow argument was an argument along the lines, “no, we shouldn’t get ever more convinced we’re on a life-friendly planet over time (just on the evidence that we’re still around). It is actually plausible that we’ve just had a lucky streak that’s about to break—and this lack of update is in some way because no one is around to observe anything in the worlds that blow up”.
To answer the first question, no, the argument doesn’t rely on SIA. Let me know if the following is helpful.
Suppose your prior (perhaps after studying plate tectonics and so on, but not after considering the length of time that’s passed without an an extinction-inducing supervolcano) is that there’s probability “P(A)”=0.5 that risk of an extinction-inducing supervolcano at the end of each year is 1⁄2 and probability “P(B)”=0.5 that the risk is 1⁄10. Suppose that the world lasts at least 1 year and most 3 years regardless.
Let “A1” be the possible world in which the risk was 1⁄2 per year and we blew up at the end of year 1, “A2” be that in which the risk was 1⁄2 per year and we blew up at the end of year 2, and “A3” be that in which the risk was 1⁄2 per year and we never blew up, so that we got to exist for 3 years. Define B1, B2, B3 likewise for the risk=1/10 worlds.
Suppose there’s one “observer per year” before the extinction event and zero after, and let “Cnk”, with k<=n, be observer #k in world Cn (C can be A or B). So there are 12 possible observers: A11, A21, A22, A31, A32, A33, and likewise for the Bs.
If you are observer Cnk, your evidence is that you are observer #k. The question is what Pr(A|k) is; what probability you should assign to the annual risk being 1⁄2 given your evidence.
Any Bayesian, whether following SIA or SSA (or anything else), agrees that
Pr(A|k) = Pr(k|A)Pr(A)/Pr(k),
where Pr(.) is the credence an observer should have for an event according to a given anthropic principle. The anthropic principles disagree about the values of these credences, but here the disagreements cancel out. Note that we do not necessarily have Pr(A)=P(A): in particular, if the prior P(.) assigns equal probability to two worlds, SIA will recommend assigning higher credence Pr(.) to the one with more observers, e.g. by giving an answer of Pr(coin landed heads) = 1⁄3 in the sleeping beauty problem, where on this notation P(coin landed heads) = 1⁄2.
On SSA, your place among the observers is in effect generated first by randomizing among the worlds according to your prior and then by randomizing among the observers in the chosen world. So Pr(A)=0.5, and
Pr(1|A) = 1⁄2 + 1/4*1/2 + 1/4*1/3 = 17⁄24
(since Pr(n=1|A)=1/2, in which case k=1 for sure; Pr(n=2|A)=1/4, in which case k=1 with probability 1⁄2; and Pr(n=3|A)=1/4, in which case k=1 with probability 1⁄3);
Pr(2|A) = 1/4*1/2 + 1/4*1/3 = 5⁄24; and
Pr(3|A) = 1/4*1/3 = 2⁄24.
For simplicity we can focus on the k=2 case, since that’s the case analogous to people like us, in the middle of an extended history. Going through the same calculation for the B worlds gives Pr(2|B) = 63⁄200, so Pr(2) = 0.5*5/24 + 0.5*63/200 = 157⁄600.
So Pr(A|2) = 125⁄314 ≈ 0.4.
On SIA, your place among the observers is generated by randomizing among the observers, giving proportionally more weight to observers in worlds with proportionally higher prior probability, so that the probability of being observer Cnk is
1/12*Pr(Cn) / [sum over possible observers, labeled “Dmj”, of (1/12*Pr(Dm))].
This works out to Pr(2|A) = 2⁄7 [6 possible observers given A, but the one in the n=1 world “counts for double” since that world is twice as likely than the n=2 or =3 worlds a priori];
Pr(A) = 175⁄446 [less than 1⁄2 since there are fewer observers in expectation when the risk of early extinction is higher], and
Pr(2) = 140⁄446, so
Pr(A|2) = 5⁄14 ≈ 0.36.
So in both cases you update on the fact that a supervolcano did not occur at the end of year 1, from assigning probability 0.5 to the event that the underlying risk is 1⁄2 to assigning some lower probability to this event.
But I said that the disagreements canceled out, and here it seems that they don’t cancel out! This is because the anthropic principles disagree about Pr(A|2) for a reason other than the evidence provided by the lack of a supervolcano at the end of year 1: namely the possible existence of year 3. How to update on the fact that you’re in year 2 when you “could have been” in year 3 gets into doomsday argument issues, which the principles do disagree on. I included year 3 in the example because I worried it might seem fishy to make the example all about a 2-period setting where, in period 2, the question is just “what was the underlying probability we would make it here”, with no bearing on what probability we should assign to making it to the next period. But since this is really the example that isolates the anthropic shadow consideration, observe that if we simplify things so that the world lasts at most 2 years (and there 6 possible observers), SSA gives
Pr(2|A) = 1⁄4, Pr(A) = 1⁄2, Pr(2) = 4⁄5 → Pr(A|2) = 5⁄14.
and SIA gives
Pr(2|A) = 1⁄3, Pr(A) = 15⁄34, Pr(2) = 14⁄34 → Pr(A|2) = 5⁄14.
____________________________
An anthropic principle that would assign a different value to Pr(A|2)--for the extreme case of sustaining the “anthropic shadow”, a principle that would assign Pr(A|2)=Pr(A)=1/2--would be one in which your place among the observers is generated by
first randomizing among times k (say, assigning k=1 and k=2 equal probability);
then over worlds with an observer alive at k, maintaining your prior of Pr(A)=1/2;
[and then perhaps over observers at that time, but in this example there is only one].
This is more in the spirit of SSA than SIA, but it is not SSA, and I don’t think anyone endorses it. SSA randomizes over worlds and then over observers within each world, so that observing that you’re late in time is indeed evidence that “most worlds last late”.
I also found this remarkably clear and definitive—a real update for me, to the point of coming with some actual relief! I’m afraid I wasn’t aware of the existing posts by Toby Crisford and Jessica Taylor.
I suppose if there’s a sociological fact here it’s that EAs and people who are nerdy in similar sorts of ways, myself absolutely included, can be quick to assume a position is true because it sounds reasonable and seemingly thoughtful other people who have thought about the question more have endorsed it. I don’t think this single-handedly demonstrates we’re too quick; not everyone can dig into everything, so at least to some extent it makes sense to specialize and defer despite the fact this is bound to happen now and then.
Of course argument-checking is also something one can specialize in, and one thing about the EA community which I think is uncommon and great is hiring people like Teru to dig into its cultural background assumptions like this...
To my mind, the first point applies to whatever resources are used throughout the future, whether it’s just the earth or some larger part of the universe.
I agree that the number/importance of welfare subjects in the future is a crucial consideration for how much to do longtermist as opposed to other work. But when comparing longtermist interventions—say, splitting a budget between lowering the risk of the world ending and proportionally increasing the fraction of resources devoted to creating happy artificial minds—it would seem to me that the “size of the future” typically multiplies the value of both interventions equally, and so doesn’t matter.
Ok—at Toby’s encouragement, here are my thoughts:
This is a very old point, but to my mind, at least from a utilitarian perspective, the main reason it’s worth working on promoting AI welfare is the risk of foregone upside. I.e. without actively studying what constitutes AI welfare and advocating for producing it, we seem likely to have a future that’s very comfortable for ourselves and our descendants—fully automated luxury space communism, if you like—but which contains a very small proportion of the value that could have been created by creating lots of happy artificial minds. So concern for creating AI welfare seems likely to be the most important way in which utilitarian and human-common-sense moral recommendations differ.
It seems to me that the amount of value we could create if we really optimized for total AI welfare is probably greater than the amount of disvalue we’ll create if we just use AI tools and allow for suffering machines by accident, since in the latter case the suffering would be a byproduct, not something anyone optimizes for.
But AI welfare work (especially if this includes moral advocacy) just for the sake of avoiding this downside also seems valuable enough to be worth a lot of effort on its own, even if suffering AI tools are a long way off. The animal analogy seems relevant: it’s hard to replace factory farming once people have started eating a lot of meat, but in India, where Hinduism has discouraged meat consumption for a long time, less meat is consumed and so factory farming is evidently less widespread.
So in combination, I expect AI welfare work of some kind or another is probably very important. I have almost no idea what the best interventions would be or how cost-effective they would be, so I have no opinion on exactly how much work should go into them. I expect no one really knows at this point. But at face value the topic seems important enough to warrant at least doing exploratory work until we have a better sense of what can be done and how cost-effective it could be, only stopping in the (I think unlikely) event that we can say with some confidence that the best AI welfare work to be done is worse than the best work that can be done in other areas.
The point that it’s better to save people with better lives than people with worse lives, all else equal, does make sense (at least from a utilitarian perspective). So you’re right that [$ / lives saved] is not a perfect approach. I do think it’s worth acknowledging this...!
But the right correction isn’t to use VSLs. The way I’d put it is: a person’s VSL—assuming it’s been ideally calculated for each individual, putting aside issues about how governments estimate it in practice—is how many dollars they value as much as slightly lowering their chance of death. So the fact that VSLs differ across people mixes together two things: a rich person might have a higher VSL than a poor person (1) because the rich person values their life more, or (2) because the rich person values a dollar less. The first thing is right to correct for (from a utilitarian perspective), but as other commenters have noted, the second isn’t.
My guess is that the second factor baked into the VSL is bigger in most real-world comparisons we might want to make, so that it’s less of a mistake to just try to maximize [$ / lives saved] than to try to maximize [$ / (lives saved * VSL)].
Ah I see, sorry. Agreed
I don’t follow—are you saying that (i) AI safety efforts so far have obviously not actually accomplished much risk-reduction, (ii) that this is largely for risk compensation reasons, and (iii) that this is worth emphasizing in order to prevent us from carrying on the same mistakes?
If so, I agree that if (i)-(ii) are true then (iii) seems right, but I’m not sure about (i) and (ii). But if you’re just saying that it would be good to know whether (i)-(ii) are true because if they are then it would be good to do (iii), I agree.
Thanks for noting this possibility—I think it’s the same, or at least very similar, to an intuition Luisa Rodriguez had when we were chatting about this the other day actually. To paraphrase the idea there, even if we have a phenomenal field that’s analogous to our field of vision and one being’s can be bigger than another’s, attention may be sort of like a spotlight that is smaller than the field. Inflicting pains on parts of the body lower welfare up to a point, like adding red dots to a wall in our field of vision with a spotlight on it adds redness to our field of vision, but once the area under the spotlight is full, not much (perhaps not any) more redness is perceived by adding red dots to the shadowy wall outside the spotlight. If in the human case the spotlight is smaller than “the whole body except for one arm”, then it is about equally bad to put the amputee and the non-amputee in an ice bath, or for that matter to put all but one arm of a non-amputee and the whole of a non-amputee in an ice bath.
Something like this seems like a reasonable possibility to me as well. It still doesn’t seem as intuitive to me as the idea that, to continue the metaphor, the spotlight lights the whole field of vision to some extent, even if some parts are brighter than others at any given moment; if all of me except one arm were in an ice bath, I don’t think I’d be close to indifferent about putting the last arm in. But it does seem hard to be sure about these things.
Even if “scope of attention” is the thing that really matters in the way I’m proposing “size” does, though, I think most of what I’m suggesting in this post can be maintained, since presumably “scope” can’t be bigger than “size”, and both can in principle vary across species. And as for how either of those variables scales with neuron count, I get that there are intuitions in both directions, but I think the intuitions I put down on the side of superlinearity apply similarly to “scope”.