Ya, I agree that many or even most people would get rid of some of their preferences if they could to be happier or more satisfied or whatever. Many people also have fears, anxieties or insecurities they’d rather not have, and those are kinds of “preferences” or “attitudes”, the way I’m using those terms.
Hmm. I’m imagining a monogamous bisexual person who prefers het relationships, but settles for a gay one because they really love their partner and reasonably believe they wouldn’t be able to find a better het relationship if they were back on the market (such that they are not avoiding suffering and also maximising utility by being in this relationship). This person would opt to take the pill that makes them exclusively gay in order to feel more life satisfaction (or even SWB), even though it destroys their preferences.
I assume this person is in your latter bucket of preferring greater life satisfaction per se? If so, I don’t think this situation is as uncommon as you imply—lots of people have hard to satisfy or unsatisfiable preferences that they would rather be rid of in favour of greater life satisfaction; in some sense, this is what it means to be human (Buddhism again).
Life satisfaction is typically considered to be a kind of (or measure of) subjective well-being, and the argument would be the same for that as a special case. Just make the number go up enough after taking the pill, while replacing what they care about. (And I’m using subjective well-being even more broadly than I think normally used.)
For example, I wonder if people who have preferences that are hard to satisfy might actually want to take such a life-satisfaction pill, if it meant their new preferences were easier to satisfy.
In my view, it only makes sense to do if they already have or were going to otherwise have preferences/attitudes that would be more satisfied by taking the pill. If they would suffer less by taking the pill, then it could make sense. If they prefer to have greater life satisfaction per se, then it can make sense to take the pill.
I agree that some instances of replacement seem good, but I suspect the ones I’d agree with are only good in (asymmetric) preference-affecting ways. On the specific cases you mention:
Generational turnover
I’d be inclined against it unless
it’s actually on the whole preferred (e.g. aggregating attitudes) by the people being replaced, or
the future generations would have lesser regrets or negative attitudes towards aspects of their own lives or suffering (per year, say). Pummer (2024) resolves some non-identity cases this way, while avoiding antinatalism (although I am fairly sympathetic to antinatalism).
not blindly marrying the first person you fall in love with
people typically (almost always?) care or will care about their own well-being per se in some way, and blindly marrying the first person you fall in love with is risky for that
more generally, a bad marriage can be counterproductive for most of what you care or will care to achieve
future negative attitudes (e.g. suffering) from the marriage or for things to be different can count against it
helping children to develop new interests:
they do or will care about their well-being per se, and developing interests benefits that
developing interests can have instrumental value for other attitudes they hold or are likely to eventually hold either way, e.g. having common interests with others, making friends, not being bored
developing new interests is often (usually? almost always?) a case of discovering dispositional attitudes they already have or would have had anyway. For example, there’s already a fact of the matter, based in a child’s brain as it already is or will be either way, whether they would enjoy certain aspects of some activity.[1] So, we can just count unknown dispositional attitudes on preference-affecting views. I’m sympathetic to counting dispositional attitudes anyway for various reasons, and whether or not they’re known doesn’t seem very morally significant in itself.
- ^
Plus, the things that get reinforced, and so may shift some of their attitudes, typically get reinforced because of such dispositional attitudes: we come to desire the things we’re already disposed to enjoy, with the experienced pleasure reinforcing our desires.
I am a bit unenlightened when it comes to moral philosophy so I would appreciate if you can help me understand this viewpoint better. Does it change if you replace ‘subjective well-being’ with ‘life satisfaction’ (in the sense of SWB being experiential and satisfaction being reflective/prospective)? i.e. are there conceptions of ‘life satisfaction’ that sort of take into account what this person wants for themselves?
For example, I wonder if people who have preferences that are hard to satisfy might actually want to take such a life-satisfaction pill, if it meant their new preferences were easier to satisfy. (Is this, in some sense, what a lot of Buddhist reframing around desire is doing?)
Good point about the degree of identity loss.
I think the hybrid view you discuss is in fact compatible with some versions of actualism (e.g. weak actualism), as entirely preference-affecting views (although maybe not exactly in the informal way I describe them in this post), so not necessarily hybrid in the way I meant it here.
Take the two outcomes of your example, assuming everyone would be well-off as long as they live, and Bob would rather continue to live than be replaced:
Bob continues to live.
Bob dies and Sally is born.
From the aggregated preferences or attitudes of the people in 1, 1 is best. From the aggregated preferences or attitudes of the people in 2, 2 is best. So each outcome is best for the (would-be) actual people in it. So, not all preference-affecting views even count against this kind of replaceability.
My next two pieces will mostly deal with actualist(-ish) views, because I think they’re best at taking on the attitudes that matter and treating them the right way, or being radically empathetic.
I like the hybrid approach, and discuss its implications for replaceability a bit here. (Shifting to the intrapersonal case: those of us who reject preference theories of well-being may still recognize reasons not to manipulate preferences, for example based on personal identity: the more you manipulate my values, the less the future person is me. To be a prudential benefit, then, the welfare gain has to outweigh the degree of identity loss. Moreover, it’s plausible that extrinsic manipulations are typically more disruptive to one’s degree of psychological continuity than voluntary or otherwise “natural” character development.)
It seems worth flagging that some instances of replacement seem clearly good! Possible examples include:
Generational turnover
not blindly marrying the first person you fall in love with
helping children to develop new interests
I guess even preference-affecting views will support instrumental replacement, i.e. where the new desire results in one’s other desires being sufficiently better satisfied (even before counting any non-instrumental value from the new desire itself) to outweigh whatever was lost.
I previously thought Mark Fuentes was someone ~ unaffiliated with this community. The article seemed to present enough evidence that I no longer believe this. (It also made me downwards somewhat on the claims in the Fuentes post, but not enough to get to pre-reading-the-post levels).
I don’t know the exact dates, but: a)proof-based methods seem to be receiving a lot of attention b) def/acc is becoming more of a thing c) more focus on concentration of power risk (tbh, while there are real risks here, I suspect most work here is net-negative)
Ah, I wasn’t clear. To bet that AI will not kill us all by the end of 2027.
I don’t think that makes sense, given the world-complexity “AI” would need to learn and evolve and get tinkered to be able to navigate. I’ve had some conversations with Greg about this.
Change your mind in what way? Could you elaborate a bit?
Could you quote which line you mean? Then I can mention where you can find it back
That’s an odd prior. I can see a case for a prior that gets you to <10^-6, maybe even 10^-9, but how can you get to substantially below 10^-9 annual with just historical data???
Sapiens hasn’t been around for that long for longer than a million years! (and conflict with homo sapiens or other human subtypes still seems like a plausible reason for extinction of other human subtypes to me). There have only been maybe 4 billion species total in all of geological history! Even if you have almost certainty that literally no species has ever died of conflict, you still can’t get a prior much lower than 1⁄4,000,000,000! (10^-9).
IDK if this actually works since I only just signed up, but, the “Join us” button in top right leads to, “https://sentinel-team.org/contact/″
Seems you can add yourself to mailing list from there.
I think this would significantly depend on what the investigation ultimately showed. It would probably be hard for the average EA reader (much less a member of the general public) to reliably estimate how much personal stress, risk, cost, etc. a cooperator bore, and thus how much respect we should assign for their choice. I think many people would use the outcome as a rough proxy. If the investigation revealed only fairly-well-known structural problems plus bad judgment by a few individuals, then people may not appreciate how much of a burden it was to work with a thorough, broad-scope investigation that went down many paths that ultimately ended up being unfruitful.
The full letter is available here — was recently posted online as part of this tweet thread.
Since we normally let humans accumulate wealth and become powerful via lawful means, I think we should allow these humanoid robots to do the same. I hope you would agree with me here.
I agree with this—and also agree with it for various non-humanoid AI systems.
However, I see this as less about rights for systems that may at some point exist, and more about our responsibilities as the creators of those systems.
Not entirely analogous, but: suppose we had a large creche of babies whom we had been told by an oracle would be extremely influential in the world. I think it would be appropriate for us to care more than normal about their upbringing (especially if for the sake of the example we assume that upbringing can meaningfully affect character).
The second and third possible motivations seem to have a Prisoner’s Dilemma element to them. They would motivate people to talk if and only if similarly situated individuals were talking. The inability to timely determine whether others have defected from the best-for-prisoners-collectively state is pretty important to the Dilemma.
Even worse, if other prisoners strongly oppose cooperation, they may find a way to collectively punish those who do defect. The original Dilemma only gives the jailers the ability to assign punishment based on defection/non-defection. None of that is meant to suggest that EA insiders would necessarily punish cooperators—I have no way of knowing that. But I expect most people would consider the possibility of who might be displeased with their cooperation.
I think it depends what sort of risks we are talking about.
Agree—I don’t think the fatalistic view applies to all Dustin-related risks, just enough to make him a suboptimal comparison here.
To take an FTX-like situation as an example, I doubt many orgs could avoid bankruptcy if they had liability for 4-6 years’ clawback of prior OP grants, and it’s not clear that getting months to years’ worth of advance notice and attempted mitigation would materially reduce the odds of bankruptcy. (As you note, this is extraordinarily unlikely!)
Encouraging more people to EtG would be mitigation for the movement as a whole, but its effectiveness would be dependent on [1] the catastrophic fraud actually existing, [2] you having enough reason to believe that to recommend action to other EAs but not enough to go to the media and/or cops and get traction,[1] [3] you persuading the would-be EtGers that circumstances warranted them choosing this path, and [4] your advocacy not indirectly causing prompt public discovery and collapse of the fraud. After all, the value would be knowing of the risk in advance to take mitigating action sufficiently in advance of public discovery. Understanding the true risk a few weeks to months in advance of everyone else isn’t likely to help much at all. Those seem like difficult conditions to meet.
- ^
Reporting, but not getting traction from external watchdogs, is possible (cf. Madoff). I have not thought through whether having enough reason to advise other EAs, but not enough to report externally, is possible.
- ^
@Lukeprog posted this few decades ago “neglected rationalist virtue of scholarship”
I think it’s mostly just cognitive science which is daniel kanehman and others(which is well known), good bunch of linguistics (which I have heard are well known), and anti-philosophy (because we dislike philosophy as it is done), rest is just ethics and objective bayesianism, with a quinean twist.
I think there is a difference between epistemic status and confidence level, I could be overtly confident and still buy the lottery ticket while knowing it won’t work. I think there is a difference between social and epistemic confidence, so it’s better to specify.