suffering-focused-altruist, autistic/traumatized, superintelligence altruist-alignment focused
quila
Karma: 135
i’m not bothered by your comments.
your first reply seemed to be about how i worded the point (you wrote “obnoxiously posed”, and reworded it) rather than pedanticness/irrelevance. i mentally replaced “this is obnoxious” with “this makes me feel annoyed”, which i think is okay to say. i also considered letting you know i’m autistic, which makes me word things differently or more literally[1] or in ways that can seem to have unintended emotional content. (i wonder if that’s what made it feel like “marking it up in red pen”)
onto object-level: what i wrote actually seemed substantive to me, i.e. it really did seem to me that the quote in point 2 was strongly misrepresenting the position the post intended to argue against, so i wouldn’t consider it pedantic. (it could separately be false, of course)
If quila really cares about where the scout mindset metaphor falls apart they could have probed that instead of just dinging me as they are the referee
it did not occur to me that you might endorse the scout/soldier metaphor, and just be using the existence of scout/soldier in ‘scout/soldier mindset’ to bring it up; so yes, if that’s actually the case, it would have been better to notice that and then either not comment on it or probe it as you say. using a metaphor is not invalid.
here’s how i perceived it at the time: ‘scout mindset’ and ‘soldier mindset’ have particular meanings, so whether traditional soldiers are necessary for traditional scouts is a different topic. writing about them instead seemed ‘opportunistic’ in some sense, as if the text was using the terminological overlap to sneak through an argument about one as about the other.
i wonder if this thread could have been mitigated if i were more clear about that in my initial comment. if anyone has advice it is welcome.
- ^
maybe ‘more structured like the thought is structured internally’
- ^
I think we have all the info we need to contradict the fear of not being a scout in her metaphor. Scouts are important for success in battle because accurate information is important to draw up a good battle plan. But those battle plans are worthless without soldiers to fight the battle! “Everyone Should be a Mapmaker and Fear that Using the Map to Actually Do Something Could Make Them a Worse Mapmaker” would be a much less rousing title, but this is how many EAs and rationalists have chosen to interpret the book.
seems locally invalid.[1]
argues from the meaning of terms in a metaphor
“Everyone Should be a Mapmaker and Fear that Using the Map to Actually Do Something Could Make Them a Worse Mapmaker” is not a description of the position you want to argue against, because you can do things with information other than optimizing what you say to persuade people.
- ^
‘locally invalid’ means ‘this is not a valid argument’, separate from the truth of the premises or conclusion
in a thread there i mentioned that even for a described ‘ultimate neartermist’, the best action is actually to cause acausal trade (i.e. by causing aligned ASI) with an ASI at an earlier point in time. for a hypothetical value which only cares about near-term beings, this would also be true, because most near-term beings are not on earth.
also, if i consider a hypothetical value which just cares about near-term beings on earth, it may prefer to destroy earth instead of slowly reducing animal suffering. ‘would want to destroy earth’ is a classical response to the idea of pure negative utilitarianism, but it would apply to standard utilitarianism too if the things valued (in this hypothetical case, just near-term beings on earth) experienced more bad than good which could not be mitigated enough in the near-term.
(disclaimer: the ‘neartermism’ of actual humans is probably importantly different to these, probably more reliant on moral intuition than these literal interpretations. i’m a longtermist myself.)
maybe wacky anthropics stuff?
this comes to mind
[strong upvoted for being well-formed criticism]
Almost any form of maximization as a lifestyle is likely to be neutral at best, unhealthy at worst, with maximization of any rational endeavor skewing towards predictably unhealthy and harmful. Maximization is fundamentally incompatible with good mental health. You can’t “just have a little bit of maximization”; it’s all or nothing.
how would you respond to the idea that good mental health is instrumental to maximization? that’s a standard position in my impression.
(Commenting as I read)
In light of the conflicting research cited above, it would be overly simplistic to assume that those with high levels of malevolence are consistently aware of and endorse their traits, with an internal monologue[9] that goes something like this: “I’m so evil and just want to maximize my own power and gratify my own desires, no matter how much suffering this causes for everyone else, hahaha.”[10] Although some people may think like that, it would be wrong to assume that everyone with high levels of malevolence thinks in this way.
I think the reason that inner monologue feels implausible is that the statement is explicit. If someone really held that attitude/goal, I’d expect it to be implicit: where their inner monologue wouldn’t directly say, “I just want to gratify my own desires at the expense of others”, but it would contain object-level reasoning about how to do that, and judgements of others that strongly correlate with whether they advance or are barriers to the goal, where the goal is an implicit background factor.
And as you note, most people do have some non-negligible level of this:
Everyday experience suggests, for example, that most people care a lot more about their self-interest than is remotely justified by impartial benevolence
a few pieces of this advice seem to be about how to manipulate others in subtle ways.
You can talk about specific things while being pleasant, I dare say, agreeable [...] pragmatically, people will be much likely more susceptible to help you if they associate you with someone who is fun/agreeable [...] try to be as agreeable as possible
i interpret ‘try to be agreeable’ to mean ‘try to appear to agree more than you would if you were being fully honest’ - because, given this is advice i.e. meant to be actionable, it’s not just saying that people who (by coincidence) genuinely agree have a natural advantage. it’s saying, actually intentionally try to seem agreeable, to cause them to associate you with a positive feeling, to make them ‘more susceptible to help you’.
Adapt/mirror people’s behaviour. If someone has a very focused way of talking about things, speaking fast, being curt and concise, mirror this. If someone likes to expand on personal anecdotes, shows a slower pacing, comments on the food, do that too. They will feel more comfortable. [...] If the vibe is good, it means that you’ll be able to reach out later for more content.
i don’t know if others may not mind this, but at least personally, i would not want people to do this with me. if someone is trying to influence my mind in ways i am not aware of, i want to know they are doing this so i can try to account for the effect (or, realistically, ask them not to, or not befriend them if they seemed to practice a wide range of such techniques—i’ve unfortunately met people who do).
i’d guess that mirroring behavior causes the one being mirrored to subtly intuit that they are more similar than they really are, leading to feeling more comfortable around that person.
i think {the net effects we’d observe on how friendships/allyships form in worlds where all EAs try to subtly manipulate each other} are not net good. i imagine it would lead to friendships/allyships being determined more (relative to before) by who’s good at applying these (and other) tactics, and so less by the substantive factors which should matter.
also, i think there is possibility for nuance about where the line is between {being kind and trying to create a positive environment} and manipulation. some forms of trying to influence how someone feels seem okay to me, like if someone is sad, and you want to make them feel less sad, (and they know you’ll be doing this and that’s why they’re talking to you). i guess the relevant line to me is whether it’s intended to help the person, like in that case, or whether it’s intended to influence how they perceive you to gain some sort of advantage from them. the two pieces of advice i quoted seem to be the latter kind.
(to be clear, this criticism doesn’t apply to most of the points, which are probably good advice; i write this because i know criticism can feel bad, and i don’t want to cause that.)
- ^
if someone told me they were doing it, i would actually ask them not to.
if it seemed like they were someone for whom this was just one thing in a wide arsenal of other such subtle tactics, i’d also probably want to not become friends with them.
- ^
i agree with some other comments, just sharing some thoughts that haven’t been posted here yet.
i think that, purely consequentially, you can say that you personally do more good by continuing to purchase products derived from animal suffering (or continuing to do any other deontologically bad thing to others), because doing so makes you happier, or is more convenient, and this lets you be more effective—and that might really be true. to that extent, this isn’t even an objection.
that said, when i consider situations involving the use of animal products, i tend to imagine what i would prefer, and how i would feel—if i were still me, with my current values and mind—but the roles were swapped; if it was me in a factory farm, and some alien altruist in the equivelant position to the one i am in, in an alien civilization similar to humans’. i ask myself, would i be okay with them doing <whatever> with <thing derived from my suffering>?
and sometimes the answer is yes. if they’re cold at night and they’re in a situation where the only blanket is made of material derived from my suffering (analogy to wool), and they’re feeling conflicted, then okay, they can use it. they’re on my side.
if the request was, “can i eat your flesh because i think i derive personal enjoyment from it and i think that lets me be more effective, given i don’t feel particularly disturbed by this situation?” then i would (metaphorically) conclude that i am in hell. that this is the altruist angel who is supposedly going to save us. that this is their moral character.[1]
again, this is not an objection per se—it’s separate from whether the consequential argument is true, and if it is i guess i prefer you to follow it—it’s just some related thoughts about the moral status of the world in which it is true. i am not saying you are wrong, but that if you are not wrong it is wrong for the world to be this way.
- ^
to be clear, i’m not saying you are evil and i don’t want you to feel bad from reading this.
- ^
5. the value of something like, how EA looks to outsiders? that seems to be the thing behind multiple points (2, 4, 7, and 8) in this which was upvoted, and i saw it other times this debate week (for example here) as a reason against the animal welfare option.
(i personally think that compromising epistemics for optics is one way movements … if not die, at least become incrementally more of a simulacrum, no longer the thing they were meant to be. and i’m not sure if such claims are always honest, or if they can secretly function to enforce the relevance of public attitudes one shares without needing to argue for them.)
Though, what if I consider the fact that many people have independently reached a certain belief to itself be evidence that that belief might be true?
that is a form of evidence. if people’s beliefs all had some truly-independent probability of being correct, then in a large society it would become extreme evidence for any belief that >50% of people have, but it’s not actually true that people’s beliefs are independent.
human minds are similar, and human cultural environments are similar. often people’s conclusions aren’t actually independent, and often they’re not actually conclusions but are unquestioned beliefs internalized from their environment (parents, peers, etc). often people make the same logical mistakes, because they are similar entities (humans).
you still have to reason about that premise, “peoples conclusions about <subject> are independent”, as you would any other belief.
and there are known ways large groups of humans can internalize the same beliefs, with detectable signs like ‘becoming angry when the idea is questioned’.
(maybe usually humans will be right, because most beliefs are about low level mundane things like ‘it will be day tomorrow’. but the cases where we’d like to have such a prior are exactly those non-mundane special cases where human consensus can easily be wrong.)
And I apologize for the confusion. I am, as you suggested, still trying to figure out my real position, and coming up with arguments on the spot that mix my internal sentiments with external pressures in ways that may seem incoherent.
Thank you for acknowledging that.
Considering or trying on different arguments is good, but I’d suggest doing it explicitly. For example, instead of “I meant X, not Y” (unless that’s true), “How about new-argument X?” is a totally valid thing to say, even if having (or appearing to have) pinned-down beliefs might be higher status or something.
Some object-level responses:
I should clarify, I am not saying we should pretend to have beliefs closer to theirs. I am saying that having such divergent views will make it harder to recruit them as EAs. It would therefore be better for EA as a movement if our views didn’t diverge as much.
This sounds like it’s saying: “to make it easier to recruit others, our beliefs should genuinely be closer to theirs.” I agree that would not entail lying about one’s beliefs to the public, but I think that would require EAs lying to themselves[1] to make their beliefs genuinely closer to what’s popular.
For one’s beliefs about what is true to be influenced by anything other than evidence it might be or not be true, is an influence which will tend to diverge from what is true, by definition.
I don’t think EAs should (somehow subtly) lie to themselves. If I imagine the EA which does this, it’s actually really scary, in ways I find hard to articulate.
And I think there can be epistemic compromise. You give the benefit of the doubt to other views by admitting your uncertainty and allowing the possibility that you’re wrong, or they’re wrong, and we could all be wrong
Sure, there can be epistemic compromise in that other sense, where you know there’s some probability of your reasoning being incorrect, or where you have no reason to expect yourself to be correct over someone who is as good at reasoning and also trying to form correct beliefs.
But it’s not something done because ‘we need popular support to get things done’.
- ^
this reminded me of this: If we can’t lie to others, we will lie to ourselves by Paul Christiano.
Many apparent cognitive biases can be explained by a strong desire to look good and a limited ability to lie; in general, our conscious beliefs don’t seem to be exclusively or even mostly optimized to track reality. If we take this view seriously, I think it has significant implications for how we ought to reason and behave.
- ^
we can’t know what their “morality” is
Agreed, I mean that just for this subject of factory farming, it’s tractable to know their preferences.
My point was that from the perspective of convincing humans to become EAs, choosing to emphasize animal welfare is going to make the job more difficult, because currently many non-EA humans are less sympathetic to animal suffering than human suffering.
That’s not the position I was responding to. Here is what you wrote:
It’s fair to point out that the majority has been wrong historically many times. I’m not saying this should be our final decision procedure and to lock in those values. But we need some kind of decision procedure for things, and I find when I’m uncertain, that “asking the audience” or democracy seem like a good way to use the “wisdom of crowds” effect to get a relatively good prior.
That seems like you’re proposing actually giving epistemic weight to the beliefs of the public, not just { pretending to have the views of normal humans, possibly only during outreach }. My response is to that.
From your current comment:
Giving more epistemic weight to popular morality is in the light that we need popular support to get things done, and is a compromise with reality, rather than an ideal
Epistemic (and related terms you used, like priors) are about how you form beliefs about what is true. They are not about how you should act, so there cannot be an ‘epistemic compromise with the human public’ in the sense you wrote—that would instead be called, ‘pretending to have beliefs closer to theirs, to persuade them to join our cause’. To say you meant the latter thing by ‘epistemic weight’ seems like a definitional retreat to me: changing the definition of some term to make it seem like one meant something different all along.
(Some humans perform definitional retreats without knowing it, typically when their real position is not actually pinned down internally and they’re coming up with arguments on the spot that are a compromise between some internal sentiment and what others appear to want them to believe. But in the intentional case, this would be dishonest.)
I agree that ideally, if we could, we should also get those other preferences taken into consideration. I’m just using the idea of human democracy as a starting point for establishing basic priors in a way that is tractable.
There’s not actually any impractical ‘ideal-ness’ to it. We already can factor in animal preferences, because we already know them, because they reactively express their preference to not be in factory farms.
(Restating your position as this also seems dishonest to me; you’ve displayed awareness of animals’ preferences from the start, so you can’t believe that it’s intractable to consider them.)
the average person on the street is likely to view the idea that you could ever elevate the suffering of any number of chickens above that of even one human child to be abhorrent.
the average animal in a factory farm is likely to view the idea that you could ever elevate the suffering of one human over that of an unbounded amount of animal children to be abhorrent, too.
[note: i only swapped the order of humans/animals. my mind predicts that, at least without this text, this statement, but not the quoted one, would elicit negative reactions or be perceived as uncivil, despite the symmetry, because this kind of rhetoric is only normal/socially acceptable in the original case.]
if giving epistemic weight to to popular morality (as you wrote you favor)[1], you’d still need to justify excluding from that the moralities of members of non-dominant species, otherwise you end up unjustly giving all that epistemic weight to whatever might-makes-right coalition takes over the planet / excludes others from ‘the public’ (such as by locking the outgroup in factory slaughter facilities, or extermination camps, or enslaving them), because only their dominant morality is being perceived.
otherwise, said weight would be distributed in a way which is inclusive of animals (or nazi-targeted groups, or enslaved people, in the case of those aforementioned moral catastrophes).
You can counter with a lot of math that checks out and arguments that make logical sense
this seems to characterize the split as: supporting humans comes from empathy, supporting animal minds comes from ‘cold logic and math’. but (1) the EA case for either would involve math/logic, and (2) many feel empathy for animals too.
- ^
(to be clear, i don’t agree, this is just a separate point)
- ^
under neartermism, which is not my view but which may be the spirit of the question, animal welfare seems obviously better because non-human animals are extremely neglected by human civilization, either left to die in the wild or cut up in mechanized torture facilities.
under longtermism, it’s basically a question of which could positively effect the values of the first agent superintelligence. probably neither would have a strong effect, but conditional on an effect being had, i’d guess it would route through the increased moral progress caused by animal welfare advocacy, somehow leading to a less human-centric forever-value.
(100% under neartermism, ~80% under longtermism in recognition of uncertainty)
i see, thanks for explaining!
i’m modelling this as: basic drive to not die → selects values that are compatible with basic drive’s fulfillment.
i’ve been wondering if humans generally do something like this. (in particular to continue having values/cares after ontological crises like: losing belief in a god, or losing a close other who one was dedicated to protecting.)
that I was a burden and that the resources expended keeping me alive were better used on someone who actually wanted to live
in case anyone has similar thoughts: to have the level of altruism to even consider the question is extremely rare. there are probably far better things you can do, than just dying and donating; like earning to give, or direct research, or maybe some third thing you’ll come up with. (most generally, the two traits i think are needed for research are intelligence and creativity. this is a creative, unintuitive moral question to ask. and my perception is that altruism and intelligence correlate, but i could be wrong about that, or biased from mostly seeing EAs.)
It’s my own experience that biases me in the way I described.
can you explain how?
i believe extreme suffering had the opposite effect on me, making me become a suffering-focused altruist. i don’t actually understand how it could make someone ~not disvalue suffering. (related: ‘small and vulnerable’).
(i mean, i have guesses about how that could happen: like, maybe ~not disvaluing it was the only way to mentally cope with the vast scale of it. living in a world one believes to be evil is hard; easier to not believe it’s evil, somehow; have heard this is a reason many new animal-suffering-boycotters find it hard to continue having an animal-caring worldview.
or, maybe experiencing that level of suffering caused a buddhist enlightenment like thing where you realized suffering isn’t real, or something. though, happiness wouldn’t be real either in that case. i’m actually adjacent to this view, but it sure feels real for the animals, and i would still like to make the world be good for those who believe in it.)
from your other comment:
hopefully you can understand why I have my rather eccentric and unorthodox views.
it still feels mysterious / that comment seems more like ‘what you prefer and uncertainty’ than ‘why / what caused you to have those preferences’
[i have the value that] almost all lives, even highly unpleasant ones, are worth living, and that I tend to weigh moments of happiness much more than equivalent moments of suffering, as this avoids what I see as philosophically problematic implications such as suicide for chronically depressed people, or nuking the rainforest as a net positive intervention.
do you mean that you chose this position because it avoids those conclusions? if so:
then the process you used was to select some (of many possible) moral axioms which lead to the conclusion you like.
i don’t think that would mean the axiom is your true value.
but if choosing axioms, you could instead just follow the conclusions you like, using an axiom such as “my morality is just complex [because it’s godshatter]”.
separately, the axiom you chose introduced a new ‘problematic’ conclusion: that someone in a mechanized torture chamber, who will be there for two more years, (during which their emotional state will mostly only change between depression and physical-harm-induced agony—maybe there will also be occasional happiness, like if another animal tries to comfort them), and then die without experiencing anything else—should be kept alive (or be created) in that situation instead of ceased to exist (or not be created), when these are the only choices.
that’s definitely something the universe allows one to prefer, as all moral preferences are. i’m just pointing it out because i think maybe it will feel immoral to you too, and you said you chose axioms to avoid problematic or immoral-feeling things.
in case it doesn’t feel wrong/‘philosophically problematic’ now, would it have before, before you started using this axiom, and so before your moral intuitions crystallized around it?
almost all lives, even highly unpleasant ones, are worth living
as i am a moral anti-realist, i cannot argue against a statement of what one values. but on priors about humans, i am not sure if you would actually want the world to be arranged in a way which follows this value, if you fully understood what it entails. have you spent time imagining, or experiencing, what it is like to live a life of extreme suffering? what it is like for it to be so bad that you desperately prefer nonexistence to it?
now, such lives could still be considered ‘worth it’ overall if they eventually get better or otherwise are considered meaningful somehow. but a life of just or almost just that? are you sure about that? and does this imply you would prefer to create a billion people whose lives last forever and almost only consist of depression/physical agony, if the only alternative was for them not to exist and no one happier to exist in their place—and if it does imply that, are you sure about that also? (maybe these fall under what ‘almost all’ doesn’t include for you, but then you’d also consider the lives of animals in mechanized torture facilities negatively worth living.)
(sometimes when humans ask, “are you sure about endorsing that”, there’s a subtext of social pressure, or of more subtly invoking someone’s social conformity bias so they will conform ~on their own. i do not mean it that way, i really mean it only as prompting you to consider.)
“The traditional way to celebrate Petrov day is to not destroy the world.”
- Rob Miles in their video on Petrov
What is malevolence? On the nature, measurement, and distribution of dark traits was posted two weeks ago (and i recommend it). there was a questionnaire discussed in that post which tries to measure the levels of ‘dark traits’ in the respondent.
i’m curious about the results[1] of EAs[2] on that questionnaire, if anyone wants to volunteer theirs. there are short and long versions (16 and 70 questions).
(or responses to the questions themselves)
i also posted the same quick take to LessWrong, asking about rationalists