Another relevant article on “machine psychology” https://arxiv.org/abs/2303.13988 (interestingly, it’s by a co-author of Peter Singer’s first AI paper)
Timothy Chan
Karma: 836
You seem to have written against proposing norms in the past. So apologies for my mistake and I’m glad that’s not your intention.
To be clear, I think we should be free to write as we wish. Regardless, it still seems to me that voicing support for an already quite popular position on restricting expression comes with the risk of strengthening associated norms and bringing about the multiple downsides I mentioned.
Among the downsides, yes, the worry that strengthening strong norms dealing with ‘offensive’ expression can lead to unfair punishments. This is not a baseless fear. There are historical examples of norms on restricting expression leading to unfair punishments; strong religious and political norms have allowed religious inquisitors and political regimes to suppress dissenting voices.
I don’t think EA is near the worst forms of it. In my previous comment, I was only pointing to a worrying trend towards that direction. We may (hopefully) never arrive at the destination. But along the way, there are more mild excesses. There have been a few instances where, I believe, the prevailing culture has resulted in disproportionate punishment either directly from the community or indirectly from external entities whose actions were, in part, enabled by the community’s behavior. I probably won’t discuss this too publicly but if necessary we can continue elsewhere.
It seems that you, correct me if I’m wrong, along with many who agree with you, are looking to further encourage a norm within this domain (on the basis of at least one example, i.e. the one example from the blog post, that challenged it).
This might benefit some individuals by reducing their emotional distress. But strengthening such a norm that already seems strong/largely uncontroversial/to a large extent popular in the context of this community, especially one within this domain, makes me concerned in several ways:
Norms like these that target expression considered offensive seem to often evolve into/come in the form of restrictions that require enforcement. In these cases, enforcement often results in:
“Assholes”/”bad people” (and who may much later even be labeled “criminals” through sufficient gradual changes) endure excessive punishments, replacing what could have been more proportionate responses. Being outside of people’s moral circles/making it low status to defend them makes it all too easy.
Well-meaning people get physically or materially (hence also emotionally) punished for honest mistakes. This may happen often—as it’s easy for humans to cause accidental emotional harm.
Enforcement can be indeed more directed but this is not something we can easily control. Even if it is controlled locally, it can go out of control elsewhere.
Individuals who are sociopolitically savvy and manipulative may exploit their environment’s aversion of relatively minor issues to their advantage. This allows them to appear virtuous without making substantial contributions or sacrifices.
At best, this is inefficient. At worst, to say the least—it’s dangerous.
Restrictions in one domain often find their way into another. Particularly, it’s not challenging to impose restrictions that are in line with illegitimate authority as well as power gained through intimidation.
This can lead people to comfortably dismiss individuals who raise valid but uncomfortable concerns, by labeling them as mere “assholes”. To risk a controversial, but probably illuminating example, people often unfairly dismiss Ayaan Hirsi Ali as an “Islamophobe”.
This burdens the rest of society with those other restrictions and their consequences. Those other restrictions can range from being a mere annoyance to being very bad.
I’d be less worried (and possibly find it good) if such a norm was strengthened in a context where it isn’t strong, which gives us more indication that the changes are net positive. However, it’s evident that a large number of individuals here already endorse some version of this norm, and it is quite influential. Enthusiasm could easily become excessive. I sincerely doubt most people intend to bring about draconian restrictions/punishments (on this or something else), but those consequences can gradually appear despite that.
FWIW, Brian Tomasik does a fuzzies/utilons split thing too. One justification is that it helps avoid cognitive dissonance between near-term causes and, in his mind, more effective longtermist causes.
My position, in contrast, is that I acknowledge the epistemic force of far-future arguments but maintain some commitment to short-term helping as an intrinsic spiritual impulse. Along the lines of Occam’s imaginary razor, this allows me to avoid distorting my beliefs about the far-future question based on emotional pulls to stop torture-level suffering in the present. In the face of emotion-based cognitive dissonance, it’s often better to change your values than to change your beliefs.
It might be overly confusing to call it “changing [my ideal] values”. It’s more that I have preferences for both. Some that seem like ones I would ideally like to keep (minimizing suffering in expectation), but some that as a human, for better or worse, I have (drives to reduce suffering in front of me, sticking to certain principles...).
If the price of a split in donations/personal focus results in me becoming more effective at the far-future stuff that I think is more important for utilons, in a way that makes those utilons go up, then that seems worth it.
Yeah, in a scenario with “nation-controlled” AGI, it’s hard to see people from the non-victor sides not ending up (at least) as second-class citizens—for a long time. The fear/lack of guarantee of not ending up like this makes cooperation on safety more difficult, and the fear also kind of makes sense? Great if governance people manage to find a way to alleviate that fear—if it’s even possible. Heck, even allies of the leading state might be worried—doesn’t feel too good to end up as a vassal state. (Added later (2023-06-02): It may be a question that comes up as AGI discussions become mainstream.)
Wouldn’t rule out both American and Chinese outside of respective allied territory being caught in the crossfire of a US-China AI race.
Political polarization on both sides in the US is also very scary.
I generally agree with the meritocratic perspective. It seems a good way (maybe the best?) to avoid tit-for-tat cycles of “those holding views popular in some context abuse power → those who don’t like the fact that power was abused retaliate in other contexts → in those other contexts, holding those views results in being harmed by people in those other contexts who abuse power”.
Good point about the priors. Strong priors about these things seem linked to seeing groups as monoliths with little within-group variance in ability. Accounting for the size of variance seems under-appreciated in general. E.g., if you’ve attended multiple universities, you might notice that there’s a lot of overlap between people’s “impressiveness”, despite differences in official university rankings. People could try to be less confused by thinking in terms of mean/median, variance, and distributions of ability/traits more, rather than comparing groups by their point estimates.
Some counter-considerations:
Religion and race seem quite different. Religion seems to come with a bunch of normative and descriptive beliefs that could affect job performance—especially in EA—and you can’t easily find out about those beliefs in a job interview. You could go from one religion to another, from no religion to some religion, or some religion to no religion. The (non)existence of that process might give you valuable information about how that person thinks about/reflects on things and whether you consider that to be good thinking/reflection.
For example, from a irreligious perspective, it might be considered evidence of poor thinking if a candidate thinks the world will end in ways consistent with those described in the Book of Revelation, or think that we’re less likely to be in a simulation because a benevolent, omnipotent being wouldn’t allow that to happen to us.
Anecdotally, on average, I find that people who have gone through the process of abandoning the religion they were raised with, especially at a young age, to be more truth-seeking and less influenced by popular, but not necessarily true, views.
Religion seems to cover too much. Some forms of it seems to offer immunity to act in certain ways, and the opportunity to cheaply attack others if they disagree with it. In other communities, religion might be used to justify poor material/physical treatment of some groups of people, e.g. women and gay people. While I don’t think being accepting of those religions will change the EA community too much, it does say something to/negatively affect the wider world if there’s sufficient buy-in/enough of an alliance/enough comfort with them.
But yeah, generally, sticking to the Schelling point of “don’t discriminate by religion (or lack-thereof)” seems good. Also, if someone is religious and in EA (i.e., being in an environment that doesn’t have too many people who think like them), it’s probably good evidence that they really want to do good and are willing to cooperate with others to do so, despite being different in important ways. It seems a shame to lose them.
I’ve been doing a 1-year CS MSc (one of the ‘conversion’ courses in the UK). I took as many AI/ML electives as I’m permitted to/can handle, but I missed out on an intro to RL course. I’m planning to take some time to (semi-independently) up-skill in AI safety after graduating. This might involve some projects and some self-study.
It seems like a good idea to be somewhat knowledgeable on RL basics going forward. I’ve taken (paid) accredited, distance/online courses (with exams etc.) concurrently with my main degree and found them to be higher quality than common perception suggests—although it does feel slightly distracting to have more on my plate.
Is it worth doing a distance/online course in RL (e.g. https://online.stanford.edu/courses/xcs234-reinforcement-learning ) as one part of the up-skilling period following graduation? Besides the Stanford online one that I’ve linked, are there any others that might be high quality and worth looking into? Otherwise, are there other resources that might be good alternatives?
So in my comment I was only trying to say that the comment you responded to seemed to point to something true about the preferences of women in general vs. the preferences of women who are “highly educated urban professional-managerial class liberals in the developed world”.
Such perspectives seem easy to miss for people (in general/of all genders, not just women) belonging to the elite U.S./U.S.-adjacent progressive class—a class that has disproportionate influence over other cultures, societies etc., which makes it seem worthwhile to discuss in spaces where many belong to this class.
About your other point, I guess I don’t have much of an opinion on it (yet), but my initial impression is that it seems like openness comes in degrees. Compared to other movements, I also rarely observe ‘EA’ openly declaring itself hostile to something (e.g. “fraud is unacceptable” but there aren’t really statements on socialism, conservatism, religions, culture...).
There might be differences between identifying with feminism and ‘being open to scholars of feminism, queer studies and gender studies’ though. Most Americans probably aren’t familiar with academia to know of its latest thinking.
And like how different people have different notions of what counts as discriminatory, racist, sexist, or not discriminatory, racist, sexist, it’s possible that different people have different notions of what ‘feminism’ means. (Some might consider it a position supporting equal rights between the sexes—others a position supporting women’s rights. They might be thinking of the second, third, or fourth wave etc.)
The supplementary document containing the survey questions suggests the question asked was “How well, if at all, do each of the following describe you?” followed by “Environmentalist”, “Feminist” and “A supporter of gun rights” (in random order), which doesn’t seem to specify one specific notion of ‘feminist’ for survey participants to consider.
Although, to be fair, maybe there’s actually more agreement among Americans on the definition of feminist (in the year of the survey, 2020) than I’m expecting.
In any case, I expect the differences in preferences of elite Anglosphere/U.S. women, and not-necessarily-elite, non-Anglosphere/non-U.S. women in general (e.g., in Europe, Asia, South America) would still be quite large.
I’d be interested in this :)
(Presumably if coding can be done faster, AI can be created more quickly too)
Wait, which mechanisms did you have in mind?
AI → software coded up faster → more software people go into AI → AI becomes more popular?
AI → coding for AI research is easier → more AI research
AI → code to implement neural networks written faster → AI implemented more quickly (afaik not too big a factor? I might be wrong though)
AI → code that writes e.g. symbolic AI from scratch → AI?
Yeah, there might be no end to how much you can understand about EA (or, more generally, stuff about the world that’s relevant to altruism).
I certainly have my own blindspots but when talking to many other EAs I do notice that there a lot of topics they seem unfamiliar with:
The extent of the uncertainty we have about the philosophy of mind
Chances of being in a simulation/percentage of copies in a simulation and how that affects the expected value of various actions
Philosophy of science/core assumptions behind how one thinks the world works
Views of people in other parts of the world/society.
Relatedly, how futures led by influential people in other countries might compare to futures led by influential people in their own countries.
Reasons to think that there are quite a lot of things that are low tractability
Noticing how they came to be who they are/their place in history
Impact of various activities on wild animal suffering
I don’t claim to know everything about the above. And of course, others who know more about other things might notice that there are a lot topics I’m unfamiliar with. Some topics I haven’t really thought about that much relative to a lot of people working at EA organizations:
Anthropics
Alien civilizations, how that affects priorities in longtermism
Ethical theories that are super formalized (my moral anti-realism doesn’t make me that motivated to look into them)
Acausal interactions and decision theory
Note that I was specifically talking about people (of all genders/in general) in parts of the Anglosphere being “sensitive”. I’ll quote myself.
In parts of the Anglosphere, people seem more sensitive to an extent that in some cases I would consider them to be overreaching.
Of course, it’s also influencing much outside of it.
Although, there does seem to be a phenomenon where a combination of being young, female, and being politically liberal, makes someone particularly vulnerable to anxiety and depression. This seems to have also increased in recent years in the U.S. https://jonathanhaidt.substack.com/p/mental-health-liberal-girls I do prefer that we can reverse such trends.
EDIT: Apart from quoting a part of my previous comment and stating a preference for there to be less anxiety and depression, everything in this comment is purely descriptive. Are people strong downvoting over offense over that? It’s really not a good sign of community epistemic health.
If you do want my (normative) opinions on all this, I think it’s beneficial and possible for the subset of people in Anglosphere whom I was referring to, to reverse recent trends and become more resilient. There is currently a combination of high false positive rates + expanded notions of perceived malice and of harm, which isn’t very good for your democratic societies, in my opinion.
EDIT2: It seems like the people responding to me don’t really consider the possibility of missteps/someone’s intent? That seems unfortunate.
EDIT: the link I posted earlier (https://hiddentribes.us/) is quite relevant in introducing nuance to what I wrote about people in “parts of the Anglosphere” becoming more sensitive. It’s different for different people in the Anglosphere. E.g. about a 49-51 split on “harassment is commonplace” to “too many ordinary behaviors are labeled as sexual harassment” in the U.S.
I suspect my probabilities are probably very different from yours mainly because of different ideas of what harassment means.
For instance, one acquaintance of mine puts his hands around me in a somewhat intimate way sometimes, but I honestly don’t consider it “harassment” (edit: it does make me feel slightly uncomfortable and I haven’t raised the issue, but I really don’t think he has bad intentions—we’re probably just raised differently etc.). One friend makes sexual remarks a lot—some people might feel very uncomfortable—I don’t really.
I think those examples point to differences in expectations of what’s comfortable/uncomfortable to people. In parts of the Anglosphere, people seem more sensitive to an extent that in some cases I would consider them to be overreaching. Sure, maybe avoid those things if you think if it makes people uncomfortable (or not, if you think there are risks of safetyism?) - but I don’t see the case for expanding a concept that comes with significant legal and social consequences.
I’m guessing I’m less inclined than you are to consider discomfort to mean harassment (I believe Aella made several great points in her post that are relevant to this view). In a larger number of cases where there’s an accusation that simply refers to “sexual harassment” I’m doubtful that it means what (many others and) I have in mind.
Update (2023-04-27): In retrospect, I think I could have underestimated the probabilities, although perhaps not by much?
I still think there are strong differences between different parts of society/the world. A lot has changed this past decade, it seems. I would probably still assign higher-than-average probabilities to accusations in places where it’s more costly to accuse (e.g. outside of English-speaking countries, more conservative areas etc.), and lower-than-average probability to accusations in places where it’s less costly (or even slightly status-elevating) to accuse (e.g. within English-speaking countries, more progressive areas etc.).
For now, I’ll avoid stating a new number so I don’t anchor to it while updating. I should also be less susceptible to pressures from the community to think in a certain way, with more time to think about it.
In any case, I don’t appreciate uncharitable interpretations of views that challenge apparent community consensus. I intend to continue to speak my mind.
lilly’s comment was on misconduct, which seems to have a lower rate of being true than assault, given that it encompasses milder problems, as well as different interpretations of what harassment means (this seems particularly the case in the Anglo-American world). My guess is that base rates could range anywhere between 20% and 90%. Recent societal trends have introduced a lot of uncertainty, and have made me doubt accusations more. I think 37.5% prior probability conditional on an accusation (maybe 25% in progressive-leaning social environments) might make sense. (I think I have higher-than-average doubt relative to many EAs on humans acting justly when they perceive an increase in power. It only takes a small percentage of individuals who are willing to exploit things.)
On assault accusations, which titotal’s comment refers to, the base rate seems higher (although the error bars seem large—Scott Alexander on a related statistic). Perhaps I should assign an ~80% prior probability conditional on an accusation. Yeah, that does seem like a “moderately high probability” actually—it makes sense for me to correct my reply to titotal. I think I’d still be less willing to punish severely with that amount of uncertainty, because of “beyond a reasonable doubt” reasons actually, as titotal pointed out, but it’s hard to be confident that that’s the right thing to do.
Thanks for bringing that up again. I realized I misparsed a bit at the end. However, the conclusion it seems to imply seems a bit dubious, i.e. it seems that our starting credence is more ‘resilient’ than the passage presents it to be if we’re anchoring to base rates. I edited in a section to my initial response to that comment.
I’m not sure that I’m equating those two. I personally wouldn’t want to be a person who doesn’t have enough information, but upon hearing a single accusation (from someone who I don’t really know), will presume guilt (i.e. assign a moderately high probability). If A credibly accuses B then that’s different, but I’d assume in most cases, things are more uncertain.
EDIT: Actually, regarding assault, yeah I think it seems to make sense to assign a moderately high probability (say ~80%), and you seem to be right that I’m using the “beyond reasonable doubt” principle here (including in social settings—though I’m uncertain whether or not it’s the right call). Although, lilly’s comment which I was responding to was actually on misconduct, and I think the influence of the “presumption of innocence” factor is larger in those cases. More in my response below.
I agree that legal standards and standards upheld in day-to-day life shouldn’t be the same, but some of the ‘willingness to believe bad things happened’ I’m seeing in the social groups that EA draws from seems a bit immoderate, so I’m cautious of straying too far from those stricter standards (and straying too far also has a risk of undermining them).
To answer your other question (which you have now deleted), I downvoted your first comment instead of disagree-voting because it appeared to me that you were concentrating on instances where we evaluate accusations made by friends, to the exclusion of the vast majority of situations where we evaluate accusations made by individuals who are not friends (fyi, just so it’s less confusing—I make a distinction between friends and acquaintances, and EA seems big enough that not everyone can be considered a friend). That made your comment potentially misleading. However, I do I agree with your comment in that I should believe a friend’s accusations (unless it turns out I’m a poor judge of character).
I now realize that by asking about what I meant by “the majority of cases accusations are not made by friends,” you indicate that you did not make that distinction as I did.
Regarding the TCS PhD, is it possible to work on it remotely from London?