Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
I think the idea of a motivational shadow is a good one, and it can be useful to think about these sorts of filters on what sorts of evidence/argument/research people are willing to share, especially if people are afraid of social sanction.
However, I am less convinced by this concrete application. You present a hierarchy of activities in order of effort required to unlock, and suggest that something like ‘being paid full time to advocate for this’ pushes people up multiple levels:
I don’t believe that the people who are currently doing high quality Xrisk advocacy would counter-factually be writing nasty newspaper hit pieces; these just seem like totally different activities, or that Timnit would write more rigourously if people gave her more money. My impression is that high quality work on both sides is done by people with strong inherent dedication to truth-seeking and intellectual inquiry, and there is no need to first pass through a valley of vitriol before your achieve a motivational level-up to an ascended state of evidence. Indeed, historically a lot of Xrisk advocacy work was done by people for whom such an activity had negative financial and social payoff.
I also think you miss a major, often dominant motivation: people love to criticize, especially to criticize things that seem to threaten their moral superiority.
I think that’s a good critique, although it can be mitigated somewhat with a narrower interpretation. In the narrower view, motivation (e.g., “effort required to unlock”) is a necessary but not sufficient precursor to various actions.
Being a jerk on X requires only low motivation, but if I’m not prone to being a jerk in the first place then my response to that level of motivation will be [no action], which will not result in any criticism. Conditional on someone posting criticism at that level of motivation, the criticism will be ~ in the form of mean tweets, because the motivation level isn’t high enough to unlock higher forms of criticism.
. . . as well as sufficient motivation and resources to do so. As with the lower levels, I suggest that high motivation unlocks high-level work in the sense that it is a necessary but not sufficient precondition. This means that people with strong inherent dedication to truth-seeking and intellectual inquiry will still not produce high-quality work unless they are motivated enough to do so.
I don’t think that’s what the OP argues though.[1] The argument is that the people motivated to seek funding to assess X-risk as a full time job tend to be disproportionately people that think X-risk and the ability to mitigate it significant. So of course advocates produce more serious research, and of course people who don’t think it’s that big a deal don’t tend to choose it as a research topic (and on the rare occasions they put actual effort in, it’s relatively likely to be motivated by animus against x-risk advocates).
If those x-risk advocates had to do something other than x-risk research for their day job, they might not write hit pieces, but there would be blogs instead of a body of high quality research to point to, and some people would still tweet angrily and insubstantially about Sam Altman and FAANG.
Gebru’s an interesting example looked at the other way, because she does write rigorous papers on her actual research interests as well as issue shallow, hostile dismissals of groups in tech she doesn’t like. But funnily enough, nobody’s producing high quality rebuttals of those papers[2] - they’re happy to dismiss her entire body of work based on disagreeing with her shallower comments. Less outspoken figures than Gebru write papers on similar lines, but these don’t get the engagement at all.
I do agree people love to criticize.
the bar chart for x-risk believers without funding actually stops short of the “hit piece” FWIW
EAs may not necessarily actually disagree with her when she’s writing about implicit biases in LLMs or concentration of ownership in tech rather than tweeting angrily about TESCREALs, but obviously some people and organizations have reason to disagree with her papers as well.
A lot of people seem to think EA is this singular, exceptional, incomparable thing, unlike other movements or ideologies or group identities. But I think EA is not special or exceptional or unique at all. It is just another human enterprise with many of the flaws that human enterprises typically have.
I studied philosophy at university and, by the time I was done, I still loved philosophy (maybe just as much or even more than ever), but developed many deep frustrations and critiques with philosophy, and a little bit of cynicism or skepticism toward the field, or some aspects of it. One thing I wished philosophers and philosophy teachers asked themselves more and more pointedly was, “Why is this worth thinking about?” or “Am I wasting my time with this?”. (I imagine if you had a glass of wine or some weed in a private place with any random philosophy PhD student or professor and asked them about the structural problems in their field, there’s a good chance they would have a lot to complain about.)
At university, I got intensely involved with political activism to the point of burnout and mental destabilization. I still believe very strongly in many of the basic ideas I believed in then (e.g. trans rights, economic justice) — maybe even more so, given the subsequent years I’ve had to think on it — but I was scared and disturbed by how amped up and out of control people can get when they spend so much time talking to people in their weird little in-group and have a strong sense of self-righteousness. Very relevant to EA.
EA is something like a combination of philosophy and political activism. When I got involved in my local EA community circa 2015, from the jump, I had a certain wariness about how something like this might go astray. One thing that bothered me then and I still don’t quite get is why people want to say “I’m an EA” or say things like “EAs like to help other EAs”. Why is this part of your personal identity? Maybe it’s fine and normal to want to be part of a group or to label yourself and I’m overthinking it. Maybe it’s largely personal preference.
A thing that weirds me out today is when I see EAs having a weird superiority complex, like the apparent idea that EAs, exceptionally, care about truth and know how to get it, whereas I guess everyone else are just lying fools.
I don’t remember where I read this, but someone pointed out that a lot of EAs just like talking about philosophy and futurism, and then for some reason feel the need to justify that it’s important. (Maybe it feels wasteful or indulgent otherwise? Maybe pleasure is a sin?) My feeling is that people should just be able to enjoy talking about philosophy and futurism, just like they enjoy sports or video games or playing in a band, and that shouldn’t be frowned upon. In fact, it should be looked on as an enriching and somewhat admirable part of life. You don’t have to furiously justify things along the lines of, “Well, if my essay has a 1 in a billion chance of making things go 0.01% better for 10^52 future lives…”
You can devote yourself to an institution, an organization, a movement, an idea or ideal or ideology, a community, a subculture, a career, a cause, or a hobby and not elevate it to the point of “we are the special few this sick, sad world needs”. It can just be fallible humans trying their best, not being perfect, not always knowing for sure what’s true, not always knowing for sure what the right thing to do is.
To the extent that feels uncomfortable or even intolerable to some people, or just not good enough, I wonder if what’s missing in their lives is just being loved for who they really are, without feeling the need to prove that they deserve to be loved.
I’m not sure that I buy that critics lack motivation. At least in the space of AI, there will be (and already are) people with immense financial incentive to ensure that x-risk concerns don’t become very politically powerful.
Of course, it might be that the best move for these critics won’t be to write careful and well reasoned arguments for whatever reason (e.g. this would draw more attention to x-risk so ignoring it is better from their perspective).
Edit: this is mentioned in the post, but I’m a bit surprised because this isn’t emphasized more.
The current situation still feels like the incentives are relatively small compared with the incentive to create the appearance that the existence of anthropogenic climate change is still uncertain. Over decades advocates have succeeded in actually reducing fossil fuel consumption in many countries as well as securing less-likely-to-be-honoured commitments to Net Zero, and direct and indirect energy costs are a significant part of everyone’ household budget.
Not to mention that Big Tech companies whose business plans might be most threatened by “AI pause” advocacy are currently seeing more general “AI safety” arguments as an opportunity to achieve regulatory capture...
Why do you think this? It seems very unclear if this is true to me.
Because their leaders are openly enthusiastic about AI regulation and saying things like “better that the standard is set by American companies that can work with our government to shape these models on important issues” or “we need a referee”, rather than arguing that their tech is too far away from AGI to need any regulation or arguing the risks of AI are greatly exaggerated, as you might expect if they saw AI safety lobbying as a threat rather than an opportunity.
Sure, but there are many alternative explanations:
There is internal and external pressure to avoid downplaying AI safety.
Regulation is inevitable, so it would be better to ensure that you can at least influence it somewhat. Purely fighting against regulation might go poorly for you.
The leaders care at least a bit about AI safety either out of a bit of altruism or self interest. (Or at least aren’t constantly manipulative to such an extent that they choose all words to maximize their power.)
I don’t disagree that these are also factors, but if tech leaders are pretty openly stating they want the regulation to happen and they want to guide the regulators, I think it’s accurate to say that they’re currently more motivated to achieve regulatory capture (for whatever reason) than they are to ensure that x-risk concerns don’t become a powerful political argument as suggested by the OP, which was the fairly modest claim I made.
(Obviously far more explicit and cynical claims about, say, Sam Altman’s intentions in founding OpenAI exist, but the point I made doesn’t rest on them)
I genuinely don’t know if this is an interesting/relevant question that’s unique to EA. To me, the obvious follow-up question here is whether EA is unique or special in having this (average) level of vitriol in critiques of us? Like is the answer to “why so much EA criticism is hostile and lazy” the same answer to “why is so much criticism, period, hostile and lazy?” Or are there specific factors of EA that’s at all relevant here?
I haven’t been sufficiently embedded in other intellectual or social movements. I was a bit involved in global development before and don’t recall much serious vitirol, maybe something like Easterly or Moyo are closest. I guess maybe MAGA implicitly doesn’t like global dev?
But otoh I’ve heard of other people involved in say animal rights who say that the “critiques” of EA are all really light and milquetoast by comparison.
I’d really appreciate answers from people who have been more “around the block” than I have.
Strongly agree. I think there’s also a motivation gap in knowledge acquisition. If you don’t think there’s much promise in an idea or a movement, it usually doesn’t make sense to spend years learning about it. This leads to large numbers of very good academics writing poorly-informed criticisms. But this shouldn’t be taken to indicate that there’s nothing behind the criticisms. It’s just that it doesn’t pay off career-wise for these people to spend years learning enough to press the criticisms better.
I don’t think it requires years of learning to write a thoughtful op-ed-level critique of EA. I’d be surprised if that’s true for an academic paper-level one either
That’s fair! But I also think most op-eds on any topic are pretty bad. As for academic papers, I have to say it took me at least a year to write anything good about EA, and that was on a research-only postdoc with 50% of my research time devoted to longtermism.
There’s an awful lot that has been written on these topics, and catching up on the state of the art can’t be rushed without bad results.
To a large extent I don’t buy this. Academics and Journalists could interview an arbitrary EA forum user on a particular area if they wanted to get up to speed quickly. The fact they seem not to do this, in addition to not giving a right to reply, makes me think they’re not truth-seeking.
I’d like to hope that academics are aiming for a level of understanding above that of a typical user on an Internet forum.
All academic works have a right to reply. Many journals print response papers and it is a live option to submit responses to critical papers, including mine. It is also common to respond to others in the context of a larger paper. The only limit to the right of academic reply is that the response must be of suitable quality and interest to satisfy expert reviewers.
Realistically, it is almost never in an academic’s professional interest to write a reply paper (unless they are completely starved of original ideas). Referees are fickle, and if the reply isn’t accepted at the original journal, very few other journals will even consider it, making it a bad time investment. (A real “right of reply”—where the default expectation switches from ‘rejection’ to ‘acceptance’—might change the incentives here.)
Example: early in my career, I wrote a reply to an article that was published in Ethics. The referees agreed with my criticisms, and rejected my reply on the grounds that this was all obvious and the original paper never should have been published. I learned my lesson and now just post replies to my blog since that’s much less time-intensive (and probably gets more readers anyway).
This sounds like… not having a right of reply? The right means a strong presumption if not an absolute policy that criticized people can defend themselves in the same place as they were criticized. If only many, not all, journals print response papers, and only if you jump through whatever hoops and criteria the expert reviewers put in front of you, I’m not sure how this is different to ‘no right of reply’.
A serious right would mean journals would send you an email with the critical paper, the code and the underlying data, and give you time to create your response (subject to some word limit, copy-editing etc.) for them to publish.
I generally agree with this to the extent that the person authoring the reply had a strong basis for standing—e.g., the published piece represented a direct and sustained criticism on their previously published piece. I would not extend it to cases where the person authoring the reply was more of a third party, as in the story Richard shares here. I am unsure about extending it to cases where the challenged work was not published in an appropriate journal in the first place. It seems a bit odd to guarantee someone journal access to defend work where there is no clear reason to believe the original work was of journal-level quality in the first place.
Yeah I agree Richard’s story is about illustrating journal incentives and behaviour, not specially about a Right of Reply. In the specific Leif Weinar case I would say that Will, and maybe CEA, [should] have a Right of Reply, but a random EA person would not.
I don’t think it’s that strange to accept replies for lower-quality work. A newspaper, when quoting the subject of an article saying “the accusations are false and malicious and we are confident the Judge will side with us” or whatever is guaranteeing them space even though they wouldn’t have given the subject a platform normally. The purpose of the reply is to allow readers to better evaluate the criticism, which was deemed sufficient quality to publish, and if the reply is low quality then that is informative by itself. Important to this is that replies should be constrained to a much shorter length than the criticism itself.
The crux for me is whether “published reply in journal” could and would be (mis?)construed by some people as a sort of quality signal.
To the extent that journals are allowing replies-by-permission by third parties, then we don’t want to diminish the value of getting one of those published. As Richard notes, the incentives are already weak. Yet I think replies-by-permission are undervalued already, because I think ~ direct dialog is usually better than ~ talking past one another.
If I were too concerned about this issue for a reply author with standing, I’d probably at least offer to publish an Editor’s Note with a link to the reply author’s off-journal response.
Moreover, even if a critic has a sufficiently high level of motivation in the abstract, it doesn’t follow that they will be incentivized to produce much (if any) “polite, charitable, good-faith, evidentiarily rigorous” work. (Many) critics want to be effective too—and they may reasonably (maybe even correctly!) think that effort devoted to producing castle memes produces a higher ROI than polishing, simplifying, promoting, and defending their more rigorous critiques.
For example, a committed e/acc’s top priority is arguably the avoidance of government regulation that seriously slows down AI development. Memes are more important for 90%, perhaps 99%, of the electorate—so “make EA / AI safety a topic of public scorn and ridicule” seems like a reasonable theory of change for the e/acc folks. When you’re mainly trying to tear someone else’s work down, you may plausibly see maintaining epistemic rigor in your own camp as relatively less important than if you were actually trying to build something.
Great post, titotal!
It looks like you meant to write something after this.
Relatedly, there is this post from Nuño Sempere.
This is interesting, thanks. Though I wanted to flag that the volume of copyediting errors means I’m unlikely to share it with others.
Executive summary: Motivation gaps between advocates and skeptics of a cause can lead to an imbalance in the quality and quantity of arguments on each side, making it difficult to accurately judge the merits of the cause based on the arguments alone.
Key points:
Advocates of a cause (e.g. religion, AI risk) are intrinsically motivated to make high-effort arguments, while skeptics lack inherent motivation to do the same.
This leads to an asymmetry where advocate arguments appear more convincing, even if the cause itself may be flawed.
Counter-motivations like moral backlash, politics, money, annoyance, and entertainment can somewhat close the motivation gap for skeptics, but introduce their own biases.
In-group criticism alone is insufficient due to issues like jargon barriers, agreement bias, evaporative cooling, and conflicts of interest.
To account for motivation gaps, adjust the weight given to each side’s arguments, be more charitable to critics, seek out neutral parties to evaluate the cause, and signal boost high-effort critiques.
EA should make an extra effort to highlight good-faith criticism to encourage more productive engagement from skeptics.
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.
Love this thanks for the insights, this definitely helps answer some of my questions about people like Weinar. When a couple of people I knew mentioned that article, I pointed them to what I think are better criticisms here in philosophy tube (also more fun)
https://youtu.be/Lm0vHQYKI-Y?si=sw_3u-9tQSvRZRut
I am encouraged though that a few people have written responses to Thorstad over the last couple of weeks. Not exactly blowing up on Twitter but some good engagement at least ;)
Call me obsessed, but : Street Epistemology, Deep Canvassing, Smart Politics. Those are ways to talk to people who are hostile and yet turn the conversation into an object-level disagreement, and you can get fast training, for free (see my post “Effectively Handling Disagreements” -conflict of interest explicit). Notoriously relied upon by constructive atheists, by the way.
I do agree, however, that polite critiques should be boosted by default, e.g. with a karma boost on the forum, and doing this might be more effective than what I just stated above.
This is overthinking things. EA is full of quokkas, quokkas attract predators, predators tend to be volatile and very mentally unstable. This pretty much perfectly describes why Torres and Gebru do what they do. In Torres’s case it’s not even the first time he’s latched onto a philosophical movement only do later flip out and decide all its adherents were evil. He has some variant of borderline personality disorder, as it very obvious from his drunken tweets blaming his ex girlfriend for all his problems.
I don’t think armchair diagnosis of the alleged psychiatric disorders of EA critics (or much of anyone, for that matter) is an appropriate activity for the Forum.
For what it’s worth, I endorse @Habryka’s old comment on this issue:
Yeah, I don’t want to make a claim that reference to an individual’s mental condition would be categorically inappropriate. However, I think at a minimum there needs to be a reason for making the assertion that furthers an important interest, that the assertion is tailored to that interest, and that there isn’t a clear yet less inflammatory & invasive way to get the information across.
I think there are few cases in which this test would be met as applied to a critic. Saying that the critic has a long history of dishonest, volatile, paranoid, or whatever kind of behavior (and showing the receipts where appropriate) is more convincing to explaining why people shouldn’t engage than a thinly-supported armchair diagnosis.
While I don’t agree with everything in that quote, I do see some points of convergence—there is awareness of downsides, consideration of what is potentially to be gained from the discussion, a suggestion that this should not occur without significant object-level engagement first, and some sense of narrow tailoring (insofar as discussion of “psychologizing explanations and intuitions” is more narrowly tailored than trotting out a stigmatized DSM/ICD diagnosis).
This is quokka logic. With Torres in particular it’s an incredibly obvious motivation for why he does what he does. If this were more widely known, he would not get nearly the amount of press attention that he does. Instead people like this get to pose in the press as sane and sober critics because they can put together barely-coherent critiques and journalists don’t know the backstory. See https://markfuentes1.substack.com/p/emile-p-torress-history-of-dishonesty, which everyone should be signal boosting aggressively on a regular basis.
As an aside, Torres uses they/them pronouns. Could you correct your comments?
He is a biological male who displays the kind of sustained malignant aggression that is vastly more common from males than females. The male pronoun is both correct and usefully informative!
We’re issuing SuperDuperForecasting a one-month ban for breaking strong Forum norms in several comments (1,2). Specifically:
Intentionally misgendering someone (see more about how a moderator thinks about this here).
Engaging in unnecessary rudeness and offensiveness.
To be clear, I’m not commenting on the more complex questions discussed elsewhere in the thread, such as whether or when it’s appropriate to speculate about someone’s psychology. But I do want to flag that mental illness is often stigmatised and we should probably be especially sensitive and compassionate when discussing it.
As a reminder, the ban affects the user, not the account. During their ban period, the user will not be permitted to rejoin the Forum under another account name. If they return to the Forum, we’ll expect a higher standard of norm-following.
You can reach out to forum-moderation@effectivealtruism.org with any questions. You can appeal the decision here.
I find it notable that this announcement would be at −1 if not for my strong upvote. The suspended user doubled-down on misgendering Torres after being asked to correct it. Are people complaining about the strong norm against intentional misgendering, or is the downvoting reflective of some sort of belief in a “Torres exception” to that norm?
And I can’t believe it needs saying, but a “Torres exception” is not a good idea here. Even completely disregarding Torres’ own feelings there are a lot of people who are not Emile Torres which those lines of attack stigmatise.
Also when, the fundamental complaint about someone is that they repeatedly make uncharitable and probably false claims about people’s true motivations and engage in odd personal attacks on people they might legitimately be unimpressed by, adding a drive-by pop-diagnosis of a mental health condition and a nasty observation on their gender identity doesn’t strengthen that observation, it just sets off the irony meter...
It seems reasonably clear that there are certain psychiatric disorders such that people would be justified in refusing to engage with, or dismiss the claims of, those who suffer from them. I think the epistemically sound norm would be to ask those who argue that someone suffers from such a disorder to provide adequate evidence for the allegation.
The armchair diagnosis doesn’t add anything to the behavior offered in support of it. If someone has a history of deceptive behavior, extreme emotional instability, seemingly delusional behavior, whatever, then that is the potential reason to disengage. What’s the marginal benefit here to justify the various harms of armchair diagnosis?
That seems like a fully general counterargument against relying on medical diagnoses for anything. There are always facts that confirm a diagnosis, and then the diagnosis itself. Presumably, it is often helpful to argue that the facts confirm the diagnosis instead of simply listing the facts alone. I don’t see any principled reason for eschewing diagnoses when they are being used to support the conclusion that someone’s testimony or arguments should be distrusted.
“Ok this guy is actually right about Torres, but I still have to pontificate about the non-existent harms of armchair diagnosis rather than just admit it”
(what harms? This particular individual isn’t going to get any worse because I gave him an entirely accurate diagnosis, and while psychiatric diagnoses are invariably more fuzzy clusters than precise categories, there’s absolutely no reason not to engage in justified pattern-matching when you have sufficient evidence, which we really do in this case. No one goes around saying “Will MacAskill has a PhD and has written multiple well-received books”, we just say “Will MacAskill is a smart guy”, even though “smart” is also a pretty fuzzy category!)
Being smart isn’t stigmatized. Having borderline personality disorder definitely is.
There is a tendency in broader society to use armchair diagnosis to demean and belittle people with whom one disagrees. That increases stigma for people who do have mental health conditions.
A person’s medical conditions are private, so speculating as to what conditions they may have is ordinarily an invasion of that privacy.
I have no idea what your professional qualifications are, but differential diagnosis of mental disorders is actually not easy. As potentially relevant here, the differential includes particularly bipolar spectrum disorders, but also major depression, schizophrenia, attention-deficit/hyperactivity disorder, and posttraumatic stress disorder.
There’s even a rule of medical ethics for psychiatrists that they—who are arguably qualified to be handing out armchair diagnoses—not to do so. While I don’t always agree with that rule where the opinion is offered to voters considering the mental health of a candidate for president (which was the original context), I think it quite sound here.
If one of the ways a person is acting unusually is holding grudges against people they once thought highly of (or against movements they were formerly a part of), I’d also consider NPD and pathological narcissism for the differential diagnosis (the latter has a vulnerable subtype that has some overlap with BPD but is separate construct). I’m adding this to underscore your point that a specific diagnosis is difficult without a lot of context.
I also agree with not wanting to add to the stigma against people with personality disorders. A stigma means some commonly held association that is either wrong or unfairly negative. I think the risk with talking about diagnoses instead of specific symptoms is that this can unfairly harm the reputation of other people with the same diagnosis. BPD in particular has 9 symptom criteria, of which people have to only meet 5 in order to be diagnosed. So, you can have two people with BPD who share 1 symptom out of 9.
Another way in which talk about personality disorders can be stigmatizing is if the implication or connotation is something like “this person is irredeemable.” To avoid this connotation (if we were to armchair-diagnose people at all), I would add caveats like “untreated” or “and they seem to lack insight.” Treatment success for BPD without comorbid narcissism is actually high, and for NPD it’s more difficult but I wouldn’t completely give up hope.
Edit: Overall, I should say that I still agree with the comments that sometimes it can make sense to highlight that a person’s destructive behavior makes up a pattern and is more unusual than what you see in conflicts between people without personality disorders. However, I don’t know if it is ever necessary for forum users to make confident claims about what specific type of cluster b personality disorder (or other, related condition) someone may have. More generally, for the reasons I mentioned in the discussion around stigma, I would prefer if this subject was handled with more care than SuperDuperForecasting was giving it. I overall didn’t downvote their initial comment because I think something in the vicinity of what they said is an important hypothesis to put out there, but SuperDuperForecasting is IMO hurting their own cause/camp in the way they were talking about it.
I must admit I did not have time to re-read your post carefully, but thought it worth pointing out that after reading it I am left a bit confused by the multiple “culture wars” references. Could you please expand on this a bit?
I guess my confusion is that “culture wars” seem to be “attention grabbing” words you used in the beginning of your post, but I feel that after reading the full post that they were not fully addressed. I would be keen to understand if you only meant these to be rhetorical devices to make the reading more captivating, or if you have opinions on the frequent “white boys” criticisms of EA? It is fine if it is the former, I just felt a bit like I was left hanging after reading the post which I think otherwise did some good analysis on financial motives for criticism, comparing AI to e.g Climate Change.
I think others might be interested in this topic as well, especially as JEID concerns was raised by many EAs, and especially women and non-binary EAs. I also think some EAs might think that the “white boys”/culture wars criticisms of EA is actually criticism we should take seriously, although the tone in which these criticisms are made are often not the most optimal for engaging in fruitful dialogue (but I can understand if people with bad experiences can find it hard to suppress their anger—and perhaps sometimes anger is appropriate).