advocacy for detente or dialogue in any form with China may risk being branded as a Chinese sympathizer.
I’m skeptical.
Trump praises President Xi, says he looks forward to ‘getting along with China’
Trump invited Xi to his inauguration. He did not invite Ursula von der Leyen.
Trump wants U.S., China and Russia to cut military spending in half
I think we’re in a surprisingly good position to negotiate an AI treaty.
this section appears to be currently in charge of the US government
What section do you put Marco Rubio in?
These people cannot be reasoned out of their positions
Not sure hostile tweets are compelling evidence here. Social media participation seems very self-selected. Generally I think hostile tweets represent tails of a bell curve. (Also, most of the replies I’m reading don’t seem overly hostile.)
Search for “Life Effects” in Scott’s survey results. Many readers say they donate more to charity as a result of his blog. His reasonable approach appears to work.
Even if we grant that some people are unpersuadable—If your goal is for Republican poll numbers to go down as a result of the USAID situation, the best advocacy strategy isn’t obvious. For all we know, fire-and-brimstone rhetoric will polarize more people against foreign aid.
Vegans have taken a super-aggressive approach to advocacy for years. My sense is that it works on about 5% of the population, and turns off the remaining 95%.
Progressive activists only make up 8% of the population in the US (they’re just very vocal): https://hiddentribes.us/
“You’re anti-human because you bought yourself a birthday cake instead of donating to malaria nets!” Even EAs don’t use this kind of language with each other. If it doesn’t work for us, why would we expect it to work on the general population?
Edit 1/29: Funding is back, baby!
A recent article in The Economist (archive) claims PEPFAR funding hasn’t actually been restored. It seems like the “restore funding to PEPFAR” lever is sorta no longer attached to anything :-(
I don’t think avoiding criticising political figures because they’re sensitive, powerful and have some public support is a way forward either.
This sounds right to me too. In general your points seem fair. Although I will note that skepticism towards foreign aid (as practiced by e.g. Givewell charity evaluators) seems rather compatible with EA in principle.
BTW, for a few reasons, I think it’s better, by default, to criticize actions rather than individuals. (For example, I suspect that makes it easier psychologically for the individuals who committed those actions to change their behavior.) So I feel good about posts like this one criticizing PEPFAR cuts.
Most of this seems unrelated to my observation that evidence of weariness toward harshly criticizing people merely for their political beliefs does not provide much evidence of weariness for criticizing extremely powerful political people for their actions.
I think that’s a valuable distinction, and important puzzle piece here. However, OP doesn’t just talk about actions: “That these kind of attitudes can be found and not swiftly and roundly condemned by moderators...” (emphasis mine)
I’m thinking aloud in these comments, trying to put my finger on what bothered me (and, I suspect, would bother other people—especially conservatives) about the OP, and where to draw the line.
Yes, I know what due process is. In a strict sense, it applies only to governmental actions. In a looser sense, it applies to actions by private organizations that are considering taking adverse action against people associated with it—e.g., a private university which is considering disciplining a student, a homeowner’s association which is considering fining a unitowner. I am not aware of its extension to citizens (or groups of citizens) who wish to criticize a senior government official, or to a social movement that wishes to express its disapproval of a prominent person and distance themselves from said person.
The title of OP is: The EA movement needs to be able to disown rogue “supporters”, starting with Musk. So we’re considering collective adverse action against a person who’s been associated with us.
The key question appears to be the nature of EA. You seem to agree with me in the sense that if we were a “private association”, due process would be appropriate. And I agree with you that if we were just a random group of citizens, due process would feel less necessary.
The problem is that we’re something in between. You could call us a “community”—a term that’s also used to describe a university or a homeowner’s assocation, which you acknowledge as venues where due process is appropriate.
You could also call us a “movement”, but EA is not a political movement. It’s a movement based on using reason and evidence to do the most good. In my view, hasty “disowning [of] rogue supporters” without due process would undermine core tenets related to reason and evidence.
The private interest of Elon Musk in not being criticized by EAs is exceedingly modest.
Note that we’re not just talking about Musk here. OP wants us to clearly ‘pick a side’. So the private interest of Musk supporters within EA sound implicated. Are they going to be fired? Deprived of funding? Forced to stay quiet? What would be sufficient to satisfy OP?
I think the expected value of additional safeguards is low—he has made numerous troubling, divorced-from-reality, or erratic public statements and I am unaware of any dispute about whether the statements are his.
“Erroneous deprivation” seems far more likely to apply if you’re incorrect about the nature of the statements, not the fact that Musk said them? This attitude is part of what I’m trying to push back against—the implicit assumption that you don’t need to worry about your own errors in judging the veracity of someone’s statements. This is part of the mistake Musk is allegedly making, so let’s not make it ourselves.
BTW, there seems to be a bit of a motte-and-bailey thing going on here, because you were previously emphasizing a focus on denouncing actions over denouncing beliefs, but now you’re talking about “public statements”. Denouncing someone for public statements feels more like denouncing them for their beliefs than denouncing them for their actions.
Finally, with the volume of problematic statements, and new ones coming out on a near-daily basis, the burden of adjudicating them all would be considerable for a relatively small social movement.
In my comment, I suggested you just focus on one or two particularly bad and particularly clear Musk actions. Again, you’re a lawyer—if someone has committed numerous crimes, it should be easy to convict them for just one of them. Imagine a prosecutor who said: “It’s too burdensome to prosecute Joe because he’s committed so many darn crimes. So due process shouldn’t apply. Just toss him in prison.” Can you see how I would find that reasoning suspicious?
Relatedly, one of the reasons we can hold the government to the dictates of due process is that we concurrently give it the powers needed to carry out its functions while providing more protections. EA cannot subpoena Musk to judge him, or force him to sit for depositions on pain of imprisonment. Nor can it seek funds from the public fisc for investigations.
Agreed. But I’m not asking for nearly as much due process from EA as I would ask for from the government, either. Maybe that wasn’t clear. I’m asking for “CEA community health team”-level due process, as opposed to “US federal court”-level due process. It’s OK if the process is improvised and imperfect. I just want there to be some sort of procedure, or at least a gesture towards a procedure, which can itself be critiqued and improved over time.
Inserting a reminder in case it wasn’t already clear: I want to emphasize that my discussion is focused on collectively “disowning” Musk, not criticizing him in an individual capacity.
Some background on my thinking here is that I see modern society as suffering from a shortage of due process in general. Public shaming is a punishment which has, at times, been regulated. Social media has made public shaming far more accessible and severe as a vigilante action, while also greatly decreasing its implicit due process requirements (like “who would actually publish this?”) I think this is a major “root cause” of political dysfunction in the US. Same way hunter-gatherer tribes would feud with each other in the absence of due process to adjudicate crimes, we’re suffering from politically-tribal feuds due to vigilante public shaming actions. The legal tradition in the US has hundreds of years of history behind it, but I’m suggesting we think more in terms of building up from the absolute basics of a society that currently exists in a state of reputational anarchy—starting with lightweight, rough-and-ready approaches such as Community Notes. This is an area where the US legal tradition can serve as a source of inspiration, but it also can’t automatically be expected to apply.
US political dysfunction is implicated in multiple EA cause areas, so if my hypothesis is correct, this sort of institution-building could be a high-impact topic to think about. EA’s Community Health Team has had some success addressing the sort of reputation warring which has destroyed other online communities and social movements. But we didn’t catch SBF. I feel more small-scale prototyping is appropriate before eventually trying to fix problems at the nation scale.
The isolated demand for “due process” rigor
I think I would likely write similar comments in response to any post calling for the EA movement to disown someone. This isn’t the first time I’ve called for greater institutional capacity along these lines.
...is somewhat ironic given Musk’s habit of shooting from the hip with criticisms about others, including federal employees when he is a public official, employment reform is within the scope of his apparent job functions, and people are suffering concrete harm (loss of jobs and contracts) as a result of his official conduct on the topic. Further irony comes from the fact that he owns a major social media platform that is infamous for people offering uninformed hot takes. These factors do not affect how we should treat him, but the irony is palpable.
Some criminal vigilantes claim they’re doing the right thing, and fail to offer their victims due process, yet the state offers these vigilantes due process anyways. I don’t think this is particularly ironic. It seems like a fairly common situation. Due process is why we generally consider the government to be “the good guys”, and vigilantes to be “the bad guys”.
If you’re living in anarchy, someone has to be the first to start doing due process, if you want to put an end to retribution-spiraling.
Where was USAID mentioned in the PDF you linked?
people who are likely to be remotely supportive of EA
FYI, this post I linked was written by someone who also has an EA Forum account. (The post I linked is not exactly supportive of Musk, but it doesn’t exactly align with the OP either.)
I suspect many of the conservatives sympathetic to some of the things EA wants to do are the ones that think he has too much power and is taking the wrong approach...
Perhaps, but I doubt they consider it obvious. They might read a detailed explainer meant to pass their ITT, but I suspect they would consider OP in its current form to be “woke cancel culture”.
Geoffrey Miller is one of a small number of conservative EAs who posts about the importance of AI alignment. He seems neutral/positive on Musk:
So the topic of discussion here is what will annoy people in Silicon Valley and people in the Trump administration. My mental model is the key annoying thing is when the ratio of social enforcement to scholarship / rational argument gets too high. “Pearl-clutching” hashtag activism feels performative and fundamentally un-serious. If an activist truly believed something was a serious and important matter, they would inform themselves deeply about it, understand the thinking of those who disagree, and set out to persuade others based on detailed factual arguments.
This is the intuition I get from observing e.g. JD Vance (relevant because he’s both from SV, and in the Trump admin): https://xcancel.com/JDVance/with_replies
If you read the OP, it doesn’t even really attempt to defend the thesis that Elon Musk is a bad EA. The OP wants to condemn, and maybe even condemn everyone who hasn’t condemned, before there has even been a trial. You’re a lawyer. Presumably you understand the importance of due process. Why wouldn’t it be called for in the court of public opinion?
If this is important, do it right, with scholarship and steelmanning and all of that. (If you want to save time, try to focus on one or two very clearly bad things he did.) If it’s not important, forget about it and move on to the next thing.
Yougov estimates that in the US, Musk is more popular than Joe Biden, Hillary Clinton, Mark Zuckerberg, or Bill Gates:
https://today.yougov.com/ratings/politics/fame/public-figures/all
the educated, intelligent population that is EA’s primary recruitment ground
One could argue that EA should be focused on influencing people leading the US government and people working on AI in Silicon Valley.
For better or for worse, I think both of those groups are very fed up with the practice of denouncing people for their political beliefs. (And both of those groups also include Elon Musk as an important member!)
I feel good about EAs writing detailed explainers meant to pass ideological turing tests and persuade people of specific important claims.
I feel bad about about EAs pressuring others to denounce people who seem obviously bad based on what they’re seeing on social media. (You can find many views on social media about pretty much any topic.)
As an American, I think this is actually a really good point. The only reason to even consider racing with China to build AI is if we believe the outcome is better if America “wins”. Seems worth keeping an eye on the political situation in the US, e.g. for stuff like PEPFAR cancellation, to check if the US “winning” is actually better from a humanitarian perspective. Hopefully lab leaders are smart enough to realize this.
This arrests EAs/longtermists from drawing on centuries of knowledge & movement-building
The track record of anticapitalist advocacy seems quite poor. See this free book: Socialism: The failed idea that never dies
If you’re doing anticapitalist advocacy for EA reasons, I think you need a really clear understanding of why such advocacy has caused so much misery in the past, and how your advocacy will avoid those traps.
I’d say what’s needed is not anticapitalist advocacy, so much as small-scale prototyping of alternative economic systems that have strong theoretical arguments for how they will align incentives better, and scale way past Dunbar’s number.
You don’t need a full replacement for capitalism to test ideas and see results. For example, central planning often fails due to corruption. A well-designed alternative system will probably need a solution for corruption. And such a solution could be usefully applied to an ordinary capitalist democracy.
I concede that AI companies are behaving in a harmful way, but I doubt that anticapitalist advocacy is a particularly tractable way to address that, at least in the short term.
In that case your strategy is just feeding the labs talent and poisoning the ability of their circles to oppose them.
It seems like your model only has such influence going one way. The lab worker will influence their friends, but not the other way around. I think two-way influence is a more accurate model.
Another option is to ask your friends to monitor you so you don’t get ideologically captured, and hold an intervention if it seems appropriate.
I see no meaningful path to impact on safety working as an AI lab researcher
This is a very strong statement. I’m not following technical alignment research that closely, but my general sense is that exciting work is being done. I just wrote this comment advertising a line of research which strikes me as particularly promising.
I noticed the other day that the people who are particularly grim about AI alignment also don’t seem to be engaging much with contemporary technical alignment research. That missing intersection seems suspicious. I’m interested in any counterexamples that come to mind.
My subjective sense is there’s a good chance we lose because all the necessary insights to build aligned AI were lying around, they just didn’t get sufficiently developed or implemented. This seems especially true for techniques like gradient routing which would need to be baked in to a big, expensive training run.
(I’m also interested in arguments for why unlearning won’t work. I’ve thought about this a fair amount, and it seems to me that sufficiently good unlearning kind of just oneshots AI safety, as elaborated in the comment I linked.)
With regard to Deepseek, it seems to me that the success of mixture-of-experts could be considered an update towards methods like gradient routing. If you could localize specific kinds of knowledge to specific experts in a reliable way, you could dynamically toggle off / ablate experts with unnecessary dangerous knowledge. E.g. toggle off experts knowledgeable in human psychology so the AI doesn’t manipulate you.
I like this approach because if you get it working well, it’s a general tool that could help address a lot of different catastrophe stories in a way that seems pretty robust. E.g. to mitigate a malicious AI from gaining root access to its datacenter, ablate knowledge of the OS it’s running on. To mitigate sophisticated cooperation between AIs that are supposed to be monitoring one another, ablate knowledge of game theory. Etc. (The broader point is that unlearning seems very generally useful. But the “Expand-Route-Ablate” style approach from the gradient routing paper strikes me as a particularly promising, and could harmonize well with MoE.)
I think a good research goal would be to try to eventually replicate Deepseek’s work, except with highly interpretable experts. The idea is to produce a “high-assurance” model which can be ablated so undesired behaviors, like deception, are virtually impossible to jailbreak out of it (since the weights that perform the behavior are inaccessible). I think the gradient routing paper is a good start. To achieve sufficient safety we’ll need new methods that are more robust and easier to deploy, which should probably be prototyped on toy problems first.
it’s possible Trump is better placed to negotiate a binding treaty with China (similar to the idea that ‘only Nixon could go to China’), even if it’s not clear he’ll want to do so.
Some related points:
Trump seems like someone who likes winning, but hates losing, and also likes making deals. If Deepseek increases the probability that the US will lose, that makes it more attractive to negotiate an end to the race. This seems true from both a Trump-psychology perspective, and a rational-game-theorist perspective.
Elon Musk seems to have a good relationship with Chinese leadership.
Releasing open-source AI seems like more of a way to prevent someone else from winning than a way to win yourself.
I suppose one possible approach would be to try to get some sort of back-channel dialogue going, to start drafting a treaty which can be invoked if political momentum appears.
That’s a good point about patient privacy. On X, you suggested that PEPFAR has had a sizable macro impact on AIDS in Africa. Maybe Africans who are old enough to remember could talk about what AIDS was like in Africa before and after PEPFAR, as a way to illustrate that macro impact without violating the privacy of any individual patients.
Of course, insofar as individual patients are willing to speak about their experiences, possibly with some light anonymization such as looking away from the camera, that seems really good to me too.
A couple other thoughts:
If access to ARV drugs is interrupted, I imagine that could lead to drug-resistant HIV strains, same way your doctor tells you to take your antibiotics consistently.
Having a big population of immunocompromised people in Africa seems bad in terms of mutation speed for future pandemics. Here’s a paper I just found on Google, about COVID variants which are thought to have arisen from immunocompromised patients.
I wonder if the response will be seen as more credible if it’s driven by Africans?
Insofar as there’s a motivation behind cutting foreign aid, I suspect that skepticism of the NGO sector is playing a role. I can imagine Trump supporters thinking: This program supposedly helps millions of poor Africans, yet the primary voices advocating for it online are rich-world progressives. Seems fishy.
I’ve received a bunch of Whatsapp messages from Ugandan friends who are very very worried about what this might mean not only for patients (the main issue), but also for jobs and the livelihood of many local NGOS.
I think it could be pretty effective for your Ugandan friends to record videos interviewing people who have been helped by the program, and post the videos online.
A lot of skepticism of foreign aid is driven by the fear that the aid is being captured somewhere along the way, either here in the US or in the target country. Hearing directly from the aid recipients helps address that fear.
As an American, it feels like good things the US does are taken for granted, and the US will be criticized relentlessly by people in other countries no matter what we do. So I wouldn’t suggest scolding the US for withdrawing funds. And in any case, rewards usually work better than punishments for modifying behavior. I think saying thanks, and talking about what it would mean for the program to continue, would be a lot more powerful than scolding. Even for Americans who aren’t particularly altruistic, this would provide tangible evidence that the program is improving America’s reputation.
One mental model I have is that the US is suffering from something akin to EA burnout, and the solution to both “US burnout” and “EA burnout” is a stronger culture of gratitude. In principle, I could imagine that a really good response to this PEPFAR situation actually ends up motivating the US to fund additional effective aid programs.
I sold all my NVIDIA stock, since their moat looks weak to me:
I think your reasoning is generally correct. Another argument: If you believe things look sufficiently grim under short timelines, maybe you should invest under the assumption that a recession, or something else, will pop the AI bubble and gives us longer timelines.
Someone here on the Forum wrote instructions for how to write a letter to state AGs and ask that they scrutinize the restructuring:
https://forum.effectivealtruism.org/posts/6P8TXrn43wib7PHLH/tyler-johnston-s-shortform?commentId=Jhv6eEPbH6jyTYpN4