Pronouns: she/âher or they/âthem.
I got interested in effective altruism back before it was called effective altruism, back before Giving What We Can had a website. Later on, I got involved in my university EA group and helped run it for a few years. Now Iâm trying to figure out where effective altruism can fit into my life these days and what it means to me.
Yarrowđ¸
Karma: 877
Have you tried Discord? Discord seems absurdly casual for any kind of business or serious use, but thatâs more about Discordâs aesthetics, brand, and reputation than its actual functionality.
My impression when Discord came out was that it copied Slack pretty directly. But Slack was a product for teams at companies to talk to each other and Discord was a tool to make it easier for friends or online communities to play video games together.
Slack is still designed for businesses and Discord is still designed primarily for gamers. But Discord has been adopted by many other types of people for many other purposes.
Discord has voice chat and makes it super easy to switch between servers. Back when people were using Slack as a meeting place for online communities (whereas today they all use Discord), one of my frustrations was switching between teams, as you described.
I think Discord is functionally much better than Slack for many use cases, but asking people to use Discord in a business context or a serious context feels absurd, like holding a company meeting over Xbox Live. If you can get over using a gaming app with a cartoon mascot, then it might be the best solution.
I played it the other way around, where I asked o4-mini to come up with a word that I would try to guess. I tried this twice and it made the same mistake both times.
The first word was âbutterflyâ. I guessed âBâ and it said, âThe letter B is not in the word.â
Then, when I lost the game and o4-mini revealed the word, it said, âApologiesâI mis-evaluated your B guess earlier.â
The second time around, I tried to help it by saying: âMake a plan for how you would play hangman with me. Lay out the steps in your mind but donât tell me anything. Tell me when youâre ready to play.â
It made the same mistake again. I guessed the letters A, E, I, O, U, and Y, and it told me none of the letters were in the word. That exhausted the number of wrong guesses I was allowed, so it ended the game and revealed the word was âschmaltzinessâ.
This time, it didnât catch its own mistake right away. I prompted it to review the context window and check for mistakes. At that point, it said that A, E, and I are actually in the word.[1]
Related to this: François Chollet has a great talk from August 2024, which I posted here, that includes a section on some of the weird, goofy mistakes that LLMs make.
He argues that when a new mistake or category of mistake is discovered and becomes widely known, LLM companies fine-tune their models to avoid these mistakes in the future. But if you change up the prompt a bit, you can still elicit the same kind of mistake.
So, the fine-tuning may give the impression that LLMsâ overall reasoning ability is improving, but really this is a patchwork approach that canât possibly scale to cover the space of all human reasoning, which is impossibly vast and can only be mastered through better generalization.
- ^
I edited my comment to add this footnote on 2025-05-03 at 16:33 UTC. I just checked and o4-mini got the details on this completely wrong. It said:
But the final word SCHMALTZINESS actually contains an A (in position 5), an I (in positions 10 and 13), and two Eâs (in positions 11 and 14).
What it said about the A is correct. It said that one letter, I, was in two positions, and neither of the positions it gave contain an I. It said there are two Es, but there is only E. It gets the position of the E right, but says there is a second E in position 14, which doesnât exist.
- ^
Good Lord! Thanks for this information!
The Twitter thread by Toby Ord is great. Thanks for linking that. This tweet helps put things in perspective:
For reference, these are simple puzzles that my 10-year-old child can solve in about 4 minutes.
Thank you!
When you say people in EA have failed at the Circle of Hell Test, what do you mean? Do you literally mean in that specific situation of people standing and talking in a circle and someone trying to join the circle?
Kudos on a well-written, well-researched, and well-argued post!
The part about âAI researchersâ (which donât actually exist; there is no such thing as an âAI researcherâ) vs. human researchers gets at a simple mistake, which is confusing inputs and outputs.For example, I believe that a GPU running a large language model (LLM) uses a lot more energy than a human brain.[1] Is this evidence that LLMs are smarter than humans?
No, of course not.
If anything, people have interpreted this as a weakness of AI. Maybe it means the way current AI systems solve problems is too âbrute forceâ and weâre missing something fundamental about the nature of intelligence.
Saying that AI systems are using more and more compute over time doesnât directly say anything about their actual intelligence, any more than saying that AI systems are using more and more energy over time does.
How intelligent are AI systems compared to humans? How much has that been changing over time? How do we measure that? These are the key questions and, as you pointed out in your section on benchmarks, a lot of AI benchmarks donât seem like they really measure this.
One potential way to measure the intelligence of AI vs. humans is the extent to which AI can displace human labour. You nicely covered this on your section on real-world adoption.
One study I can contribute to the discussion is this one I found about a customer support centre adopting LLMs. The outcome was mixed, with the study finding that LLMs harmed the productivity of the centreâs most experienced employees.
I think this post does a great job of pointing out the ways in which the arguments for near-term artificial general intelligence (AGI) rely on hand-wavy reasoning and make simple, fundamental mistakes â mistakes so simple and so fundamental that Iâm confused why people like Will MacAskill donât catch these mistakes themselves, since surely they should be able to clearly see these mistakes themselves and donât need people like you or I to point them out.- ^
Hereâs one estimate I found after a quick Google, but I donât know if itâs accurate.
- ^
I donât understand what youâre trying to say here. By âthe trendâ, do you mean Nvidiaâs revenue growth? And what do you mean by âhave diverted normal compute expenditures into the AI boomâ?
I know very little about the context of the disagreements between Oliver Habryka and Dustin Moskovitz. Iâve read one of their backs-and-forths in comments on the EA Forum and it was almost impossible to follow what they were talking about, partly due to both of their writing styles, but also probably due to there being a lot of context and background they werenât trying to explain to people like me who werenât already in the know.
I think Dustin may also have purposely been trying to be a bit vague because he was sick of being criticized by people on the EA Forum and felt that the more he said, the more he would be criticized (he made a comment to that effect).
So, I really donât know all the details here and could be getting this all wrong. This is just my impression of things knowing as little as I do right now.
One of things Oliver has done a lot in his comments on the EA Forum which has bothered me is to try to shift a debate about what the right thing to do is on a specific topic (e.g., should EA buy a castle, should EA-related organizations invite people with extreme racist views to its conferences) into questioning the motives of people who disagree with him, accusing them of being too concerned with reputation rather than doing the right thing. Oliver seems to think he prioritizes doing the right thing over having a good reputation, but other people do it the other way around.
For example, Oliver holds views and is willing to take actions that I would categorize as racist and that I find morally objectionable for that reason. Iâm not nearly the first person to express this. But Oliverâs response is not âsome people disagree with me because they have different opinions about racismâ, itâs more like âpeople pretend to disagree with me because theyâre scared about what people will think and arenât willing to speak the truthâ. (Just to be clear, these are not real quotes. Iâm just paraphrasing what I understood from reading some of Oliverâs comments.)
Itâs a lot less compelling, rhetorically, to say âme and Dustin disagree about what constitutes racismâ than to say âDustin is overly concerned about his personal reputation (and Iâm not)â. (Again, these are not real quotes.) But itâs also dishonest and mean-spirited.
I think part of the reason some of the discussions about racism in EA get diverted into discussions about EAâs reputation is that people are trying to leave a quick comment without getting dragged into an interminable and stressful debate about racism. LessWrong users have an inexhaustible capacity for getting into protracted, technical, and verbose forum debates. In general, people are averse to getting into debates about politics, race and racism, and social justice online. Itâs tempting to try to get around a 100,000-word debate on the definition of racism by saying âthese kinds of words and actions will alienate many people from effective altruism and worsen our reputationâ.
Maybe that kind of response makes it seem like reputation is the primary concern. But itâs not the primary concern. The primary concern is that racism is evil and the racist words and actions of Oliver, et al. are evil. And you donât want Oliver to write a 5,000-word comment that cites Astral Codex Ten seven times and LessWrong fourteen times arguing that holding racist views is actually smart that youâre going to feel obligated to read and respond to. So, instead youâll just say âthis kind of thing is really off-putting to many people, and damaging to our communityâ. And yet Oliver still found a way to respond to this that is about equally as annoying as the thing you were hoping to avoid. He says, âAha! You care about reputation! I care about truth!â (Again, just to be clear, this is a fake quote.)
Let me repeat the caveat that I get the sense that thereâs a whole lot of context and background to Dustin and Oliverâs disagreements that I donât understand and Iâm giving my impression of their disagreements despite this limited understanding. So, I could be getting Dustinâs perspective wrong and I could be getting Oliverâs perspective wrong.
But, with this limited understanding, my interpretation is that Dustin thinks that Lightcone Infrastructureâs and the rationalist communityâs views and actions are racist and immoral and doesnât want to be morally responsible for funding or supporting racism, either directly or indirectly. That, I think, is his primary reason for cutting ties with Lightcone and the rationalist community, not reputation. Reputation is one thing heâs considered, but itâs not the only thing and I donât think itâs the primary thing. The primary thing is that racism is evil.
Wow. This is my first time reading this post.
The last section â âProblems with the EA community/âmovementâ â was surprising. I was surprised that people in leadership positions at EA or EA-related organizations seemed to feel dismayed and irritated by the EA Forum and the online EA community for many of the same reasons I do.
I guess I feel relieved that they also see these problems, but, on the other hand, this survey is from 2019. I wonder how many respondents since left EA because these things put them off too much â or other things, like the sexual harassment, the racism, etc. If they stayed in EA, I wonder why I donât get a sense of any leaders in the EA community/âmovement trying to address these problems.
We should promote AI safety ideas more than other EA ideas
AGI is probably a long time away. No one knows when AGI will be created. No one knows how to create AGI. AGI safety is such a vague, theoretical concept that thereâs essentially nothing you can do about it today or in the near future.
A comment from François Chollet on this topic posted on Bluesky on January 6, 2025:
I donât think people really appreciate how simple ARC-AGI-1 was, and what solving it really means.
It was designed as the simplest, most basic assessment of fluid intelligence possible. Failure to pass signifies a near-total inability to adapt or problem-solve in unfamiliar situations.
Passing it means your system exhibits non-zero fluid intelligenceâyouâre finally looking at something that isnât pure memorized skill. But it says rather little about how intelligent your system is, or how close to human intelligence it is.
o3 gets 3% on ARC-AGI-2.
How much advance notice would be appropriate in an ordinary case?
I donât have a strong opinion on this, but I put my icon where I imagined 2 weeks would be. This is just an off-the-cuff stab at what a good rule of thumb might be.
More than 2 weeks feels like an onerous amount of time to wait to publish something.
2 weeks also seems like a reasonable amount of time for an organization to draft at least a short response. I donât think we should expect organizations to write a detailed, comprehensive response to every piece of criticism they receive â either immediately or ever. (How much of a response feels warranted depends on how harsh the criticism is and how convincing it comes across.)
But 2 weeks is plenty of time to write a short reply of a few sentences or a few paragraphs, which can do a lot to defuse criticism if itâs convincing enough. For example, if you can point out a specific, provable error in the criticism that is actually important to the case itâs making (i.e., not just nitpicking). That might be enough to defuse the criticism as much as you care to defuse it, or it might be enough to convince people to withhold judgment while you take time to write a longer response.
But as I said, this is just my attempt to come up with a good rule of thumb, and, as with the other question, the real answer is âit dependsâ.
Giving meaningful advance notice of a post that is critical of an EA person or organization should be
I put my answer at the midway point between neutral on the question and 100% agreeing with âalmost always doneâ because the answer is âit dependsâ. It depends, for example, on how much money the organization being criticized has, how much criticism it is already getting, and how harsh your criticism is.
ďOpenAIâs o3 model scores 3% on the ARC-AGI-2 benchÂmark, comÂpared to 60% for the avÂerÂage human
âTruthseekingâ is a strange piece of jargon. Iâm not sure what purpose it serves. It seems like the meaning of âtruthseekingâ ambiguates between âpracticing good epistemologyâ and âbeing intellectually honestâ, as you describe. So, why not use one of those terms instead?
One thing that annoys me about the EA Forum (which I previously wrote about here) is that thereâs way too much EA Forum-specific jargon. One negative effect of this is it makes it harder to understand what people are trying to say. Another negative effect is it elevates a lot of interesting conjecture to the level of conventional wisdom. If you have some interesting idea in a blog post or a forum post, and then people are quick to incorporate that into the lingo, youâve made that idea part of the culture, part of the conventional wisdom. And it seems like people do this too easily.
If you see someone using the term âtruthseekingâ on the EA Forum, then:
-
There is no clear definition of this term anywhere that you can easily Google or search on the forum. There is a vague definition on the Effective Altruism Australia website. There is no entry for âtruthseekingâ in the EA Forum Wiki. The Wikipedia page for truth-seeking says, âTruth-seeking processes allow societies to examine and come to grips with past crimes and atrocities and prevent their future repetition. Truth-seeking often occurs in societies emerging from a period of prolonged conflict or authoritarian rule.[1] The most famous example to date is the South African Truth and Reconciliation Commission, although many other examples also exist.â
-
To the extent EA Forum users even have a clear definition of this term in their heads, they may be bringing along their own quirky ideas about epistemology or intellectual honesty or whatever. And are those good ideas? Who knows? Probably some are and a lot arenât. Making âtruthseekingâ a fundamental value and then defining âtruthseekingâ in your own quirky way elevates something you read on an obscure blog last year to the level of an idea that has been scrutinized and debated by a diverse array of scholars across the world for decades and stood the test of time. Thatâs a really silly, bad way to decide which ideas are true and which are false (or dubious, or promising, or a mixed bag, or whatever).
-
Chances are the person is using it passive-aggressively, or with the implication that theyâre more truthseeking than someone else. Iâve never seen someone say, âI wasnât being truthseeking enough and changed my approach.â This kinda makes it feel like the main purpose of the word is to be passive-aggressive and act superior.
So, is this jargon anything but a waste of time?
-
This post is about criticism of EA organizations, so it doesnât apply to OpenAI or the U.S. government.
I interpreted this post as mostly being about charities with a small number of employees and relatively small budgets that either actively associate themselves with EA or that fall into a cause area EA generally supports, such as animal welfare or global poverty.
For example, if you wanted to criticize 80,000 Hours, New Harvest, or one of these charities focusing on mental health in poor countries, then Iâd say you should send them a copy of your criticism before publishing and give them a chance to prepare a reply before you post. These organizations are fairly small in terms of their staff, have relatively little funding, and arenât very well-known. So, I think itâs fair to give them more of an opportunity to defend their work.
If you wanted to criticize Good Ventures, Open Philanthropy, GiveWell, GiveDirectly, or the Against Malaria Foundation, then I think you could send them a courtesy email if you wanted, but they have so much funding and â in the case of Open Philanthropy at least â a large staff. Theyâre also already the subject of media attention and public discourse. With one of the smaller charities, you could plausibly hurt them with your post, so I think more caution is warranted. With these larger charities with more resources that are already getting debated and criticized a lot, an EA Forum post has a much lower chance of doing accidental harm.
I think you behaved inappropriately, as I and others explained in the comments on that post about the dubious âfraudâ accusation. I completely understand why Sinergia said they donât want to engage with your criticism anymore.
Upvotes/âdownvotes are not a meaningless number in this context, but a sign of EA Forum usersâ opinion on whether you behaved appropriately and whether your claim that Sinergia committed fraud was true or misleading. You can see this in the comments on that post as well. It seems like there is, so far, unanimous agreement that you behaved inappropriately and that your claim was misleading or false.
Iâm not sure if Vetted Causes is a salvageable project at this point. Its reputation is badly damaged. It might be best to put the project to an end and move on to something else.
Speaking for myself, I will never trust any evaluation that Vetted Causes ever publishes about any charity, and I would feel an obligation to warn people in this community that Vetted Causes is an untrustworthy and unreliable source for charity evaluations.
Iâm not familiar with the context, but my comment might address this sort of situation.
Iâm guessing this is probably a response to the post that unfairly accused a charity of fraud? (The post Iâm thinking of currently has â60 karma, 0 agree votes, 6 disagree votes, and 4 top-level comments that are all critical.)
Some criticism might be friendly and constructive enough that giving the organization a chance to write a reply before publishing is not that important. Or if the organization is large, powerful, and has lots of money, like Open Philanthropy, and especially if your criticisms are of a more general or a more philosophical kind, it might not be important to send them a copy before you publish. This depends partly on how influential you are in EA and on how harsh your criticisms are.
Definitely accusing a small charity of fraud is something you should run by the charity beforehand. In that case, though, the charity was already so frustrated with the criticâs poor-quality criticism that they had publicly stated (before the fraud accusation) they didnât want to engage with it anymore.
This is the kind of idea that has a superficial sheen of plausibility but begins to look plainly absurd when you take time to reflect on the deep reasons our moral views are the way they are.
Here are a few reasons this argument doesnât make sense.
1. Itâs not an apples-to-apples comparison
The Nazis also didnât donate money to help with global poverty, so to make this comparison you have to count the deaths they could have averted by donating but didnât on top of the people they actively killed.
2. Even if we are focused just on outcomes, intent still matters
Intent is important to outcome.
If you give money to one of GiveWellâs top charities, like the Against Malaria Foundation, you are relying on the intent of the people who work there to produce a good outcome â more money leads to more people helped, such as more anti-malarial bednets deployed. If you somehow found out the people who work at a charity secretly have bad intent (like that they wanted to abscond with the money), you wouldnât donate because your prediction of the outcome would change. Your view that the charity is good would change.
Part of what made the Nazis bad was not just what they did â although that, of course, was among the worst things anyoneâs ever done, among the worst things imaginable â but also what they intended to do if they had won World War II and gained more power. The outcome would have been bad. One of the ways to assess someoneâs moral character is to ask what the outcome would be if they had a lot more money, power, or influence.
3. Actively killing is morally worse than passively letting die
Moral agency, moral responsibility, and moral luck are complex and vexing topics. But I will still say that actively killing someone, deliberately and violently, is morally worse than failing to save someoneâs life by spending money on personal consumption rather than donating to charity.
Even if you think of things in a pure, rigid consequentialist or utilitarian way, it makes sense to believe that directly, violently killing people is worse because of what I just said about the connection between intent and outcomes.
What kind of world would we live in if we elevated people prone to violently killing others to positions of wealth, power, and influence, rather than treating this as morally evil? The proclivity to actively kill people is strongly correlated with a failure to respond to global poverty humanely, and itâs also strongly correlated with everything else bad.
4. Altruistic self-sacrifice is part of moral assessment
A factor that seems important in assessing the morality of an action or the moral character of a person is altruistic self-sacrifice.
In thought experiments (what the philosopher Daniel Dennett would call âintuition pumpsâ) like Peter Singerâs parable of the drowning child,[1] the amount of sacrifice the person is required to make to save a life is typically stipulated to be minimal. In reality, the amount of personal sacrifice required to save a life is probably greater â more like 3,000 to 5,000 USD rather than the cost of ruined shoes or clothes in Singerâs hypothetical.
And these thought experiments donât directly address the question (donât directly pump our intuitions about) the morality of a personâs actions if they are faced with millions of drowning children and now face real trade-offs between living their own life normally and saving the children. In Singerâs parable, the dilemma is small because the sacrifice is small. Extrapolating from that parable to an argument or an intuition that larger sacrifices should also be required may or may not be justified, but we need to consider how you get from the drowning child parable to that conclusion.
But more importantly to the topic at hand, how much self-sacrifice a person should be willing to endure in order to altruistically help others and whether a person has sacrificed enough is a different topic entirely than what the Nazis did. The Nazis went out of their way to kill people.
This point, #4, builds on points #2 and #3. The willingness to engage in altruistic self-sacrifice matters even if weâre just focused on outcomes. Going out of your way to kill people seems much worse than failing to engage in âenoughâ altruistic self-sacrifice, and if someone insists on it, we can even justify that on consequentialist grounds.
Someone somewhere once came up with a clever variation of this parable where you are holding your phone and have to drop it on the ground, smashing it, to catch a child falling off a building.