Mechinterp researcher under Adrià Garriga-Alonso.
Thomas Kwa
Karma: 3,120
EA forum content might be declining in quality. Here are some possible mechanisms:
Newer EAs have worse takes on average, because the current processes of recruitment and outreach produce a worse distribution than the old ones
Newer EAs are too junior to have good takes yet. It’s just that the growth rate has increased so there’s a higher proportion of them.
People who have better thoughts get hired at EA orgs and are too busy to post. There is anticorrelation between the amount of time people have to post on EA Forum and the quality of person.
Controversial content, rather than good content, gets the most engagement.
Although we want more object-level discussion, everyone can weigh in on meta/community stuff, whereas they only know about their own cause areas. Therefore community content, especially shallow criticism, gets upvoted more. There could be a similar effect for posts by well-known EA figures.
Contests like the criticism contest decrease average quality, because the type of person who would enter a contest to win money on average has worse takes than the type of person who has genuine deep criticism. There were 232 posts for the criticism contest, and 158 for the Cause Exploration Prizes, which combined is more top-level posts than the entire forum in any month except August 2022.
EA Forum is turning into a place primarily optimized for people to feel welcome and talk about EA, rather than impact.
All of this is exacerbated as the most careful and rational thinkers flee somewhere else, expecting that they won’t get good quality engagement on EA Forum
- EA forum content might be declining in quality. Here are some possible mechanisms. by (24 Sep 2022 22:24 UTC; 130 points)
- 's comment on Agree/disagree voting (& other new features September 2022) by (17 Sep 2022 9:18 UTC; 18 points)
- 's comment on EA Forum feature suggestion thread by (24 Sep 2022 21:31 UTC; 11 points)
Crossposted from LessWrong.
Maybe I’m being cynical, but I’d give >30% that funders have declined to fund AI Safety Camp in its current form for some good reason. Has anyone written the case against? I know that AISC used to be good by talking to various colleagues, but I have no particular reason to believe in its current quality.
MATS has steadily increased in quality over the past two years, and is now more prestigious than AISC. We also have Astra, and people who go directly to residencies at OpenAI, Anthropic, etc. One should expect that AISC doesn’t attract the best talent.
If so, AISC might not make efficient use of mentor / PI time, which is a key goal of MATS and one of the reasons it’s been successful.
Why does the founder, Remmelt Ellen, keep linkposting writing by Forrest Landry which I’m 90% sure is obvious crankery? It’s not just my opinion; Paul Christiano said “the entire scientific community would probably consider this writing to be crankery”, one post was so obviously flawed it gets −46 karma, and generally the community response has been extremely negative. Some AISC work is directly about the content in question. This seems like a concern especially given the philosophical/conceptual focus of AISC projects, and the historical difficulty in choosing useful AI alignment directions without empirical grounding. [Edit: To clarify, this is not meant to be a character attack. I am concerned that Remmelt does not have the skill of distinguishing crankery from good research, even if he has substantially contributed to AISC’s success in the past.]
All but 2 of the papers listed on Manifund as coming from AISC projects are from 2021 or earlier. Because I’m interested in the current quality in the presence of competing programs, I looked at the two from 2022 or later: this in a second-tier journal and this in a NeurIPS workshop, with no top conference papers. I count 52 participants in the last AISC so this seems like a pretty poor rate, especially given that 2022 and 2023 cohorts (#7 and #8) could both have published by now. (though see this reply from Linda on why most of AISC’s impact is from upskilling)
The impact assessment was commissioned by AISC, not independent. They also use the number of AI alignment researchers created as an important metric. But impact is heavy-tailed, so the better metric is value of total research produced. Because there seems to be little direct research, to estimate the impact we should count the research that AISC alums from the last two years go on to produce. Unfortunately I don’t have time to do this.
The main assumption of this post seems to be that, not only are the true values of the parameters independent, but a given person’s estimates of stages are independent. This is a judgment call I’m weakly against.
Suppose you put equal weight on the opinions of Aida and Bjorn. Aida gives 10% for each of the 6 stages, and Bjorn gives 99%, so that Aida has an overall x-risk probability of 10^-6 and Bjorn has around 94%.
If you just take the arithmetic mean between their overall estimates, it’s like saying “we might be in worlds where Aida is correct, or worlds where Bjorn is correct”
But if you take the geometric mean or decompose into stages, as in this post, it’s like saying “we’re probably in a world where each of the bits of evidence Aida and Bjorn have towards each proposition are independently 50% likely to be valid, so Aida and Bjorn are each more correct about 2-4 stages”.
These give you vastly different results, 47% vs 0.4%. Which one is right? I think there are two related arguments to be made against the geometric mean, although they don’t push me all the way towards using the arithmetic mean:
Aida and Bjorn’s wildly divergent estimates on probably come from some underlying difference in their models of the world, not as independent draws. In this case where Aida is more optimistic about Bjorn on each of the 6 stages, it is unlikely that this is due to independent draws. I think this kind of multidimensional difference in optimism between alignment researchers is actually happening, so any model should take this into account.
If we learn that Bjorn was wrong about stage 1, then we should put less weight on his estimates for stages 2-6. (My guess is there’s some copula that corresponds to a theoretically sensible way to update away from Bjorn’s position treating his opinions as partially correlated, but I don’t know enough statistics)
[Warning: long comment] Thanks for the pushback. I think converting to lives is good in other cases, especially if it’s (a) useful for judging effectiveness, and (b) not used as a misleading rhetorical device [1].
The basic point I want to make is that all interventions have to pencil out. When donating, we are trying to maximize the good we create, not decide which superficially sounds better between the different strategies “empower beneficiaries to invest in their communities’ infrastructure” and “use RCTs to choose lifesaving interventions” [2]. Lives are at stake, and I don’t think those lives are less important simply because it’s harder to put names and faces to the ~60 lives that were saved from a 0.04% chance of reduction of malaria deaths from a malaria net. Of course this applies equally to the Wytham Abbey purchase or anything else. But to point (a), we actually can compare the welfare gain from 61 lives saved to the economic security produced by this project. GiveWell has weights for doubling of consumption, partly based on interviews from Africans [3]. With other projects, this might be intractable due to entirely different cause areas or different moral preferences e.g. longtermism.
Imagine that we have a cost-effectiveness analysis made by a person with knowledge of local conditions and local moral preferences, domain expertise in East African agricultural markets, and the quantitative expertise of GiveWell analysts. If it comes out that one intervention is 5 or 10 times better than the other, as is very common, we need a very compelling reason why some consideration was missed to justify funding the other one. Compare this to our currently almost complete state of ignorance as to the value of building this plant, and you see the value of numbers. We might not get a CEA this good, but we should get close as we have all the pieces.
As to point (b), I am largely pro making these comparisons in most cases just to remind people of the value of our resources. But I feel like the Wytham and HPMOR cases, depending on phrasing, could exploit peoples’ tendency to think of projects that save lives in emotionally salient ways as better than projects that save lives via less direct methods. It will always sound bad to say that intervention A is funded rather than saving X lives, and we should generally not shut down discussion of A by creating indignation. This kind of misleading rhetoric is not at all my intention; we all understand that allowing a large enough number of farmers access to sorghum markets can produce more welfare than preventing 61 deaths from malaria. We have the choice between saving 61 of someones’ sons and daughters, and allowing X extremely poor people to perhaps buy metal roofs, send their children to school, and generally have some chance of escaping a millennia-long poverty trap. We should think: “I really want to know how large X is”.
[1] and maybe (c) not bad for your mental health?
[2] Unless you believe empowering people is inherently better regardless of the relative cost, which I strongly disagree with.
[3] This is important—Westerners may be biased here because we place different values on life compared to doubling consumption. But these interviews were from Kenya and Ghana, so maybe Uganda’s weights slightly differ.
I think funding is a bottleneck. Everything I’ve heard suggests the funding environment is really tight: CAIS is not hiring due to lack of funding. FAR is only hiring one RE in the next few months due to lack of funding. Less than half of this round of MATS scholars were funded for independent research. I think this is because there are not really 5-10 EA funders able to fund at large scale, just OP and SFF; OP is spending less than they were pre-FTX. At LTFF the bar is high, LTFF’s future is uncertain, and they tend not to make huge grants anyway. So securing funding should be a priority for anyone trying to start an org.
Edit: I now think the impact of these orgs is uncertain enough that one should not conclude with certainty there is a funding bottleneck.
I think it’s valuable to point out a problem. The fact is that the majority of media articles about EA are negative (and often inaccurate), and this has been the case for years. Inasmuch as this is a problem, all existing efforts to solve it have failed! Listing upcoming efforts seems like more of a nice addition than a mandatory component.
Thanks for writing this.
I thought about why I buy the AI risk arguments despite the low base rate, and I think the reason touches on some pretty important and nontrivial concepts.
When most people encounter a complicated argument like the ones for working on AI risk, they are in a state of epistemic learned helplessness: that is, they have heard many convincing arguments of a similar form be wrong, or many convincing arguments for both sides. The fact that an argument sounds convincing fails to be much evidence that it’s true.
Epistemic learned helplessness is often good, because in real life arguments are tricky and people are taken in by false arguments. But when taken to an extreme, it becomes overly modest epistemology: the idea that you shouldn’t trust your models or reasoning just because other people whose beliefs are similar on the surface level disagree. Modest epistemology would lead you to believe that there’s a 1⁄3 chance you’re currently asleep, or that the correct religion is 31.1% likely to be Christianity, 24.9% to be Islam, and 15.2% to be Hinduism.
I think that EA does have something in common with religious fundamentalists: an orientation away from modest epistemology and towards taking weird ideas seriously. (I think the number of senior EAs who used to be religious fundamentalists or take other weird ideas seriously is well above the base rate.) So why do I think I’m justified in spending my career either doing AI safety research or field-building? Because I think the community has better epistemic processes than average.
Whether it’s through calibration, smarter people, people thinking for longer or more carefully, or more encouragement of skepticism, you have to have a thinking process that results in truth more often than average, if you want to reject modest epistemology and still believe true things. From the inside, the EA/rationalist subcommunity working on AI risk is clearly better than most millenarians (you should be well-calibrated about this claim, but you can’t just say “but what about from the outside?”—that’s modest epistemology). If I think about something for long enough, talk about it with my colleagues, post it on the EA forum, invite red-teaming, and so on, I expect to reach the correct conclusion eventually, or at least decide that the argument is too tricky and remain unsure (rather than end up being irreversibly convinced of the wrong conclusion). I’m very worried about this ceasing to be the case.
Taking weird ideas seriously is crucial for our impact: I think of there being a train to crazy town which multiplies our impact by >2x at every successive stop, has increasingly weird ideas at every stop, and at some point the weird ideas cease to be correct. Thus, good epistemics are also crucial for our impact.
Sure, here’s the ELI12:
Suppose that there are two billionaires, April and Autumn. Originally they were funding AMF because they thought working on AI alignment would be 0.01% likely to work and solving alignment would be as good as saving 10 billion lives, which is an expected value of 1 million lives, lower than you could get by funding AMF.
After being in the EA community a while they switched to funding alignment research for different reasons.
April updated upwards on tractability. She thinks research on AI alignment is 10% likely to work, and solving alignment is as good as saving 10 billion lives.
Autumn now buys longtermist moral arguments. Autumn thinks research on AI alignment is 0.01% likely to work, and solving alignment is as good as saving 10 trillion lives.
Both of them assign the same expected utility to alignment-- 1 billion lives. As such they will make the same decisions. So even though April made an epistemic update and Autumn a moral update, we cannot distinguish them from behavior alone.
This extends to a general principle: actions are driven by a combination of your values and subjective probabilities, and any given action is consistent with many different combinations of utility function and probability distribution.
As a second example, suppose Bart is an investor who makes risk-averse decisions (say, invests in bonds rather than stocks). He might do this for two reasons:
He would get a lot of disutility from losing money (maybe it’s his retirement fund)
He irrationally believes the probability of losing money is higher than it actually is (maybe he is biased because he grew up during a financial crash).
These different combinations of probability and utility inform the same risk-averse behavior. In fact, probability and utility are so interchangeable that professional traders—just about the most calibrated, rational people with regard to probability of losing money, and who are only risk-averse for reason (1) -- often model financial products as if losing money is more likely than it actually is, because it makes the math easier.
I’m worried about EA values being wrong because EAs are unrepresentative of humanity and reasoning from first principles is likely to go wrong somewhere. But naively deferring to “conventional” human values seems worse, for a variety of reasons:
There is no single “conventional morality”, and it seems very difficult to compile a list of what every human culture thinks of as good, and not obvious how one would form a “weighted average” between these.
most people don’t think about morality much, so their beliefs are likely to contradict known empirical facts (e.g. cost of saving lives in the developing world) or be absurd (placing higher moral weight on beings that are physically closer to you).
Human cultures have gone through millennia of cultural evolution, such that values of existing people are skewed to be adaptive, leading to greed, tribalism, etc.; Ian Morris says “each age gets the thought it needs”.
However, these problems all seem surmountable with a lot of effort. The idea is a team of EA anthropologists who would look at existing knowledge about what different cultures value (possibly doing additional research) and work with philosophers to cross-reference between these while fixing inconsistencies and removing values that seem to have an “unfair” competitive edge in the battle between ideas (whatever that means!).
The potential payoff seems huge, as it would expand the basis of EA moral reasoning from the intuitions of a tiny fraction of humanity to that of thousands of human cultures, and allow us to be more confident about our actions. Is there a reason this isn’t being done? Is it just too expensive?
Epistemic status: I am a university student who has read a lot of EA material but has little knowledge about B1G1 programs. I thought carefully about this post for a few hours.
I think there’s a wide spectrum of possible effectiveness depending on implementation, but in practice they seem unlikely to be much more effective than the average non-EA charity, and a factor of at least 10 behind many EA causes.
Overall, the strictest forms of B1G1, where a company gives the exact same product they’re selling, seems gimmicky to me. The reason is that needs of people in the developing world are vastly different from those of the wealthy people buying the products. I think market forces might even dictate that these programs are not much more effective than direct cash transfers: If they were much more effective, the target population would be willing to buy them, which would cannibalize the sales of the company. [1] None of the 3 companies you list is so naive—they mostly outsource their work to charities. But this comes with its own problems: they don’t apply their own domain knowledge to their interventions.
Warby Parker works with Pupils Project and VisionSpring. Pupils Project operates in the US, so it’s unlikely they are cost-effective. VisionSpring at least works in Bangladesh. According to a [GiveWell interview][2], they do undercut commercial prices by a factor of 2 by selling glasses at cost for 150 taka ($1.77) [3], but I doubt that glasses are a leveraged intervention in the developing world. GiveWell does not currently recommend VisionSpring as a top or standout charity, instead recommending charities that can beat cash by a factor of 5-60 and are supported by very strong evidence.
TOMS has stopped distributing shoes in favor of donating 1⁄3 of their profits to a fund managed by their giving team. Their 2019 impact report is basically a marketing document full of infographics; it appears they make some attempt at evaluating impact of charities, but don’t follow effective altruist principles. For example, they fund projects in the US, and clean water programs (The Gates foundation has studied the water, sanitation, and hygiene sector extensively and finds better opportunities in sanitation).
P&G’s MNT vaccine program is through UNICEF, which is massively overfunded by comparison to charities recommended by GW and the Open Philanthropy Project.
There are more fundamental problems. The B1G1 website says they primarily evaluate causes by “progress of the project activity” and financial records; it’s likely they’re falling for the overhead myth and vastly underemphasizing the effectiveness of the cause area, which is left up to the company. EA has at least three branches where effective cause areas are found: global health/poverty, farm/wild animal welfare, and existential risk. It would be ideal if companies’ B1G1 programs either supported effective programs in one of these areas, or found a unique niche. B1G1 programs need to yield good PR, and sometimes have the additional constraint of providing a tangible product, so it appears they’re limited to a small subset of global health interventions, which in these three examples look no better than the average charity in terms of effectiveness. I don’t see any companies with B1G1 programs in farm or wild animal welfare, probably because it is politically contentious. Existential risk causes seem even less likely to yield good PR because they’re the exact opposite of the tangible transaction at the heart of B1G1. And B1G1 seems unlikely to let companies find a unique niche given that they’re outsourcing to nonprofits.
Finally, I have other concerns. B1G1 companies could be decreasing the amount given to more effective charities, which given that some charities are hundreds or thousands of times more effective than others, might cause net harm. They also might be using such programs to cover up being socially irresponsible (e.g. poor treatment of factory workers, or contributing to high-suffering animal agriculture).
Since this comment is rather long, I’ve split it into two, with the second comment directly answering the 12 questions.
[1]: See https://www.givewell.org/international/charities/income-raising-goods for why. Other GiveWell charities manage to outperform cash because they don’t sell commodities—individual families can’t buy a school deworming program.
[2]: https://files.givewell.org/files/conversations/VisionSpring_05-17-19_(public).pdf
[3]: Strangely, they sell glasses for $0.85 each on their website. Perhaps they have high distribution costs.
I think someone should do an investigation much wider in scope than what happened at FTX, covering the entire causal chain from SBF first talking to EAs at MIT to the damage done to EA. Here are some questions I’m particularly curious about:
Did SBF show signs of dishonesty early on at MIT? If so, why did he not have a negative reputation among the EAs there?
To what extent did EA “create SBF”—influence the values of SBF and others at FTX? Could a version of EA that placed more emphasis on integrity, diminishing returns to altruistic donations, or something else have prevented FTX?
Alameda was started by various traders from Jane Street, especially EAs. Did they do this despite concerns about how the company would be run, and were they correct to leave at the time?
[edited to add] I have heard that Tara Mac Aulay and others left Alameda in 2018. Mac Aulay claims this was “in part due to concerns over risk management and business ethics”. Do they get a bunch of points for this? Why did this warning not spread, and can we even spread such warnings without overloading the community with gossip even more than it is?
Were Alameda/FTX ever highly profitable controlling for the price of crypto? (edit: this is not obvious; it could be that FTX’s market share was due to artificially tight spreads due to money-losing trades from Alameda). How should we update on the overall competence of companies with lots of EAs?
SBF believed in linear returns to altruistic donations (I think he said this on the 80k podcast), unlike most EAs. Did this cause him to take on undue risk, or would fraud have happened if FTX had a view on altruistic returns similar to that of OP or SFF but linear moral views?
What is the cause of the exceptionally poor media perception of EA after FTX? When i search for “effective altruism news”, around 90% of articles I could find negative and none positive, including many with extremely negative opinions unrelated to FTX. One would expect at least some article saying “Here’s why donating to effective causes is still good”. (In no way do I want to diminish the harms done to customers whose money was gambled away, but it seems prudent to investigate the harms to EA per se)
My guess is that this hasn’t been done simply because it’s a lot of work (perhaps 100 interviews and one person-year of work), no one thinks it’s their job, and conducting such an investigation would somewhat entail someone both speaking for the entire EA movement and criticizing powerful people and organizations.
See also: Ryan Carey’s comment
As a non-vegan, here’s how I think about this:
I basically buy the arguments that the relative value of being vegan is small compared to my career (the strongest counterargument for me is that being vegan improves moral clarity)
Being vegan is really inconvenient for me for nutritional reasons, so I just avoid chicken and some eggs, the most suffering-dense foods. This is kind of an arbitrary policy but it does have ~0 cost and get me partial moral clarity + some sense of the moral clarity I’m missing.
I think I would be at least vegetarian if I had a visceral disgust response to eating meat, like if I were raised vegetarian. But that doesn’t mean I endorse it! Giving myself a disgust response now would be net bad for my impact, and I think I’m consequentialist enough that this is most of what I care about. (edit: and I’d also remove a disgust response if I already had one)
Realistically, I might eat the humans in this thought experiment, if this were as widely accepted as eating pigs and I’d been raised with the custom. But maybe I’d have a strong disgust response anyway, or maybe my current meta-policy would avoid human meat if I thought it were a very morality-dulling food. If it were more suffering-dense than chicken, or perceived as a high-suffering delicacy like foie gras or shark fin, eating humans regularly would be more morality-dulling because it would reinforce my identity as an immoral person or something.
If I had a disgust response to eating humans, this doesn’t mean I’d endorse it either! “Is human meat suffering-dense?” is different from “Does the idea of eating human meat produces a strong disgust response?” is different from “is eating humans morality-dulling?”, and the last one is what drives my impact.
This thought experiment makes me update slightly towards eating meat being morality-dulling but probably not enough to change my diet.
My comment is not an ad hominem. An ad hominem attack would be if someone is arguing point X and you distract from X by attacking their character. I was questioning only Remmelt’s ability to distinguish good research from crankery, which is directly relevant to the job of an AISC organizer, especially because some AISC streams are about the work in question by Forrest Landry. I apologize if I was unintentionally making some broader character attack. Whether it’s obnoxious is up to you to judge.
I want to slightly push back against this post in two ways:
I do not think longtermism is any sort of higher form of care or empathy. Many longtermist EAs are motivated by empathy, but they are also driven by a desire for philosophical consistency, beneficentrism and scope-sensitivity that is uncommon among the general public. Many are also not motivated by empathy—I think empathy plays some role for me but is not the primary motivator? Cold utilitarianism is more important but not the primary motivator either [1]. I feel much more caring when I cook dinner for my friends than when I do CS research, and it is only because I internalize scope sensitivity more than >99% of people that I can turn empathy into any motivation whatsoever to work on longtermist projects. I think that for most longtermists, it is not more empathy, nor a better form of empathy, but the interaction of many normal (often non-empathy) altruistic motivators and other personality traits that makes them longtermists.
Longtermists make tradeoffs between other common values and helping vast future populations that most people disagree with, and without ideosyncratic EA values there is no reason that a caring person should make the same tradeoffs as longtermists. I think the EA value of “doing a lot more good matters a lot more” is really important, but it is still trading off against other values.
Helping people closer to you / in your community: many people think this has inherent value
Beneficentrism: most people think there is inherent value in being directly involved in helping people. Habitat for Humanity is extremely popular among caring and empathic people, and they would mostly not think it is better to make more of an overall difference by e.g. subsidizing eyeglasses in Bangladesh.
Justice: most people think it is more important to help one human trafficking victim than one tuberculosis victim or one victim of omnicidal AI if you create the same welfare, because they place inherent value on justice. Both longtermists and GiveWell think they’re similarly good modulo secondary consequences and decision theory.
Discount rate, risk aversion, etc.: There is no reason that having a 10% chance of saving 100 lives in 6,000 years is better than a 40% chance of saving 5 lives tomorrow, if you don’t already believe in zero-discount expected value as the metric to optimize. The reason to believe in zero-discount expected value is a thought experiment involving the veil of ignorance, or maybe the VNM theorem. It is not caring doing the work here because both can be very caring acts, it is your belief in the thought experiment connecting your caring to the expected value.
In conclusion, I think that while care and empathy can be an important motivator to longtermists, and it is valid for us to think of longtermist actions as the ultimate act of care, we are motivated by a conjunction of empathy/care and other attributes, and it is the other attributes that are by far more important. For someone who has empathy/care and values beneficentrism and scope-sensitivity, preventing an extinction-level pandemic is an important act of care; for someone like me or a utilitarian, pandemic prevention is also an important act. But for someone who values justice more, applying more care does not make them prioritize pandemic prevention over helping a sex trafficking victim, and in the larger altruistically-inclined population, I think a greater focus on care and empathy conflict with longtermist values more than they contribute.
[1] More important for me are: feeling moral obligation to make others’ lives better rather than worse, wanting to do my best when it matters, wanting future glory and social status for producing so much utility.
Clickbait title. It should be something like “Universal Ownership, internalizing externalities, and applications to big cause areas”
Thanks for this comment, I made minor edits to that point clarifying that academia can be good or bad.
First off, I think we should separate concerns of truth from those of offputtingness, and be clear about which is which. With that said, I think “concession to other selfish or altruistic goals” is true to the best of my knowledge. Here’s a version of it that I think about, which is still true but probably less offputting, and could have been substituted for that bullet point if I were more careful and less concise:
When your goal is to maximize impact, but parts of you want things other than maximizing impact, you must either remove these parts or make some concession to satisfy them. Usually stamping out a part of yourself is impossible or dangerous, so making some concession is better. Some of these concessions are cheap (from an impact perspective), like donating 2% of your time to a cause you have personal connection to rather than the most impactful one. Some are expensive in that they remove multipliers and lose >50% of your impact, like changing your career from researching AI safety to working at Netflix because your software engineer friends think AI safety is weird. Which multipliers are cheap vs expensive depends on your situation; living in a particular location can be free if you’re a remote researcher for Rethink Priorities but expensive if by far the best career opportunity for you is to work in a particular biosecurity lab. I want to caution people against making an unnecessarily expensive concession, or making a cheap concession much more expensive than necessary. Sometimes this means taking resources away from your non-EA goals, but it does not mean totally ignoring them.
Regarding having a child, I’m not an expert or a parent, but my impression is it’s rare for having kids to actually create more impact than not having the desire in the first place. I vaguely remember Julia Wise having children due to some combination of (a) non-EA goals, and (b) not having kids would make her sad, potentially reducing productivity. In this case, the impact-maximizer would say that (a) is fine/unavoidable—not everyone is totally dedicated to impact—and (b) means that being sad is a more costly concession than not having kids, so having kids is the least costly concession available. Maybe for some, having kids makes life meaningful and gives them something to fight for in the world, which would increase their impact. But I haven’t met any such people.
It’s possible to have non-impact goals that actually increase your impact. Some examples are being truth-seeking, increasing your status in the EA community, or not wanting to let down your EA friends/colleagues. But I have two concerns with putting too much emphasis on this. First, optimizing too hard for this other goal has Goodheart concerns: there are selfish rationalists, EAs who add to an echo chamber, and people who stay on projects that aren’t maximally impactful. Second, the idea that we can directly optimize for impact is a core EA intuition, and focusing on noncentral cases of other goals increasing impact might distract from this. I think it’s better to realize that most of us are not pure impact-maximizers, we must make concessions to other goals, and that which concessions we make is extremely important to our impact.
Having done a few doom circles I think they’re much better when
there are <6 people in the circle, so everyone can feel attended to
I have high trust in everyone, which might come from knowing them well or having been vulnerable in other ways
everyone is in the right mood for doom circles; vague insecurities or pressure create the wrong mood
It’s 2027, and Redwood has failed to be useful while spending hundreds of person-years of researcher time. What happened?
I think all these groups need to be concerned, but about different things:
a largely white, educated, western, and male group talking about how to fix everything should (i) not repeat mistakes like racist eugenics, colonialism, etc., historically made by such groups, (ii) also think about other possible problems caused by its members being privileged/powerful
an uneducated group should not repeat mistakes made by past such groups (perhaps famines and atrocities caused by the Great Leap Forward), and anticipate other problems caused by its demographics by ensuring it has good epistemics, talent, and ways of being taken seriously
a largely black group should look at mistakes made by other predominantly black groups including black supremacists (perhaps becoming cults), …
a group talking about how it’s impossible to fix anything should look at mistakes made by past such groups (perhaps Calvinism), …
Also, any group that isn’t demographically diverse might want to become diverse if they think adding voices makes the direction of the movement better.
Some of these concerns can be easily dismissed (the NAACP doesn’t need to try especially hard to not become a black supremacist cult because the prior probability of that is very low). But when thinking about plausible failure modes even a ~3:1 Bayes factor from demographics can be important, until we know about the actual causes of these failures and whether they apply to us.
There’s value in giving the average person a broadly positive impression of EA, and I agree with some of the suggested actions. However, I think some of them risk being applause lights—it’s easy to say we need to be less elitist, etc., but I think the easy changes you can make sometimes don’t address fundamental difficulties, and making sweeping changes have hidden costs when you think about what they actually mean.
This is separate from any concern about whether it’s better for EA to be a large or small movement.
Edit: big tent actually means “encompassing a broad spectrum of views”, not “big movement”. I now think this section has some relevance to the OP but does not centrally address the above point.
As I understand it, this means spending more resources on people who are “less elite” and less committed to maximizing their impact. Some of these people will go on to make career changes and have lots of impact, but it seems clear that their average impact will be lower. Right now, EA has limited community-building capacity, so the opportunity cost is huge. If we allocate more resources to “big tent” efforts, it would mean less field-building at top-20 universities (Cambridge AGISF), less highly scalable top-funnel (80,000 Hours), less workshops for people who are committed to career changes and get huge speedups from workshops.
One could still make a neglectedness case for big-tent efforts, but the cost-benefit calculation definitely can’t be summed up in one line.
I’m uncomfortable doing too much celebrating of actions that are much lower impact than other actions (e.g. donating blood), from both an honesty/transparency perspective and a consequentialist perspective. From a consequentialist perspective, we should probably celebrate actions that create a lot of expected impact in order to encourage people to take those actions. So the relevant question is whether donating blood makes one closer to having a very high-impact career. I think the answer is often no: it often doesn’t practice careful scope-sensitive thinking, or bring high-impact actions into one’s action space.
From a transparency perspective, celebration disproportionate to the good done also feels kind of fake. In the extreme, we’re basically distorting our impressions of people’s actions to get people to join a movement. I’m not saying we should shun people for taking a suboptimal action, but we should be transparent about the fact that (a) some altruistic actions aren’t very good and don’t deserve celebration, and (b) some actions are good but only because they’re on the path to an impactful career.
Communication is hard. There’s a tradeoff between fidelity, brevity, scale, and speed (time spent writing/editing/talking to distill 1 idea):
Long one-on-ones get very high fidelity, low brevity, low scale, and high speed
80k podcasts are high fidelity, low brevity, high scale, and low speed
A tabling pitch is low fidelity, high brevity, moderate scale, and moderate speed
A short, polished EA forum post is moderate fidelity, high brevity, high scale, and very low speed. If you’re not a gifted writer it takes multiple editing cycles to create a really high-quality post. Usually this includes copy-editing, sending the Google Doc draft to friends, having discussions in the comments, maybe adding visuals.
If we max out fidelity and brevity, we have to have lower scale and/or speed. I think this is okay if we’re targeting communication, but it doesn’t play well with the big-tent approach where we also need high scale. One could say we should just get closer to the Pareto frontier, but I think everyone is already trying to do this.
I don’t strongly disagree with this—it’s bad to put off people unnecessarily—but I think it can easily be taken too far.
I’m worried that people will avoid looking dogmatic by adding unwarranted uncertainty about what actions are best, and in particular being unwilling to reject popular ideas. I think the best remedy to looking dogmatic is actually having good, legible epistemics, not avoiding coming across as dogmatic by adding false uncertainty. (This is related to the post “PR is corrosive; “reputation” is not.) When someone asks whether volunteering in an animal shelter is high-impact, we should give well-reasoned arguments that there are probably higher-value things to do under almost every scope-sensitive moral view (perhaps starting from first principles if they’re new), not avoid looking dogmatic by telling them something largely false like “Some people might find higher impact at an animal shelter because they have comparative advantage / are much more motivated, and there could also be unknown unknowns that place really high value on the work at animal shelters”. It’s impossible to spend 1% of our resources on every idea with as much true merit as volunteering at animal shelters because there are more than 100 such ideas, so we only would because of bias towards popular things. But when we require a well-reasoned case using the ITN framework to allocate 1% of our effort to a problem, and therefore refuse to spend 1% of our effort on animal shelters, plastic bag bans, or the NYC homelessness problem, we will come off as dogmatic to some people. OP addresses the need to protect our epistemics at the end, but I think doesn’t stress this enough.
There are also many crucial EA things that sound or are elitist.
More resources are focused on top universities than community colleges (because talent is concentrated there and this ultimately helps the most sentient beings).
Over 80% of EA funding is from billionaires.
People are flown across the world to retreats (because this is often the most efficient way to network or learn, and we think their time can do more good than spending the money on anything else).
We are looking for people who produce 1000x the impact as others (because they have more multipliers available).
We shouldn’t be exclusionary for no reason when talking to new people. But based on community-building at two universities, ~10 retreats/EAGs, much of the reason EA looks elitist is not because we’re exclusionary for no reason, it’s because EAs do important things that look elitist.
Maybe the most elitist-sounding practices should even be slightly reduced for PR reasons. But going further to reduce the appearance of elitism would hamstring EA by taking away some of the most valuable direct and meta interventions.