GET AMBITIOUS SLOWLY
Most approaches to increasing agency and ambition focus on telling people to dream big and not be intimidated by large projects. I’m sure that works for some people, but it feels really flat for me, and I consider myself one of the lucky ones. The worst case scenario is big inspiring speeches get you really pumped up to Solve Big Problems but you lack the tools to meaningfully follow up.
Faced with big dreams but unclear ability to enact them, people have a few options.
try anyway and fail badly, probably too badly for it to even be an educational failure.
fake it, probably without knowing they’re doing so
learned helplessness, possible systemic depression
be heading towards failure, but too many people are counting on you so someone steps in and rescue you. They consider this net negative and prefer the world where you’d never started to the one where they had to rescue you.
discover more skills than they knew. feel great, accomplish great things, learn a lot.
The first three are all very costly, especially if you repeat the cycle a few times.
My preferred version is ambition snowball or “get ambitious slowly”. Pick something big enough to feel challenging but not much more, accomplish it, and then use the skills and confidence you learn to tackle a marginally bigger challenge. This takes longer than immediately going for the brass ring and succeeding on the first try, but I claim it is ultimately faster and has higher EV than repeated failures.
I claim EA’s emphasis on doing The Most Important Thing pushed people into premature ambition and everyone is poorer for it. Certainly I would have been better off hearing this 10 years ago
What size of challenge is the right size? I’ve thought about this a lot and don’t have a great answer. You can see how things feel in your gut, or compare to past projects. My few rules:
stick to problems where failure will at least be informative. If you can’t track reality well enough to know why a failure happened you definitely* need an easier project.
if your talk gives people a lot of ambitions to save the world/build billion dollar companies but their mind goes blank when they contemplate starting a freelancing business, the ambition is fake.
Hmm, I personally think “discover more skills than they knew. feel great, accomplish great things, learn a lot” applies a fair amount to my past experiences, and I think aiming too low was one of the biggest issues in my past, and I think EA culture is also messing up by discouraging aiming high, or something.
I think the main thing to avoid is something like “blind ambition”, where your plan involves multiple miracles and the details are all unclear. This seems also a fairly frequent phenomenon.
I think that you in particular might be quite non-representative of EAs in general, in terms of “success” in the EA context. If I imagine a distribution of “EA success,” you are probably very far to the right.
Accepting your self-report as a given, I have a bunch of questions.
I want to say that I’m not against ambition. From my perspective I’m encouraging more ambition, by focusing on things that might actually happen instead of daydreams.
Does the failure mode I’m describing (people spinning their wheels on fake ambition) make sense to you? Have you seen it?
I’m really surprised to hear you describe EA as discouraging aiming high. Everything I see encourages aiming high, and I see a bunch of side effects of aiming too high littered around me. Can you give some examples of what you’re worried about?
What do you think would have encouraged more of the right kind of ambition for you? Did it need to be “you can solve global warming?”, or would “could you aim 10x higher?” be enough?
Feeling a bit tired to type a more detailed response, but I think I mostly agree with what you say here.
This post is very popular on Twitter
I’m a bit confused about this because “getting ambitious slowly” seems like one of those things where you might not be able to successfully fool yourself: once you can conceive that your true goal is to cure cancer, you are already “ambitious”; unless you’re really good at fooling yourself, you will immediately view smaller goals as instrumental to the big one. It doesn’t work to say I’m going to get ambitious slowly.
What does work is focusing on achievable goals though! Like, I can say I want to cure cancer but then decide to focus on understanding metabolic pathways of the cell, or whatever. I think if you are saying that you need to focus on smaller stuff, then I am 100% in agreement.
Does what I said here and here answer this? the goal isn’t “put the breaks on internally motivated ambition”, it’s “if you want to get unambitious people to do bigger projects, you will achieve your goal faster if you start them with a snowball rather than try to skip them straight to Very Big Plans”.
I separately think we should be clearer on the distinction between goals (things you are actively working on, have a plan with concrete next steps and feedback loops, and could learn from failure) and dreams (things you vaguely aspire and maybe are working in the vicinity of, but no concrete plans). Dreams are good, but the proper handling of them is pretty different from that of goals.
I also liked this quote from Obama on a similar theme. The advice is pretty common for very good reasons but hearing it from former POTUS had more emotional strength on me:”how do we sustain our own sense of hope, drive, vision, and motivation? And how do we dream big? For me, at least, it was not a straight line. It wasn’t a steady progression. It was an evolution that took place over time as I tried to align what I believed most deeply with what I saw around me and with my own actions.
The first stage is just figuring out what you really believe. What’s really important to you, not what you pretend is important to you. And what are you willing to risk or sacrifice for it? The next phase is then you test that against the world, and the world kicks you in the teeth. It says, “You may think that this is important, but we’ve got other ideas. And who are you? You can’t change anything.”
Then you go through a phase of trying to develop skills, courage, and resilience. You try to fit your actions to the scale of whatever influence you have. I came to Chicago and I’m working on the South Side, trying to get a park cleaned up or trying to get a school improved. Sometimes I’m succeeding, a lot of times I’m failing. But over time, you start getting a little bit of confidence with some small victories. That then gives you the power to analyze and say, “Here’s what worked, here’s what didn’t. Here’s what I need more of in order to achieve the vision or the goals that I have.” Now, let me try to take it to the next level, which means then some more failure and some more frustration because you’re trying to expand the orbit of your impact.
I think it’s that iterative process. It’s not that you come up with a grand theory of “here’s how I’m going to change the world” and then suddenly it all just goes according to clockwork. At least not for me. For me, it was much more about trying to be the person I wanted to believe I was. And at each phase, challenging myself and testing myself against the world to see if, in fact, I could have an impact and make a difference. Over time, you’ll surprise yourself, and it turns out that you can.”
The problem with this advice is that many people in EA don’t think we have enough time to slowly build up. If you think EA might take control of the future within the next 15 years, you don’t have much time to build skills in the first half of your career and exercise power after you have 30 years of experience. There is an extreme sense of urgency, and I am not sure what’s the right response.
“we don’t have time” is only an argument for big gambles if they work. If ambition snowballs work better, then a lack of time is all the more reason not to waste time with vanity projects whose failures won’t even be educational.
I could steel man this as something of a lottery, where n% of people with way-too-big goals succeed and those successes are more valuable than the combined cost of the failures. I don’t think we’re in that world, because I think goals in the category I describe aren’t actually goals, they’re dreams, and by and large can’t succeed.
You could argue that’s defining myself into correctness and some big goals are genuinely goals even if they pattern match my criteria like “failure is uninformative” and “contemplating a smaller project is scary or their mind glances off the option (as opposed to being rejected for being too small)”. I think that’s very unlikely to be true for my exact critieria, but agree that in general overly broad definitions of fake ambition could do a lot of damage. I think creating a better definition people can use to evaluate their own goals/dreams is useful for that exact reason.
I also think that even if there are a few winning tickets in that lottery- people pushed into way-too-big projects that succeed- there aren’t enough of them to make a complete problem-solving ecosystem. The winning tickets still need staff officers to do the work they don’t have time for, or require skills inimical to swinging for the fences.
I should note that my target audience here is primarily “people attempting to engender ambition in others”, followed by “the people who are subject to those attempts”. I think engendering fake ambition is actively harmful, and the counterfactual isn’t “30 years in a suit”, it’s engendering ambition snowballs that lead to more real projects. I don’t think discouraging people who are naturally driven to do much-too-big projects is helpful.
I’d also speculate that if you tell a natural fence-swinger to start an ambition snowball, they end up at mind-bogglingly ambitious quickly, not necessarily slower than if you’d pushed them directly to dream big. Advice like “Do something that’s scary but at least 80% tractable” scales pretty well across natural ambition levels.
This is fantastic, and mirrors the method that has helped things work well in my own life.
I think that people should break down their goals, no matter how easy they seem, into easier and smaller steps, especially if they feel lazy. Laziness appears when we feel like we need to do tasks that seem unecessary for us, even when we know that they’re necessary. One reason why they appear unecessary is their difficulty of achievement. Why exercise for 30 minutes per day if things are “fine” without that? As such, one way to deal with that is by taking whatever goal you have and breaking it down into a lot of easy steps. As an example, imagine that you want to write the theoretical part of your thesis. So, you could start by writing what is the topic, what questions you might want to research, what key uncertainties you have about those questions, then you search for papers in order to clarify those uncertainties, and so on, immediate step by step, until you finish your thesis. If a step seems difficult, break it down even more. That’s why I think that breaking down your goals into smaller and easier steps might help when you feel lazy.
Anyways, thanks for your quick take!
There’s a thing in EA where encouraging someone to apply for a job or grant gets coded as “supportive”, maybe even a very tiny gift. But that’s only true when [chance of getting job/grant] x [value of job/grant over next best alternative] > [cost of applying].
One really clear case was when I was encouraged to apply for a grant my project wasn’t a natural fit for, because “it’s quick and there are few applicants”. This seemed safe, since the deadline was in a few hours. But in those few hours the number of applications skyrocketed- I want to say 5x but my memory is shaky- presumably because I wasn’t the only person the grantmaker encouraged. I ended up wasting several hours of my and co-founders time before dropping out, because the project really was not a good fit for the grant.
[if the grantmaker is reading this and recognizes themselves: I’m not mad at you personally].
I’ve been guilty of this too, defaulting to encouraging people to try for something without considering the costs of making the attempt, or the chance of success. It feels so much nicer than telling someone “yeah you’re probably not good enough”.
A lot of EA job postings encourage people to apply even if they don’t think they’re a good fit. I expect this is done partially because orgs genuinely don’t want to lose great applicants who underestimate themselves, and partially because it’s an extremely cheap way to feel anti-elitist.
I don’t know what the solution is here. Many people are miscalibrated on their value or their competition, all else being equal you do want to catch those people. But casting wider net entails more bycatch.
It’s hard to accuse an org of being mean to someone who they encouraged to apply for a job or grant. But I think that should be in the space of possibilities, and we should put more emphasis on invitations to apply for jobs/grants/etc being clear, and less on welcoming. This avoids wasting the time of people who were predictably never going to get the job.
I think this falls into a broader class of behaviors I’d call aspirational inclusiveness.I do think shifting the relative weight from welcoming to clear is good. But I’d frame it as a “yes and” kind of shift. The encouragement message should be followed up with a dose of hard numbers.Something I’ve appreciated from a few applications is the hiring manager’s initial guess for how the process will turn out. Something like “Stage 1 has X people and our very tentative guess is future stages will go like this”. Scenarios can also substitute in areas where numbers may be misleading or hard to obtain. I’ve gotten this from mentors before, like here’s what could happen if your new job goes great. Here’s what could happen if your new job goes badly. Here’s the stuff you can control and here’s the stuff you can’t control.Something I’ve tried to practice in my advice is giving some ballpark number and reference class. I tell someone they should consider skilling up in hard area or pursuing competitive field, then I tell them I expect success in <5% of people I give the advice to, and then say you may still want to do it because of certain reasonsYes, it’s all very noisy. But numbers seem far far better than expecting applicants to read between the lines on what a heartwarming message is supposed to mean, especially early-career folks who would understandably assign a high probability of success with it
Oh I like this phrase a lot
Yeah this sounds right.
One thing is just that discouragement is culturally quite hard and there are strong disincentives to do so; eg I think I definitely get more flak for telling people they shouldn’t do X than telling them they should (including a recent incidence which was rather personally costly). And I think I’m much more capable of diplomatic language than the median person in such situations; some of my critical or discouraging comments on this forum are popular.I also know at least 2 different people who were told (probably wrongly) many years ago that they can’t be good researchers, and they still bring it up as recently as this year. Presumably people falsely told they can be good researchers (or correctly told that they cannot) are less likely to e.g. show up at EA Global. So it’s easier for people in positions of relative power or prestige to see the positive consequences of encouragement, and the negative consequences of discouragement, than the reverse.
Sometimes when people ask me about their chances, I try to give them off-the-cuff numerical probabilities. Usually the people I’m talking to appreciate it but sometimes people around them (or around me) get mad at me.(Tbf, I have never tried scoring these fast guesses, so I have no idea how accurate they are).
How my perspective has changed on this during the last few years is to advise others not to give much weight to a single point of feedback. Especially for those who’ve told me only one or two people have discouraged them from be(com)ing a researcher, I tell them not to stop trying in spite of that. That’s even when the person giving the discouraging feedback is in a position of relative power or prestige.
The last year seems to have proven that the power or prestige someone has gained in EA is a poor proxy for how much weight their judgment should be given on any, single EA-relsted topic. If Will MacAskill and many of his closest peers are doubting how they’ve conceived of EA for years in the wake of the FTX collapse, I expect most individual effective altruists confident enough to judge another’s entire career trajectory are themselves likely overconfident.
Another example is AI safety. I’ve talked to dozens of aspiring AI safety researchers who’ve felt very discouraged
An illusory consensus thrust upon them that their work was essentially worthless because it didn’t superficially resemble the work being done by the Machine Intelligence Research Institute or whatever other approach was in vogue at the time. For years, I suspected that was bullshit.
Some of the brightest effective altruists I’ve met were being inundated by personal criticism harsher than any even Eliezer Yudkowsky would give. I told those depressed, novice AIS researchers to ignore those dozens of jerks who concluded the way to give constructive criticism, like they presumed Eliezer would, was to emulate a sociopath. These people were just playing a game of ‘follow the leader’ not even the “leaders” would condone. I distrusted their hot takes based on clout and vibes about who was competent and who wasn’t
Meanwhile, increasingly over the last year or two, more and more of the AIS field, including some of its most reputed luminaires, have come out of the woodwork more and more to say, essentially, “lol, turns out we didn’t know what we were doing with alignment the whole time, we’re definitely probably all gonna die soon, unless we can convince Sam Altman to hit the off switch at OpenAI.” I feel vindicated in my skepticism of the quality of the judgement of many of our peers.
Thanks for this post, as I’ve been trying to find a high-impact job that’s a good personal fit for 9 months now. I have noticed that EA organizations use what appears to be a cookie-cutter recruitment process with remarkable similarities across organizations and cause areas. This process is also radically different from what non-EA nonprofit organizations use for recruitment. Presumably EA organizations adopted this process because there’s evidence behind its effectiveness but I’d love to see what that evidence actually is. I suspect it privileges younger, (childless?) applicants with time to burn, but I don’t have data to back up this suspicion other than viewing the staff pages of EA orgs.
Can you say more about cookie-cutter recruitment? I don’t have a good sense of what you mean here.
I think solving this is tricky. I want hiring to be efficient, but most ways hiring orgs can get information take time, and that’s always going to be easier for people with more free time. I think EA has an admirable norm of paying for trials and deserves a lot of credit for that.
One possible solution is to have applicants create a prediction market on their chance of getting a job/grant, before applying—this helps grant applicants get a sense of how good their prospects are. (example 1, 2) Of course, there’s a cost to setting up a market and making the relevant info legible to traders, but it should be a lot less than the cost of writing the actual application.
Another solution I’ve been entertaining is to have grantmakers/companies screen applications in rounds, or collaboratively, such that the first phase of application is very very quick (eg “drop in your Linkedin profile and 2 sentences about why you’re a good fit”).
I’d be interested in seeing some organizations try out the very very quick method. Heck, I’d be willing to help set it up and trial run it. My rough/vague perception is that a lot of the information in a job application is superfluous.
I also remember Ben West posting some data about how a variety of “how EA is this person” metrics held very little predictive value in his own hiring rounds.
EA hiring gets a lot of criticism. But I think there are aspects at which it does unusually well.
One thing I like is that hiring and holding jobs feels way more collaborative between boss and employee. I’m much more likely to feel like a hiring manager wants to give me honest information and make the best decision, whether or not that’s with them.Relative to the rest of the world they’re much less likely to take investigating other options personally.
Work trials and even trial tasks have a high time cost, and are disruptive to people with normal amounts of free time and work constraints (e.g. not having a boss who wants you to trial with other orgs because they personally care about you doing the best thing, whether or not it’s with them). But trials are so much more informative than interviews, I can’t imagine hiring for or accepting a long-term job without one.
Trials are most useful when you have the least information about someone, so I expect removing them to lead to more inner-ring dynamics and less hiring of unconnected people.
EA also has an admirable norm of paying for trials, which no one does for interviews.
The impression I get from the interview paradigm vs work trial paradigm is: so much of today’s civilization is less than 100 years old, and really big transformations happen during each decade. The introduction of work trials is one of those things.
Two popular responses to FTX are “this is why we need to care more about honesty” and “this is why we need to not do weird/sketchy shit”. I pretty strongly believe the former. I can see why people would believe the latter, but I worry that the value lost is too high.
But I think both side can agree that representing your weird/sketchy thing as mundane is highly risky. If you’re going to disregard a bunch of the normal safeguards of operating in the world, you need to replace them with something, and most of those somethings are facilitated by honesty.
None of my principled arguments against “only care about big projects” have convinced anyone, but in practice Google reorganized around that exact policy (“don’t start a project unless it could conceivably have 1b+ users, kill if it’s ever not on track to reach that”) and they haven’t grown an interesting thing since.
My guess is the benefits of immediately aiming high are overwhelmed by the costs of less contact with reality.
Can you link to a source about this?
the policy was commonly announced when I worked at google (2014), I’m sure anyone else who was there at the time would confirm its existence. In terms of “haven’t grown anything since”, I haven’t kept close track but can’t name one and frequently hear people say the same.
I like the Google Pixels. Well specifically I liked 2 and 3a but my current one (6a) is a bit of a disappointment. My house also uses Google Nest and Chromecast regularly. Tensorflow is okay. But yeah, overall certainly nothing as big as Gmail or Google Maps, never mind their core product.
Google was producing the Android OS and its own flagship phones well before the Pixel, so I consider it to predate my knowledge of the policy (although maybe the policy started before I got there, which I’ve now dated to 4/1/2013)
Please send me links to posts with those arguments you’ve made, as I’ve not read them, though my guess would be that you haven’t convinced anyone because some of the greatest successes in EA started out so small. I remember the same kind of skepticism being widely expressed some projects like that.
Rethink Priorities comes to mind as one major example. The best example is Charity Entrepreneurship. It was not only one of those projects with potential scalability that was doubted. It keeps incubating successful non-profit EA startups for across almost every EA-affiliated cause. CE’s cumulative track record might the best empirical argument against the broad applicability to the EA movement of your own position here.
Your comment makes the most sense to me if you misread my post and are responding to exactly the opposite of my position, but maybe I’m the one misreading you.
Upvoted. Thanks for clarifying. The conclusion to your above post was ambiguous to me, though I now understand.
As of October 2022, I don’t think I could have known FTX was defrauding customers.
If I’d thought about it I could probably have figured out that FTX was at best a casino, and I should probably think seriously before taking their money or encouraging other people to do so. I think I failed in an important way here, but I also don’t think my failure really hurt anyone, because I am such a small fish.
But I think in a better world I should have had the information that would lead me to conclude that Sam Bankman-Fried was an asshole who didn’t keep his promises, and that this made it risky to make plans that depended on him keeping even explicit promises, much less vague implicit commitments. I have enough friends of friends that have spoken out since the implosion that I’m quite sure that in a more open, information-sharing environment I would have gotten that information. And if I’d gotten that information, I could have shared it with other small fish who were considering uprooting their lives based on implicit commitments from SBF. Instead, I participated in the irrational exuberance that probably made people take more risks on the margin, and left them more vulnerable to the collapse of FTX. Assigning culpability is hard here, but this isn’t just an abstract worry: I can think of one person I might bear some responsibility for, and another who I would be almost 100% responsible for, except they didn’t get the grant.
I think the encouragement I gave people represents a moral failure on my part. I should have realized I didn’t have enough information to justify it, even if I never heard about specific bad behavior. Hell even if SBF wasn’t an unreliable asshole, Future Fund could have turned off the fire hose for lots of reasons. IIRC they weren’t even planning on continuing the regrantor project.
But it would also have been cool if that low key, “don’t rely on Sam- I’m not accusing him of anything malicious, he’s just not reliable” type of information had circulated widely enough that it reached me and the other very small fish, especially the ones taking major risks that only made sense in an environment where FTX money flowed freely.
I don’t know what the right way to do that would have been. But it seems important to figure out.
I also suspect that in an environment where it was easy to find out that SBF was an unreliable asshole, it would have been easier to discover or maybe even prevent the devastating fraud, because people would have felt more empowered to say no to him. But that might be wishful thinking.
I think the encouragement I gave people represents a moral failure on my part. I should have realized I didn’t have enough information to justify it, even if I never heard about specific bad behavior.
I don’t know the specific circumstances of your or anyone else’s encouragement, so I want to be careful not to opine on any specific circumstances. But as a general matter, I’d encourage self-compassion for “small fish”  about getting caught up in “irrational exuberance.” Acting in the presence of suboptimal levels of information is unavoidable, and declining to act until things are clearer carries moral weight as well.
In retrospect, we know that the EA whispernet isn’t that reliable, that prominence in EA shouldn’t be seen as a strong indicator of reliability, that the media was asleep at the wheel, and that crypto investors exercise very minimial due dillgence. But I don’t think we should expect “small fish” to have known those things in 2021 and 2022.
Hell even if SBF wasn’t an unreliable asshole, Future Fund could have turned off the fire hose for lots of reasons. IIRC they weren’t even planning on continuing the regrantor project.
As far as other potential failure modes, I think an intelligent individual doing their due dilligence before making a major life decision would have spotted those risks. It would have been easy to find out (without relying on inside EA knowledge) that anything crypto is risky as hell, that anything involving a company that has only been in operation a few years is pretty risky on top of that, that SBF didn’t have a long track record of consistent philantrophy at this level, and (in most cases) that the grants were fairly short-term with no guarantee of renewal.
Given that we should be gentle and understanding toward small fish who relied on FTX funding to their detriment, I would extend similar gentleness and understanding toward small fish who encouraged others (at least without actively downplaying risks). So I think there’s a difference between encouragement that actively downplayed those risks, and encouragement that did not affirmatively include a recognition of the relatively high risk.
But it would also have been cool if that low key, “don’t rely on Sam- I’m not accusing him of anything malicious, he’s just not reliable” type of information had circulated widely enough that it reached me and the other very small fish, especially the ones taking major risks that only made sense in an environment where FTX money flowed freely.
I directionally agree, but think it is important to recognize that the signal of SBF’s unreliability would likely be contained in a sea of noise of inaccurate information, accurate information that didn’t predict future bad outcomes, and outright malicious falsehoods. A rational individual would discount the reports, with the degree of discount based on the signal:noise ratio of the whispernet and other factors.
Given the reasons FTX funding could fall through for reasons unrelated to SBF’s character or reliability, I predict that such a signal getting out would have had a meaningful—yet fairly modest—effect on proper risk evaluation by a small fish. For sure, an increase from 30% to 40% risk [making numbers up, but seems roughly plausible?] would have changed some decisions at the margin (either to accept grants or not to take more precautions).
But we would also need to weigh that other decisions would have changed for the worse because of noise in the whispernet about other people. While the tradeoff can be mitigated to some extent, I think it is largely inherent to whispernets and most other reputation-based systems. I generally think that assessing and communicating this sort of risk is very difficult, and that some sort of system for ameliorating the situation of people who get screwed is therefore a necessary piece of the solution. To me, this is similiar to how a rational response to the risk of fire includes both fire prevention (being aware of and mitigating risks) and fire insurance (because prevention is not a foolproof process).
I’m not intending to offer an opinion about larger fish, or people with more direct information about SBF, either way.
I think expecting myself to figure out the fraud would be unreasonable. As you say, investors giving him billions of dollars didn’t notice, why should I, who received a few tens of thousands, be expected to do better due diligence? But I think a culture where this kind of information could have bubbled up gradually is an attainable and worthwhile goal.
E.g. I think my local community handled covid really well. That didn’t happen because someone wrote a big scary announcement. It was an accumulation of little things, like “this is probably nothing but always good to keep a stock of toilet paper” and “if this is airborne masks are probably useful”. And that could happen because those small statements were allowed. And I think it would have been good if people could similarly share small warnings about SBF as casually as they shared good things, and an increasingly accurate picture would emerge over time.
Am I understanding right that the main win you see here would have been protecting people from risks they took on the basis that Sam was reasonably trustworthy?
I also feel pretty unsure but curious about whether a vibe of “don’t trust Sam / don’t trust the money coming through him” would have helped discover or prevent the fraud—if you have a story for how it could have happened (e.g. via as you say people feeling more empowered to say no to him—maybe it would have via been his staff making fewer crazy moves on his behalf / standing up to him more?), I’d be interested.
“protect people from dependencies on SBF” is the thing for which I see a clear causal chain and am confident in what could have fixed it.
I do have a more speculative hope that an environment where things like “this billionaire firehosing money is an unreliable asshole” are easy to say would have gotten better outcomes for the more serious issues, on the margin. Maybe the FTX fraud was overdetermined, even if it wasn’t and I definitely don’t have enough insight to be confident in picking a correction. But using an abstract version of this case as an example for how I think a more open environment could have led to better outcomes:
My sense is SBF just kept taking stupid unethical bets and having them work out for him financially and socially. Maybe small consequences early on would have reduced the reward to stupid unethical bets.
Before the implosion, SBF(’s public persona) was an EA success story that young EAs aspired to copy. Less of that on the margin would probably lead to less fraud 5 years from now, especially in the world where the FTX fraud took longer to discover.
I think aping SBF’s persona was bad for other reasons, but they’re harder to justify.
SBF would have gotten more push back from staff (unless the fact that he was a known asshole made people more likely to leave, which seems good for them but not an improvement vis a vis fraud).
FTX would have had a harder time recruiting, which would have slowed them down.
Some EAs chose to trade on FTX out of ingroup loyalty, and maybe that would have happened less.
An environment where you’re free to share information about SBF being an unreliable asshole is more hospitable to sharing and hearing other negative information, and this has a snowball effect. Who knows what else would have been shared if the door had been open a crack.
Maybe Will MacAskill would have spent less time telling the press that SBF was a frugal virtue wunderkind.
Maybe other people would have told Will MacAskill to stop telling the press that SBF was a frugal virtue wunderkind.
Maybe Will MacAskill would have pushed that line to the press, but other people would have told the press “no he isn’t”, and that could have been a relatively gentle lesson for SBF and Will.
My sense is Will isn’t the only prominent EA who gave SBF a lot of press, just the most prominent and the one I heard the most about. Hopefully all of that would be reduced.
Maybe people would have been more open when considering if FTX money was essentially casino money, and what are the ethical implications of that?
Good posts generate a lot of positive externalities, which means they’re undersupplied, especially by people who are busy and don’t get many direct rewards from posting. How do we fix that? What are rewards relevant authors would find meaningful?
Here are some possibilities off the top of my head, with some commentary. My likes are not universal and I hope the comments include people with different utility functions.
Money. Always a classic, rarely disliked although not always prioritized. I’m pretty sure this is why LTFF and EAIF are writing more now.
Appreciation (broad). Some people love these. I definitely prefer getting them over not getting them, but they’re not that motivating for me. Their biggest impact on motivation is probably cushioning the blow of negative comments.
Appreciation (specific). Things like “this led to me getting my iron tested” or “I changed my mind based on X”. I love these, they’re far more impactful than generic appreciation.
High quality criticism that changes my mind.
Arguing with bad commenters.
One of the hardest parts of writing for me is getting a shitty, hostile comment, and feeling like my choices are “let it stand” or “get sucked into a miserable argument that will accomplish nothing”. Commenters arguing with commenters gets me out of this dilemma, which is already great, but then sometimes the commenters display deep understanding of the thing I wrote and that’s maybe my favorite feeling.
Deliberately not included: longer term rewards like reputation that can translate into jobs, employees, etc. I’m specifically looking for quick rewards for specific posts
I definitely agree that funding is a significant factor for some institutional actors.
For example, RP’s Surveys and Data Analysis team has a significant amount of research that we would like to publish if we had capacity / could afford to do so: our capacity is entirely bottlenecked on funding and as we are ~ entirely reliant on paid commissions (we don’t receive any grants for general support) time spent publishing reports is basically just pro bono, adding to our funding deficit.
Example of this sort of unpublished research include:
The two reports mentioned by CEA here about attitudes towards EA post-FTX among the general public, elites, and students on elite university campuses.
Followup posts about the survey reported here about how many people have heard of EA, to further discuss people’s attitudes towards EA, and where members of the general public hear about EA (this differs systematically)
Updated numbers on the growth of the EA community (2020-2022) extending this method and also looking at numbers of highly engaged longtermists specifically
Several studies we ran to develop reliable measure of how positively inclined towards longtermism people are, looking at different predictors of support for longtermism and how these vary in the population
Reports on differences between neartermists and longtermists within the EA community and on how neartermist / longtermist efforts influence each other (e.g. to what extent does neartermist outreach, like GiveWell, Peter Singer articles about poverty, lead to increased numbers of longtermists)
Whether the age at which one first engaged with EA predicts lower / higher future engagement with EA
A significant dynamic here is that even where we are paid to complete research for particular orgs, we are not funded for the extra time it would take to write up and publish the results for the community. So doing so is usually unaffordable, even where we have staff capacity.
Of course, much of our privately commissioned research is private, such that we couldn’t post it. But there are also significant amounts of research that we would want to conduct independently, so that we could publish it, which we can’t do purely due to lack of funding. This includes:
More message testing research related to EA /longtermism (for an example see Will MacAskill’s comment referencing our work here), including but not limited to:
Testing the effectiveness of specific arguments for these causes
Testing how “longtermist” or “existential risk” or “effective altruist” or “global priorities” framings/brandings compare in terms of how people respond to them (including comparing this to just advocating for specific concrete x-risks without
Testing effectiveness of different approaches to outreach in different populations for AI safety / particular policies
“We want to publish but can’t because the time isn’t paid for” seems like a big loss, and a potentially fixable one. Can I ask what you guys have considered for fixing it? This seems to me like an unusually attractive opportunity for crowdfunding or medium donors, because it’s a crisply defined chunk of work with clear outcomes. But I imagine you guys have already put some thought into how to get this paid for.
To be totally honest, I have qualms about the specific projects you mention, they seem centered on social reality not objective reality. But I value a lot of RP’s other work, think social reality investigations can be helpful in moderation, and my qualms about these questions aren’t enough to override the general principle.
“We want to publish but can’t because the time isn’t paid for” seems like a big loss, and a potentially fixable one. Can I ask what you guys have considered for fixing it? This seems to me like an unusually attractive opportunity for crowdfunding or medium donors, because it’s a crisply defined chunk of work with clear outcomes.
Thanks! I’m planning to post something about our funding situation before the end of the year, but a couple of quick observations about the specific points you raise:
I think funding projects from multiple smaller donors is just generally more difficult to coordinate than funding from a single source
A lot of people seem to assume that our projects already are fully funded or that they should be centrally funded because they seem very much like core community infrastructure, which reduces inclination to donate
they seem centered on social reality not objective reality. But I value a lot of RP’s other work, think social reality investigations can be helpful in moderation, and my qualms about these questions aren’t enough to override the general principle.
I’d be curious to understand this line of thinking better if you have time to elaborate. “Social” vs “objective” doesn’t seem like a natural and action-guiding distinction to me. For example:
Does everyone we want to influence hate EA post-FTX?
Are people more convinced by outreach based on “longtermism” or “existential risk” or principles-based effective altruism or specific concrete causes more effective?
Do people who first engage with EA when they are younger end up less engaged with EA than those who first engage when they are older?
How fast is EA growing?
all strike me as objective social questions of clear importance. Also, it seems like the key questions around movement building will often be (characterisable as) “social” questions. I could understand concerns about too much meta but too much “social” seems harder to understand.
A possible interpretation I would have some sympathy for is distinguishing between concern with what is persuasive vs what is correct. But I don’t think this raises concerns about these kinds of projects, because:
- A number of these projects are not about increasing persuasiveness at all (e.g. how fast is EA growing? Where are people encountering EA ideas?). Even findings like “does everyone on elite campuses hate EA?” are relevant for reasons other than simply increasing persuasiveness, e.g. decisions about whether we should increase or decrease spending on outreach at the top of the funnel.
- Even if you have a strong aversion to optimising for persuasiveness (you want to just present the facts and let people respond how they will), you may well still want to know if people are totally misunderstanding your arguments as you present them (which seems exceptionally common in cases like AI risk).
- And, of course, I think many people reasonably think that if you care about impact, you should care about whether your arguments are persuasive (while still limiting yourself to arguments which are accurate, sincerely held etc.).
- The overall EA portfolio seems to assign a very small portion of its resources to this sort of research as it stands (despite dedicating a reasonably large amount of time to a priori speculation about these questions (1)(2)(3)(4)(5)(6)(7)(8)) so some more empirical investigation of them seems warranted.
Yeah, “objective” wasn’t a great word choice there. I went back and forth between “objective”, “object”, and “object-level”, and probably made the wrong call. I agree there is an objective answer to “what percentage of people think positively of malaria nets?” but view it as importantly different than “what is the impact of nets on the spread of malaria?”
I agree the right amount of social meta-investigation is >0. I’m currently uncomfortable with the amount EA thinks about itself and its presentation; but even if that’s true, professionalizing the investigation may be an improvement. My qualms here don’t rise to the level where I would voice them in the normal course of events, but they seemed important to state when I was otherwise pretty explicitly endorsing the potential posts.
I can say a little more on what in particular made me uncomfortable. I wouldn’t be writing these if you hadn’t asked and if I hadn’t just called for money for the project of writing them up, and if I was I’d be aiming for a much higher quality bar. I view saying these at this quality level as a little risky, but worth it because this conversation feels really productive and I do think these concerns about EA overall are important, even though I don’t think they’re your fault in particular:
several of these questions feel like they don’t cut reality at the joints, and would render important facets invisible. These were quick summaries so it’s not fair to judge them, but I feel this way about a lot of EA survey work where I do have details.
several of your questions revolve around growth; I think EA’s emphasis on growth has been toxic and needs a complete overhaul before EA is allowed to gather data again.
I especially think CEA’s emphasis on Highly Engaged people is a warped frame that causes a lot of invisible damage. My reasoning is pretty similar to Theo’s here.
I don’t believe EA knows what to do with the people it recruits, and should stop worrying about recruiting until that problem is resolved.
Asking “do people introduced to EA younger stick around longer?” has an implicit frame that longer is better, and is missing follow-ups like “is it good for them? what’s the counterfactual for the world?”
I think we need to be a bit careful with this, as I saw many highly upvoted posts that in my opinion have been actively harmful. Some very clear examples:
Theses on Sleep, claiming that sleep is not that important. I know at least one person that tried to sleep 6 hours/day for a few weeks after reading this, with predictable results
A Chemical Hunger, “a series by the authors of the blog Slime Mold Time Mold (SMTM) that has been received positively on LessWrong, argues that the obesity epidemic is entirely caused by environmental contaminants.” It wouldn’t surprise me if it caused several people to update their diets in worse ways, or in general have a worse model of obesity
In general, I think we should promote more posts like “Veg*ns should take B12 supplements, according to nearly-unanimous expert consensus” while not promoting posts like “Veg*nism entails health tradeoffs”, when there is no scientific evidence of this and expert consensus of the contrary. (I understand that your intention was not to claim that a vegan diet was worse than an average non-vegan diet, but that’s how most readers I’ve spoken to updated in response to your posts.)I would be very excited about encouraging posts that broadcast knowledge where there is expert consensus that is widely neglected (e.g. Veg*ns should take B12 supplements), but I think it can also be very easy to overvalue hard-to-measure benefits, and we should keep in mind that the vast majority of posts get forgotten after a few days.
I think you are incorrectly conflating being mistaken and being “actively harmful” (what does actively mean here?) I think most things that are well-written and contain interesting true information or perspectives are helpful, your examples included.
Truth-seeking is a long game that is mostly about people exploring ideas, not about people trying to minimize false beliefs at each individual moment.
I think you are incorrectly conflating being mistaken and being “actively harmful”
That’s a fair point, I listed posts that were clearly not only mistaken but also harmful, to highlight that the cost-benefit analysis of “good posts” as a category is very non-obvious.
(what does actively mean here?)
I shouldn’t have used the term “actively”, I edited the comment.
I think most things that are well-written and contain interesting true information or perspectives are helpful, your examples included.
I fear that there’s a very real risk of building castles in the sky, where interesting true information gets mixed with interesting not-so-true information and woven into a misleading narrative that causes bad consequences, that this happens often, and that we should be mindful of that.
I should have explicitly mentioned it, but I mostly agree with Elizabeth’s quick take. I just want to highlight that while some “good posts” “generate a lot of positive externalities”, many other “good posts” are wrong and harmful (and many many more get forgotten after a few days). I’m also probably more skeptical of hard-to-measure diffuse benefits without a clear theory of change or observable measures and feedback loops.
LessWrong’s emoji palette is great
That palette is not just great in the abstract, it’s great as a representation of LW. I did some very interesting anthropology with some non-rationalist friends explaining the meaning and significance of the weirder reacts.
A lot of what I explained was how specific reacts relate to one of the biggest pain points on LW (and EAF): shitty comments. The reacts are weirdly powerful, in part because it’s not the comments’ existence that’s so bad, it’s knowing that other people might read them and not understand they are shitty. I could explain why in a comment of my own, but that invites more shitty comments and draws attention to the original one. It’s only worth it if many people are seeing and believing the comment.
Emojis neatly resolve this. If several people mark a comment as soldier mindset, I feel off the hook for arguing with it. And if several people (especially people I respect) mark a comment as insightful or changing their mind, that suggests that at a minimum it’s worth the time to engage with the comment, and quite possibly I am in the wrong.
You might say I should develop a thicker skin so shitty comments bug me less, and that is probably be true on the margin, but I think it’s begging the question. Emojis give me valuable information about how a comment is received; positive emojis suggest I am wrong that it is shitty, or at least how obvious it is. It is good to respond differently to good comments, obviously shitty comments, and controversial comments differently, and detailed reacts make it much easier. So I think this was a huge win for LessWrong.
Meanwhile on the EAForum…
[ETA 2023-11-10: turns out I picked a feel-good thread with a special react palette to get my screen shots. I still think my point holds overall but regret the accidental exaggeration. I should have been more surprised when I went to get a screen shot and the palette wasn’t what I expected]
This palette has 5 emojis (clapping, party, heart, star, and surprise) covering maybe 2.5 emotions if you’re generous and count heart as care and not just love. It is considerably less precise than Facebook’s palette. I suspect the limited palette is an attempt to keep things positive, but given that negative comments are (correctly) allowed, this only limits the ability to cheaply push back.
I’ll bet EAF put a lot of thought into their palette. This isn’t even their first palette, I found out the palette had changed when I went to get a screen shot for this post. I would love to hear more about why they chose these 5.
This is not quite as bad as the feel-good palette (I’m so sorry!), but I still think leaves a tremendous amount of value on the table. It gives no way to give specific negative feedback to bad comments like “too combative” or “misunderstands position?” . It’s not even particularly good at compliments, except for “Changed my mind”.
How common do you think “shitty comments” are? And how well/poorly do you think the existing karma system provides an observer with knowledge that the user base “understand[s] they are shitty”? (To be sure, it doesn’t tell you if the voting users understand exactly why the comment is shitty.)
I’m not sure how many people would post attributed-to-them emojis if they weren’t already anonymously downvoting a comment for being shitty. So if they aren’t already getting significant downvotes, I don’t know how many negative emojis they would get here.
They’re especially useful for comments of mixed quality- e.g. someone is right and making an important point, but too aggressively. Or a comment is effortful, well-written, and correct within its frame, but fundamentally misunderstood your position. Or, god forbid, someone make a good point and a terrible point in the same comment. I was originally skeptical of line-level reacts but ended up really valuing them due to that.
There’s also reacts like “elaborate”, “taboo this word” and “example” that invite a commenter to correct problems, at which point the comment may become really valuable. Unfortunately there’s no notifications for reacts so this can really easily go unnoticed, but I it at least raises the option.
If I rephase your question as “how often do I see comments for which reacts convey something important I couldn’t say with karma?”; most of my posts since reacts came out have been controversial, so I’m using many comment reacts per post (not always dismissively).
I also find positive emojis much more rewarding than karma, especially Changed My Mind.
I like the LW emoji palette, but it is too much. Reading forum posts and parsing through comments can be mentally taxing. I don’t want to spend additional effort going through a list of forty-something emojis and buttons to react to something, especially comments. I am often pressed for time, so almost always I would avoid the LW emoji palette entirely. Maybe a few other important reactions can be added instead of all of them? Or maybe there could be a setting which allows people to choose if they want to see a “condensed” or “extended” emoji palette? Either way, just my two cents.
I agree EAF shouldn’t have a LW-sized palette, much less LW’s specific palette. I want EAF to have a palette that reflects its culture as well as LW’s palette reflects its culture. And I think that’s going to take more than 4 reacts (note that my original comment mortifyingly used a special palette made for a single post, the new version has the normal EAF reacts of helpful, insightful, changed my mind, and heart), but way less than is in the LW palette.
I do think part of LessWrong’s culture is preferring to have too many options rather than making do with the wrong one. I know the team has worked really hard to keep reacts to a manageable level, while making most of them very precise, while covering a wide swath of how people want to react. I think they’ve done an admirable job (full disclosure: I’m technically on the mod team and give opinions in slack, but that’s basically the limit of my power). This is something I really appreciate about LW, but I know shrinks its audience.
I’m not on LW very often, how frequently do you see these emojis being used?
From a UX perspective, I agree with Akash—it seems like there are way too many options and my prior is that people wouldn’t use >80% of them.
Hi! I think we might have a bug — I’m not sure where you’re seeing those emojis on the Forum. For me, here are the emojis that show up:
@Agnes Stenlund might be able to say more about how we chose those, but I do think we went for this set as a way to create a low-friction way of sharing non-anonymous positive feedback (which authors and commenters have told us they lack, and some have told us that they feel awkward just commenting with something non-substantive but positive like “thanks!”) while also keeping the UX understandable and easy to use. I think it’s quite possible that it would be better to also add some negative/critical emojis, but I’m not very convinced right now and not convinced that it’s super promising relative to the others stuff we’re working on, & something we should dive deeper into. It won’t be my call in the end, regardless, but I’m definitely up for hearing arguments about why this is wrong!
I don’t view this as a finalized set — I think there’s a >50% chance (75%?) that we’ve changed at least something about it in the next ~6 months.
Not a bug—it’s from Where are you donating this year, and why? which is grandfathered into an old experimental voting system (and it’s the only post with this voting system—there are a couple of others with different experimental systems).
I’m so sorry- I should have been more surprised when I went to get a screenshot and it wasn’t the palette I expected. I have comments set to notify me only once per day, so I didn’t get alerted to the issue until now.
I wrote this with the standard palette so I still think there is a problem, but I feel terrible for exaggerating it with a palette that was perfectly appropriate for its thread.
Semi-tangential question: what’s the rationale for making the reactions public but the voting (including the agree/disagree voting) anonymous?
Where are you seeing that emoji palette on here?
See sister thread- this was for a specific positivity focused thread I picked completely at random 😱.
I’ll bet EAF put a lot of thought into their palette.
As Ollie mentioned, I made the set you referenced for just this one thread. As far as I remember it was meant to to support positive vibes in that thread and was done very quickly, so I would not say a lot of thought went into that palette.
@Lizka and co: could I ask for some commentary on this?
A repost from the discussion on NDAs and Wave (a software company). Wave was recently publicly revealed to have made severance dependent on non-disparagement agreements, cloaked by non-disclosure agreements. I had previously worked at Wave, but negotiated away the non-disclosure agreement (but not the non-disparagement agreement).
But my guess is that most of the people you sent to Wave were capable of understanding what they were signing and thinking through the implications of what they were agreeing to, even if they didn’t actually have the conscientiousness / wisdom / quick-thinking to do so. (Except, apparently, Elizabeth. Bravo, @Elizabeth!)
I appreciate the kudos here, but feel like I should give more context.
I think some of what led to me to renegotiate was a stubborn streak and righteousness about truth. I mostly hear when those traits annoy people, so it’s really nice to have them recognized in a good light here. But that righteous streak was greatly enabled by the fact that my mom is a lawyer who modeled reading legal documents before signing (even when it’s embarrassing your kids who just want to join their friends at the rockclimbing birthday party), and that I could afford to forgo severance. Obviously I really wanted the money, and I couldn’t afford to take this kind of stand every week. But I believe there were people who couldn’t even afford to add a few extra days, and so almost had to cave
To the extent people in that second group were unvirtuous, I think the lack of virtue occurred when they didn’t create enough financial slack to even have the time to negotiate. By the time they were laid off without a cushion it was too late. And that’s not available to everyone- Wave paid well, but emergencies happen, any one of them could have a really good reason their emergency fund was empty.
So the main thing I want to pitch here is that “getting yourself into a position where virtue is cheap” is an underrated strategy.This is one benefit to paying people well, and a reason having fewer better-paid workers is sometimes better than more people earning less money. If your grants or salary give you just enough to live as long as the grants are immediately renewed/you don’t get fired, even a chance of irritating your source of income imperils your ability to feed yourself. 6 months expenses in savings gives you the ability to risk an individual job/grant. Skills valued outside EA give you the ability to risk pissing off all of EA and still be fine.
I’m emphasizing risk here because I think it’s the bigger issue. If you know something is wrong, you’ll usually figure out a way to act on it. The bigger problem is when you some concerns but they legitimately could be nothing, but worry that investigating will imperil your livelihood.
I sometimes argue against certain EA payment norms because they feel extractive, or cause recipients to incur untracked costs. E.g. “it’s not fair to have a system that requires unpaid work, or going months between work in ways that can’t be planned around and aren’t paid for”. This was the basis for some of what I said here. But I’m not sure this is always bad, or that the alternatives are better. Some considerations
if it’s okay for people to donate money I can’t think of a principled reason it’s not okay for them to donate time → unpaid work is not a priori bad.
If it would be okay for people to solve the problem of gaps in grants by funding bridge grants, it can’t be categorically disallowed to self-fund the time between grants.
If partial self-funding is required to do independent, grant-funded work, then only people who can afford that will do such work. To the extent the people who can’t would have done irreplaceably good work, that’s a loss, and it should be measured. And to the extent some people would personally enjoy doing such work but can’t, that’s sad for them. But the former is an empirical question weighed against the benefits of underpaying, and the latter is not relevant to impact.
I think the costs of blocking people who can’t self-fund from this kind of work are probably high, especially the part where it categorically prevents segments of society with useful information from participating. But this is much more relevant for e.g. global development than AI risk.
A norm against any unpaid work would mean no one could do anything unless they got funder approval ahead of time, which would be terrible.
A related problem is when people need to do free work (broadly defined, e.g. blogging counts) to get a foot in the door for paid work. This has a lot of the same downsides as requiring self-funding, but, man, seems pretty stupid to insist on ignoring the information available from free sources, and if you don’t ban it there will be pressure to do free work.
To me, “creating your own projects, which people use to inform their opinions of you” feels pretty different from “you must do 50 hours of task X unpaid before we consider a paying position”, but there are ambiguous cases.
it’s pretty common for salaried EAs to do unpaid work on top of their normal job. This feels importantly different to me from grant-funded people funding their own bridge loans, because of the job security and predictability. The issue isn’t just “what’s your take home pay per hour?”, it’s “how much ability to plan do you have?”
Any money you spend on one independent can’t be spent on someone else. To the extent EA is financially constrained, that’s a big cost.
It feels really important to me that costs of independence, like self-bridge-funding or the headache of grant applications, get counted in some meaningful sense, the same as donating money or accepting a low salary.
I feel like a lot of castle discourse missed the point.
By default, OpenPhil/Dustin/Owen/EV don’t need anyone’s permission for how they spend their money.
And it is their money, AFAICT open phil doesn’t take small donations. I assume Dustin can advocate for himself here.
One might argue that the castle has such high negative externalities it can be criticized on that front. I haven’t seen anything to convince me of that, but it’s a possibility and “right to spend one’s own money” doesn’t override that.
You could argue OpenPhil etc made some sort of promise they are violating by buying the castle. I don’t think that’s true- but I also think the castle-complainers have a legitimate grievance.
I do think the word “open” conveys something of a promise, and I will up my sympathy for open phil if they change their name. But my understanding is they are more open than most foundations.
My guess is that lots of people entered EA with inaccurate expectations, and the volume at which this happens indicates a systemic problem, probably with recruiting. They felt ~promised that EA wasn’t the kind of place where people bought fancy castles, or would at least publicly announce they’d bought a retreat center and justify it with numbers.
Highly legible, highly transparent parts of EA exist, and I’m glad they do. But it’s not all of EA, and I don’t think it should be. I think it’s important to hold people to commitments, and open phil at one point did have a commitment to transparency, but they publicly renounced it years ago so that’s no longer in play. I think the problem lies with the people who set the false expectations, which I imagine happened in recruiting.
It’s hard for me to be more specific than this because I haven’t followed EA recruiting very closely, so what reaches me tends to be complaints about the worst parts. My guess is this lies in the more outward facing parts of Effective Ventures (GWWC, 80k, CEA’s university recruiting program, perhaps the formalization of EA groups in general).
[I couldn’t quickly verify this but my understanding is open phil provides a lot of the funding for at least some of these orgs, in which case it does bear some responsibility for the misleading recruiting]
I would like to see recruiting get more accurate about what to expect within EA. I want that partially because honesty is generally good, partially because this seems like a miserable experience for people who have been misled. And partially because I want EA to be a weird do-ocracy, and recruiting lots of people who object to doing weird things without permission slows that down.
I think the first point here—that the buyers “don’t need anyone’s permission” to purchase a “castle”—isn’t contested here. Other than maybe the ConcernedEA crowd, is anyone claiming that they were somehow required to (e.g.) put this to a vote?
I think the “right to spend one’s own money” in no way undermines other people’s “right to speak one’s own speech” by lambasting that expenditure. In the same way, my right to free speech doesn’t prevent other people from criticizing me for it, or even deciding not to fund/hire me if I were to apply for funding or a job. There are circumstances in which we have—or should have—special norms against negative reactions by third parties; for instance, no one should be retailiated against for reporting fraud, waste, abuse, harassment, etc. But the default rule is that what the critics have said here is fair game.
A feeling of EA having breached a “~promise” isn’t the only basis for standing here. Suppose a non-EA megadonor had given a $15MM presumably tax-deductible donation to a non-EA charity for buying a “castle.” Certainly both EAs and non-EAs would have the right to criticize that decision, especially because the tax-favored nature of the donation meant that millions’ worth of taxes were avoided by the donation. If one wishes to avoid most public scrutiny, one should make it clear that the donation was not tax-advantaged. In that case, it’s the same as the megadonor buying a “castle” for themselves.
Moreover, I think the level of negative externalities required to give third-party EAs standing to criticize is quite low. The “right to speak one’s own speech” is at least as fundamental as the proposed “right to spend one’s own money.” If the norm is going to be that third parties shouldn’t criticize—much less take adverse actions against—an EA entity unless the negative PR & other side effects of the entity’s action exceed those of the “castle” purpose, then that would seem a pretty fundamental shift in how things work. Because the magnitude of most entities’ actions—especially individuals—are generally an order of magnitude (or more) less than the magnitude of OP and EVF’s actions, the negative externalities will almost never meet this standard.
I 100% agree with you that people should be and are free to give their opinions, full stop.
Many specific things people said only make sense to me if they have some internal sense that they are owed a justification and input (example, example, example, example).
I almost-but-don’t-totally reject PR arguments. EA was founded on “do the thing that works not the thing that looks good”. EAs encourage many other things people find equally distasteful or even abhorrent, because they believe it does the most good. So “the castle is bad PR” is not a good enough argument, you need to make a case for “the castle is bad PR and meaningfully worse than these other things that are bad PR but still good”. I believe things in that category exist, and people are welcome to make arguments that the castle is one of them, but you do have to make the full argument.
I think you’re slightly missing the point of the ‘castle’ critics here.
By default, OpenPhil/Dustin/Owen/EV don’t need anyone’s permission for how they spend their money. And it is their money, AFAICT open phil doesn’t take small donations. I assume Dustin can advocate for himself here.One might argue that the castle has such high negative externalities it can be criticized on that front. I haven’t seen anything to convince me of that, but it’s a possibility and “right to spend one’s own money” doesn’t override that.
By default, OpenPhil/Dustin/Owen/EV don’t need anyone’s permission for how they spend their money. And it is their money, AFAICT open phil doesn’t take small donations. I assume Dustin can advocate for himself here.
Technically this is obviously true. And it was the main point behind one of the most popular responses to FTX and all the following drama. But I think that point and the post misses people’s concerns completely and comes off as quite tone-deaf.
To pick an (absolutely contrived) example, let’s say OpenPhil suddenly says it now believes that vegan diets are more moral and healthier than all other diets, and that B12 supplementation increases x-risk, and they’re going to funnel billions of dollars into this venture to persuade people to go Vegan and to drone-strike any factories producing B12. You’d probably be shocked and think that this was a terrible decision and that it had no place in EA.
OpenPhil saying “it’s our money, we can do what we want” wouldn’t hold much water for you, and the same thing I think goes for the Wytham Abbey critics—who I think do have a strong initial normative point that £15m counterfactually could do a lot of good with the Against Malaria Foundation, or Hellen Keller International.
Like it’s not just a concern about ‘high negative externalities’, many people saw this purchase, along with the lack of convincing explanation (to them), and think that this is just a negative EV purchase, and also negative with externalities—and then there was little in explanation forthcoming to change their mind.
I think OpenPhil maybe did this thinking it was a minor part of their general portfolio, without realising the immense power, both explicit and implicit, they have over the EA community, its internal dyanmics, and its external perception. They may not officially be in charge of EA, but by all accounts unofficially it works something like that (along with EVF), and I think that should at least figure into their decision-making somewhere
My guess is that lots of people entered EA with inaccurate expectations, and the volume at which this happens indicates a systemic problem, probably with recruiting. They felt ~promised that EA wasn’t the kind of place where people bought fancy castles, or would at least publicly announce they’d bought a retreat center and justify it with numbers.open phil at one point did have a commitment to transparency, but they publicly renounced it years ago so that’s no longer in play.
My guess is that lots of people entered EA with inaccurate expectations, and the volume at which this happens indicates a systemic problem, probably with recruiting. They felt ~promised that EA wasn’t the kind of place where people bought fancy castles, or would at least publicly announce they’d bought a retreat center and justify it with numbers.
open phil at one point did have a commitment to transparency, but they publicly renounced it years ago so that’s no longer in play.
Is the retreat from transparency true? If there are some references you could provide me for this? I also feel like there’s a bit of ‘take-it-or-leave-it’ implicit belief/attitude from OpenPhil here if true which I think is unfortunate and, honestly, counterproductive.
I would like to see recruiting get more accurate about what to expect within EA, but I’m not sure what that would look like. I mean I still think that EA “not being the kind of place where people buy fancy castles” is is a reasonable thing to expect and want from EA overall? So I’m not sure that I disagree that people are entering with these kind of expectations, but I’m confused about why you think it’s innacurate? Maybe it’s descriptively inaccurate but I’m a lot less sure that it’s normatively inaccurate?
Bombing B12 factories has negative externalities and is well covered by that clause. You could make it something less inflammatory, like funding anti-B12 pamphlets, and there would still be an obvious argument that this was harmful. Open Phil might disagree, and I wouldn’t have any way to compel them, but I would view the criticism as having standing due to the negative externalities. I welcome arguments the retreat center has negative externalities, but haven’t seen any that I’ve found convincing.
who I think do have a strong initial normative point that £15m counterfactually could do a lot of good with the Against Malaria Foundation, or the Hellen Keller Foundation.
My understanding is:
Open Phil deliberately doesn’t fill the full funding gap of poverty and health-focused charities.
While they have set a burn rate and are currently constrained by it, that burn rate was chosen to preserve money for future opportunities they think will be more valuable. If they really wanted to do both AMF and the castle, they absolutely could.
Given that, I think the castle is a red herring. If people want to be angry about open phil not filling the full funding gaps when it is able I think you can make a case for that, but the castle is irrelevant in the face of its many-billion dollar endowment.
Is the retreat from transparency true? If there are some references you could provide me for this?
Even assuming OP was already at its self-imposed cap for AMF and HKI, it could have asked GiveWell for a one-off recommendation. The practice of not wanting to fill 100% of a funding gap doesn’t mean the money couldn’t have been used profitably elsewhere in a similar organization.
are you sure GW has charities that meet their bar that they aren’t funding as much as they want to? I’m pretty sure that used to not be the case, although maybe it has changed. There’s also value to GW behaving predictably, and not wildly varying how much money it gives to particular orgs from year to year.
This might be begging the question, if the bar is raised due to anticipated under funding. But I’m pretty sure at one point they just didn’t have anywhere they wanted to give more money to, and I don’t know if that has changed.
2023: “We expect to find more outstanding giving opportunities than we can fully fund unless our community of supporters substantially increases its giving.”
Giving Season 2022: “We’ve set a goal of raising $600 million in 2022, but our research team has identified $900 million in highly cost-effective funding gaps. That leaves $300 million in funding gaps unfilled.”
July 2022: “we don’t expect to have enough funding to support all the cost-effective opportunities we find.” Reports rolling over some money from 2021, but much less than originally believed.
Giving Season 2021: GiveWell expects to roll over $110MM, but also believes it will find very-high-impact opportunities for those funds in the next year or two.
Giving Season 2020: No suggestion that GW will run out of good opportunities—“If other donors fully meet the highest-priority needs we see today before Open Philanthropy makes its January grants, we’ll ask Open Philanthropy to donate to priorities further down our list. It won’t give less funding overall—it’ll just fund the next-highest-priority needs.”
Thanks for the response Elizabeth, and the link as well, I appreciate it.
On the B12 bombing example, it was deliberately provocative to show that, in extremis, there are limits to how convincing one would find the justification “the community doesn’t own its donor’s money” as a defence for a donation/grant
On the negative externality point, maybe I didn’t make my point that clear. I think a lot of critics I think are not just concerned about the externalities, but the actual donation itself, especially the opportunity cost of the purchase. I think perhaps you simply disagree with castle critics on the object level of ‘was it a good donation or not’.
I take the point about Open Phil’s funding gap perhaps being the more fundamental/important issue. This might be another case of decontextualising vs contextualising norms leading to difficult community discussions. It’s a good point and I might spend some time investigating that more.
I still think, in terms of expectations, the new EA joiners have a point. There’s a big prima facie tension between the drowning child thought experiment and the Wytham Abbey purchase. I’d be interested to hear what you think a more realistic ‘recruiting pitch’ to EA would look like, but don’t feel the need to spell that out if you don’t want.
I think a retreat center is a justifiable idea, I don’t have enough information to know if Wytham in particular was any good, and… I was going to say “I trust open phil” here, but that’s not quite right, I think open phil makes many bad calls. I think a world where open phil gets to trust its own judgement on decisions with this level of negative externality is better than one where it doesn’t.
I understand other people are concerned about the donation itself, not just the externalities. I am arguing that they are not entitled to have open phil make decisions they like, and the way some of them talk about Wytham only makes sense to me if they feel entitlement around this. They’re of course free to voice their disagreement, but I wish we had clarity on what they were entitled to.
I’d be interested to hear what you think a more realistic ‘recruiting pitch’ to EA would look like, but don’t feel the need to spell that out if you don’t want.
This is the million dollar question. I don’t feel like I have an answer, but I can at least give some thoughts.
I think the drowning child analogy is deceitful, manipulative, and anti-epistemic, so it’s no hardship for me to say we should remove that from recruiting.
Back in 2015 three different EA books came out- Singer’s The Most Good You Can Do, MacAskill’s Doing Good Better, and Nick Cooney’s How To Be Great At Doing Good. My recollection is that Cooney was the only one who really attempted to transmit epistemic taste and a drive to think things through. MacAskill’s book felt like he had all the answers and was giving the reader instructions, and Singer’s had the same issues. I wish EA recruiting looked more like Cooney’s book and less like MacAskill’s.
That’s a weird sentence because Nick Cooney has a high volume of vague negative statements about him. No one is very specific, but he shows up on a lot of animal activism #metoo type articles. So I want to be really clear this preference is for that book alone, and it’s been 8 years since I read it.
I think the emphasis on doing The Most Possible Good (* and nothing else counts) makes people miserable and less effective. It creates a mix of decision paralysis, excess deference, and pushes people into projects too ambitious for them to learn from, much less succeed at.
I’m interested in what Charity Entrepreneurship thinks we should do. They consistently incubate the kind of small, gritty projects I think make up the substrate of a healthy ecosystem. TBH I don’t think any of their cause areas are as impactful as x-risk, but succeeding at them is better than failing to influence x-risk, and they’re skill-building while they do it. I feel like CE gets that real work takes time, and I’d like to see that attitude spread.
(@Judith, @Joey would love to get your take here)
@Caleb Parikh has talked about how he grades people coming from “good” EA groups more harshly, because they’re more likely to have been socially pressured into “correct” views. That seems like a pretty bad state of affairs.
I think my EA group (seattle, 2014) handled this fantastically, there was a lot of arguing with each other and with EA doctrine. I’d love to see more things look like that. But that was made up heavily of adult rationalists with programming jobs, not college students.
Addendum: I just checked out Wytham’s website, and discovered they list six staff. Even if those people aren’t all full-time, several of them supervise teams of contractors. This greatly ups the amount of value the castle would need to provide to be worth the cost. AFAIK they’re not overstaffed relative to other venues, but you need higher utilization to break even.
Additionally, the founder (Owen Cotton-Barrat) has stepped back for reasons that seem merited (history of sexual harassment), but a nice aspect of having someone important and busy in charge was that he had a lot less to lose if it was shut down. The castle seems more likely to be self-perpetuating when the decisions are made by people with fewer outside options.
I still view this as fundamentally open phil’s problem to deal with, but it seemed good to give an update.
“I think the drowning child analogy is deceitful, manipulative, and anti-epistemic, so it’s no hardship for me to say we should remove that from recruiting. ”—I’m interested in why you think this?
It puts you in a high SNS activation state, which is inimical to the kind of nuanced math good EA requires
As Minh says, it’s based in avoidance of shame and guilt, which also make people worse at nuanced math.
The full parable is “drowning child in a shallow pond”, and the shallow pond smuggles in a bunch of assumptions that aren’t true for global health and poverty. Such as
“we know what to do”, “we know how to implement it”, and “the downside is known and finite”, which just don’t hold for global health and poverty work. Even if you believe sure fire interventions exist and somehow haven’t been fully funded, the average person’s ability to recognize them is dismal, and many options make things actively worse. The urgency of drowningchildgottasavethemnow makes people worse as distinguishing good charities from bad. The more accurate analogy would be “drowning child in a fast moving river when you don’t know how to swim”.
I think Peter Singer believes this so he’s not being inconsistent, I just think he’s wrong.
“you can fix this with a single action, after which you are done.” Solving poverty for even a single child is a marathon.
“you are the only person who can solve this”. I think there is something good about getting people to feel ownership over the problem and avoiding bystandard effect, but falsely invoking an analogy to a situation where that’s true is not the way to do it.
A single drowning child can be fixed via emergency action. A thousand drowning children scattered across my block, replenishing every day, requires a systemic fix. Maybe a fence, or draining the land. And again, the fight or flight mode suitable for saving a single child in a shallow pond is completely inappropriate for figuring out and implementing the systemic solution.
EA is much more about saying “sorry actively drowning children, I can more good by putting up this fence and preventing future deaths”.
When Singer first made the analogy clothes were much more expensive than they are now, and when I see the argument being made it’s typically towards people who care very little about clothes. What was “you’d make a substantial sacrifice if a child’s life was on the line” has become “you aren’t so petty as to care about your $30 fast fashion shoes, right?”. Just switching the analogy to “ruining your cell phone” would get more of the original intent.
I think this might be a good top level post—would be keen for you more people to see and discuss this point
Do people still care about drowning child analogy? Is it still used in recruiting? I’d feel kind of dumb railing against a point no one actually believed in.
I’m not sure (my active intro cb days were ~2019), but I think it is possibly still in the intro syllabus ? You could add a disclaimer at the top.
I will say I also never use the Drowning Child argument. For several reasons:
I generally don’t think negative emotions like shame and guilt are a good first impression/initial reason to join EA. People tend to distance themselves from sources of guilt. It’s fine to mention the drowning child argument maybe 10-20 minutes in, but I prefer to lead with positive associations.
I prefer to minimise use of thought experiments/hypotheticals in intros, and prefer to use examples relatable to the other person. IMO, thought experiments make the ethical stakes seem too trivial and distant.
What I often do is to figure out what cause areas the other person might relate to based on what they already care about, describe EA as fundamentally “doing good, better” in the sense of getting people to engage more thoughtfully with values they already hold.
Thanks that’s helpful!
Just a quick comment that I strong upvoted this post because of the point about violated expectations in EA recruitment, and disagree voted because it’s missing some important points of why EAs should be concerned about how OP and other EA orgs spend their EA money.
if you have the energy, I’d love to hear your disagreement on open phil or ownership of money.
I feel similarly to Jason and JWS. I don’t disagree with any of the literal statements you made but I think the frame is really off. Perhaps OP benefits from this frame, but I probably disagree with that too.
Another frame: OP has huge amounts of soft and hard power over the EA community. In some ways, it is the de facto head of the EA community. Is this justified? How effective is it? How do they react to requests for information about questionable grants that have predictably negative impacts on the wider EA community? What steps do they take to guard against motivated reasoning when doing things that look like stereotypical examples of motivated reasoning? There are many people who have a stake in these questions.
Thanks, that is interesting and feels like it has conversational hooks I haven’t heard before.
What would it mean to say Open Phil was justified or not justified in being the de facto head of the community? I assume you mean morally justified, since it seems pretty logical on a practical level.
Supposing a large enough contingent of EA decided it was not justified; what then? I don’t think anyone is turning down funding for the hell of it, so giving up open phil money would require a major restructuring. What does that look like? Who drives it? What constitutes large enough?
Briefly in terms of soft and hard power:
Deferring to OP
Example comment about how much some EAs defer to OP even when they know it’s bad reasoning.
OP’s epistemics are seen as the best in EA and jobs there are the most desirable.
The recent thread about OP allocating most of its neartermist budget to FAW and especially its comments shows much reduced deference (or at least more openly taking such positions) among some EAs.
As more critical attention is turned towards OP among EAs, I expect deference will reduce further. I particularly hope EAs pay attention to David Thorstad’s writings on biorisk and OP’s funding of (ahem) low quality papers on that topic (1).
I expect this will continue happening organically, particularly in response to failures and scandals, and the castle played a role in reduced deference.
I agree no one is turning down money willy-nilly, but if we ignore labels, how much OP money and effort actually goes into governance and health for the EA community, rather than recruitment for longtermist jobs?
In other words, I’m not convinced it would require restructuring or just structuring.
A couple of EAs I spoke to about reforms both talked about how huge sums of money are needed to restructure the community and it’s effectively impossible without a megadonor. I didn’t understand where they were coming from. Building and managing a community doesn’t take big sums of money and EA is much richer than most movements and groups.
Why can’t EAs set up a fee-paying society? People could pay annual membership fees and in exchange be part of a body that provided advice for donations, news about popular cause areas and the EA community, a forum, annual meetings, etc. Leadership positions could be decided by elections. I’m just spitballing here.
Of course this depends on what one’s vision for the EA community is.
What do you think?
Why can’t EAs set up a fee-paying society? People could pay annual membership fees and in exchange be part of a body that provided advice for donations, news about popular cause areas and the EA community, a forum, annual meetings, etc. Leadership positions could be decided by elections. I’m just spitballing here.
The math suggests that the meta would look much different in this world. CEA’s proposed budget for 2024 is $31.4MM by itself, about half for events (mostly EAG), about a quarter for groups. There are of course other parts of the meta. There were 3567 respondents to the EA Survey 2022, which could be an overcount or undercount of the number of people who might join a fee-paying society. Only about 60% were full-time employed or self-employed; most of the remainder were students.
Maybe a leaner, more democratic meta would be a good thing—I don’t have a firm opinion on that.
To make sure I understand; this is an answer to “what should EA do if it decides OpenPhil’s power isn’t justified?” And the answer is “defer less, and build a grassroots community structure?”
I’m not sure what distinction you’re pointing at with structure vs. restructure. They both take money that would have to come from somewhere (although we can debate how much money). Maybe you mean OP wouldn’t actively oppose this effort?
To the first: Yup, it’s one answer. I’m interested to hear other ideas too.
Structure vs restructuring: My point was that a lot of the existing community infrastructure OP funds is mislabelled and is closer to a deep recruitment funnel for longtermist jobs rather than infrastructure for the EA community in general. So for the EA community to move away from OP infrastructure wouldn’t require relinquishing as much infrastructure as the labels might suggest.
For example, and this speaks to @Jason’s comment, the Center for Effective Altruism is primarily funded by the OP longtermist team to (as far as I can tell) expand and protect the longtermist ecosystem. It acts and prioritizes accordingly. It is closer to a longtermist talent recruitment agency than a center for effective altruism. EA Globals (impact often measured in connections) are closer to longtermist job career fairs than a global meeting of effective altruists. CEA groups prioritize recruiting people who might apply for and get OP longtermist funding (“highly engaged EAs”).
I think we have a lot of agreement in what we want. I want more community infrastructure to exist, recruiting to be labeled as recruiting, and more people figuring out what they think is right rather than deferring to authorities.
I don’t think any of these need to wait on proving open phil’s power is unjustified. People can just want to do them, and then do them. The cloud of deference might make that harder, but I don’t think arguing about the castle from a position of entitlement makes things better. I think it’s more likely to make things worse.
Acting as if every EA has standing to direct open phil’s money reifies two things I’d rather see weakened. First it reinforces open phil’s power, and promotes deference to it (because arguing with someone implies their approval is necessary). But worse, it reinforces the idea that the deciding body is the EA cloud, and not particular people making their own decisions to do particular things. If open phil doesn’t get to make its own choices without community ratification, who does?
I remember reading a post about a graveyard of projects CEA had sniped from other people and then abandoned. I can’t find that post and it’s a serious accusation so I don’t want to make it without evidence, but if it is true, I consider it an extremely serious problem and betrayal of trust.
yes, everyone has standing to object to negative externalities
narrow is meant to be neutral to positive here. No event can be everything to all people, I think it’s great they made an explicit decision on trade-offs. They maybe could have marketed it more accurately. They’re moving that way now and I wish it had gone farther earlier. But I think even perfectly accurate marketing would have left a lot of people unhappy.
Maybe some people argued from a position of entitlement. I skimmed the comments you linked above and I did not see any entitlement. Perhaps you could point out more specifically what you felt was entitled, although a few comments arguing from entitlement would only move me a little so this may not be worth pursuing.
The bigger disagreement I suspect is between what we think the point of EA and the EA community is. You wrote that you want it to be a weird do-ocracy. Would you like to expand on that?
Maybe you two might consider having this discussion using the new Dialogue feature? I’ve really appreciated both of your perspectives and insights on this discussion, and I think the collaborative back-and-forth your having seems a very good fit for how Dialogues work.
So in this hypothetical, certain functions transfer to the fee-paying society, and certain functions remain funded by OP. That makes sense, although I think the range of what the fee-paying society can do on fees alone may be relatively small. If we estimate 2,140 full fee-payers at $200 each and 1,428 students at $50 each, that’s south of $500K. You’d need a diverse group of EtGers willing to put up $5K-$25K each for this to work, I suspect. I’m not opposed; in fact, my first main post on the Forum was in part about the need for the community to secure independent funding for certain epistemically critical functions. I just want to see people who advocate for a fee-paying society to bite the bullet of how much revenue fees could generate and what functions could be sustained on that revenue. It sounds like you are willing to do so.
But looping back to your main point about “huge amounts of soft and hard power over the EA community” held by OP, how much would change in this hypothetical? OP still funds the bulk of EA, still pays for the “recruitment funnel,” pays the community builders, and sponsors the conferences. I don’t think characterizing the bulk of what CEA et al. do as a “recruitment funnel” for the longtermist ecosystem renders those functions less important as sources of hard and soft power. OP would still be spending ~ $20-$30MM on meta versus perhaps ~ $1-2MM for the fee-paying society.
OP and most current EA community work takes a “Narrow EA” approach. The theory of change is that OP and EA leaders have neglected ideas and need to recruit elites to enact these ideas. Buying castles and funding expensive recruitment funnels is consistent with this strategy.
I am talking about something closer to a big tent EA approach. One vision could be to help small and medium donors in rich countries spend more money more effectively on philanthropy, with a distinctive emphasis on cause neutrality and cause prioritization. This can and probably should be started in a grassroots fashion with little money. Spending millions on fancy conferences and paying undergraduate community builders might be counter to the spirit and goals of this approach.
A fee-paying society is a natural fit for big tent EA and not for narrow EA.
I didn’t know that the huge amounts of power held by OP was my main point! I was trying to use that to explain why EA community members were so invested in the castle. I’m not sure I succeeded, especially since I agree with @Elizabeth’s points that no one needs to wait for permission from OP or anyone else to pursue what they think is right, and the EA community cannot direct OP’s donations.
I personally would love to see a big-tent organization like the one you describe! I think it less-than-likely that the existence of such an organization would have made most of the people who were “so invested in the castle” significantly less so. But there’s no way to test that. I agree that a big-tent organization would bring in other people—not currently involved in EA—who would be unlikely to care much about the castle.
I think an underappreciated part of castlegate is that it fairly easily puts people in an impossible bind.
EA is a complicated morass, but there are a few tenets that are prominent, especially early on. These may be further simplified, especially in people using EA as treatment for their scrupulosity issues. For most of this post I’m going to take that simplified point of view (I’ll mark when we return to my own beliefs).
Two major, major tenets brought up very early in EA are:
You should donate your money to the most impactful possible cause
Some people will additionally internalize “The most impactful in expectation”
GiveWell and OpenPhil have very good judgment.
The natural conclusion of which is that donating GiveWell or OpenPhil-certified causes is a safe and easy way to fulfill your moral duty.
If you’re operating under those assumptions and OpenPhil funds something without making their reasoning legible, there are two possibilities:
The opportunity is bad, which at best means OpenPhil is bad, and at worst means the EA ecosystem is trying to fleece you.
The opportunity is good but you’re not allowed to donate to it, which leaves you in violation of tenet #1.
Both of which are upsetting, and neither of which really got addressed by the discourse.
I don’t think these tenets are correct, or at least they aren’t complete. I think goodharting on a simplified “most possible impact” metric leads very bad places. And I think that OpenPhil isn’t even trying to have “good judgment” in the sense that tenet #2 means it. Even if they weren’t composed of fallible humans, they’re executing a hits-based strategy that means you shouldn’t expect every opportunity to be immediately, legibly good. That’s one reason they don’t ask for money from small donors. Which means OpenPhil funding things that aren’t legibly good doesn’t put me in any sort of bind.
I think it would be harmful to force all of EA to fit the constraints imposed by these two tenets. But I think enough people are under the impression it should that it rises to a level of problem worth addressing, probably through better messaging.
Where does the “ you’re not allowed to donate to it” part of #2 come from?
because it’s not legible, and willingness to donate to illegible things opens you up to scams.
OpenPhil also discourages small donations, I believe specifically because they don’t want to have to justify their decisions to the public, but I think will accept them.
Saying you’re not allowed to donate to the projects is much stronger than either of these things though. E.g. re your 2nd point, nothing is stopping someone from giving top up funding to projects/people that have received OpenPhil funding, and I’m not sure anyone feels like they’re being told they shouldn’t? E.g. the Nonlinear Fund was doing exactly this kind of marginal funding.
I agree they’re allowed to seek out frontier donations, or for that matter give to Open Phil. I believe that this doesn’t feel available/acceptable, on an emotional level, to a meaningful portion of the EA population, who have a strong need for both impact and certainty.
Utilitarianism without strong object-level truthseeking be like
(credit: I found on twitter, uncredited)
Original source is here.
A good summary of pop-Bayesianism failure modes. Garbage in is still garbage out, even if you put the garbage through Bayes theorem.
Salaries at direct work orgs are a frequent topic of discussion, but I’ve never seen those conversations make much progress. People tend to talk past each other- they’re reading words differently (“reasonable”), or have different implicit assumptions that change the interpretation. I think the questions below could resolve a lot of the confusion (although not all of it, and not the underlying question. Highlighting different assumptions doesn’t tell you who’s right, it just lets you focus discussions on the actual disagreements).
Here’s my guess for the important questions. Some of them are contingent- e.g. you might think new grad generalists and experienced domain experts should be paid very differently. Feel free to give as many sets of answers as you want, just be clear which answers lump together, so no one misreads your expert salary as if it was for interns.
What kind of position are you thinking about?
Experienced vs. new grad
Domain expertise vs generalist?
Many outside options vs. few?
Founder vs employee?
What salary are you thinking about?
What living conditions do you expect this salary to buy?
What is your bar for “enough” money? Not keeling over dead? Peak productivity but miserable? Luxury international travel 2x/year?
What percentage of people can reach that state with your suggested salary?
Some things that might make someone’s existence more expensive:
health issues (physical and mental)
Distant family necessitating travel.
Burnout requiring unemployed period.
What do you expect to happen for people who can’t thrive in those conditions?
If you lost your top choice due to insufficient salary, how good do you expect the replacement to be?
What is your counterfactual for the money saved on salary?
People often cite EA salaries as higher than other non-profits, but my understanding is that most non-profits pay pretty badly. Not “badly” as in “low”, but “badly” as in “they expect credentials, hours, and class signals that are literally unaffordable on the salary they pay. The only good employees who stick around for >5 years have their bills paid by a rich spouse or parent.”
So I don’t think that argument in particular holds much water.
Do you have any citations for this claim?
Implict and explicit from https://askamanager.com/ and https://nonprofitaf.com/ (which was much epistemically stronger in its early years)
n = 1 but my wife has worked in non-EA non-profits her whole career, and this is pretty much true. Its mostly women earning poorly at the non-profit, while husbands makes bank at the big corporate.
Where does this idea come from Elizabeth? From my experience (n=10) this argument is incorrect. I know a bunch of people who work in these “badly” paying jobs you talk of who defy your criteria, they don’t have their bills paid for by a rough parent—instead they are content with their work and accept a form of “salary sacrifice” mindset even if they wouldn’t phrase it in those EA terms.
EA doesn’t have a monopoly on altruism, there are plenty of folks out there living simply and working for altruistic causes they believe in even thought it doesn’t pay well and they could be earning way more elsewhere., outside of conventional market forces.
The sense I get reading this is that you feel I’ve insulted your friends, who have made a big sacrifice to do impactful work. That wasn’t my intention and I’m sorry it came across that way. From my perspective, I am respecting the work people do by suggesting they be paid decently.
First, let me take my own advice and specify what I mean by decently: I think people should be able to have kids, have a sub-30 minute commute, live in conditions they don’t find painful (people only live with housemates if they like it, not physically dangerous, outdoor space if they need that to feel good. Any of these may come at at trade off with the others, probably no one gets all of them, but you shouldn’t be starting out from a position where it’s impossible to get reasonable needs met), save for retirement, have cheap vacations, have reasonably priced hobbies, pay their student loans, and maintain their health (meaning both things like healthcare, and things like good food and exercise). If they want to own their home, they shouldn’t be too many years behind their peers in being able to do so.
I think it is both disrespectful to the workers and harmful to the work to say that people don’t deserve these things, or should be willing to sacrifice it for the greater good. Why on earth put the pressure on them to accept less, and not on high-earners to give more? This goes double for orgs that require elite degrees or designer clothes: if you want those class signals, pay for them.
There’s an argument here that low payment screens for mission alignment. I think this effect is real, but is insignificant at the level I’ve laid out.
Hey Elizabeth—just to clarify I don’t think you’ve insulted my friends at all don’t worry about that—I just disagreed from my experience at least that was the situation with most NGO workers like you claimed. I get that you are trying to respect people by pushing for them to be paid more it’s all good.As a small note, I don’t think they have made a “big sacrifice” at all, most wouldn’t say they have made any sacrifice at all. They have traded earning money (which might mean less to them than for other people anyway) for a satisfying job while living a (relatively) simple lifestyle which they believe is healthy for themselves and the planet. Personally I don’t consider this a sacrifice either, just living your best life!
I’m going to leave it here for now (not in a bad way at all) because I suspect our underlying worldviews differ to such a degree here that it may be hard to debate these surface salary and lifestyle issues without first probing at deeper underlying assumptions here about happiness, equality, “deserving” etc., which would take a deeper and longer discussion that might be tricky on a forum back and forth
Not saying I’m not up for discussing these things in general though!
I tested a version of these here, and it worked well. A low-salary advocate revealed a crux they hadn’t before (there is little gap between EA orgs’ first and later choice candidates), and people with relevant data shared it (the gap may be a 50% drop in quality, or not filling the position at all).
This is an interesting model—but what level of analysis do you think is best for answering question 7? One could imagine answering this question on:
the vacancy level at the time of hire decision (I think Bob would be 80% as impactful as the frontrunner, Alice)
the vacancy level at the time of posting (I predict that on average the runner-up candidate will be 80% as the best candidate would be at this org at this point in time)
the position level (similar, but based on all postings for similiar positions, not just this particular vacancy at this point in time)
the occupational field level (e.g., programmer positions in general)
the organizational level (based on all positions at ABC Org; this seems to be implied when an org sets salaries mainly by org-wide algorithm)
the movement-wide level (all EA positions)
the sector-wide level (which could be “all nonprofits,” “all tech-related firms,” etc.)
the economy-wide level.
I can see upsides and downsides to using most of these to set salary. One potential downside is, I think, common to analyses conducted at a less-than-organizational level.
Let’s assume for illustrative purposes that 50% of people should reach the state specified in question 4 with $100K, and that the amount needed is normally distributed with a standard deviation of $20K due to factors described in step five and other factors that make candidates need less money. (The amount needed likely isn’t normally distributed, but one must make sacrifices for a toy model.) Suppose that candidates who cannot reach the question-4 state on the offered salary will decline the position, while candidates who can will accept. (Again, a questionable but simplifying assumption.)
One can calculate, in this simplified model, the percentage of employees who could achieve the state at a specific salary. One can also compute the amount of expected “excess” salary paid (i.e., the amounts that were more than necessary for employees to achieve the desired state).
If the answer to question 7 is that losing the top candidate would have severe impact, one might choose a salary level at which almost all candidates could achieve the question-four state—say, +2.5 SD (i.e., $150K) or even +3 SD ($160K). But this comes at a cost, the employer has likely paid quite a bit of “excess” salary (on average, $50K of the $150K salary will be “excess”).
On the other hand, if there are a number of candidates of almost equivalent quality, it might be rational to set the salary offer at $100K, or even at −0.5 SD ($90K), accepting that the organization will lose a good percent of the candidates as a result.
I suspect you would then have a morale problem with certain employees running the numbers and concluding that they were seen as considerably more replaceable than others who were assigned the same level!
You can fix that by answering question 7 at the organizational or movement levels, averaging the answers for all positions. Suppose that analysis led to the conclusion that your org should offer salaries at this position grade level based on +1 SD ($120K). But you’re still running a 16% risk that the top candidate for the position with no good alternative will decline, while you’re not getting much ROI for the “excess” money spent for certain other positions. You could also just offer $150K to everyone at that level, but that’s harder to justify in the new world of greater funding constraints.
In sum, the mode of analysis that I infer from your questions seems like it would be very helpful when looking at a one-off salary setting exercise, but I’m unsure how well it would scale.
Ambition snowballs/Get ambitious slowly works very well for me, but sonepeople seem to hate it. My first reaction is that these people need to learn to trust themselves more, but today I noticed a reason I might be unusually suited for this method.
two things that keep me from aiming at bigger goals are laziness and fear. Primarily fear of failure, but also of doing uncomfortable things. I can overcome this on the margin by pushing myself (or someone else pushing me), but that takes energy, and the amount of energy never goes down the whole time I’m working. It’s like holding a magnet away from its twin; you can do it, but the minute you stop the system will snap back into place.
But more than I am lazy and fearful, I am easily bored, and hate boredom even more than I hate work or failure. If I hang around my comfort zone long enough I get bored of it and naturally start exploring outside. And that expansion doesn’t take energy; in fact it takes energy to keep me in at that point.
My mom used a really simple example of this on my brother when he was homeschooled (6th grade). He’d had some fairly traumatic experiences in English class and was proving resistant to all her teaching methods. Finally she sat him down in front a computer and told him he had to type continuously for X minutes. It could be literally anything he wanted, including “I can’t think of anything to write about”, he just had to keep his fingers typing the entire time (he could already touch type at this point, mom bribed us with video games until we got to 60WPM). I don’t remember exactly how long this took to work, I think it took her a while to realize she had to ban copy/paste but the moment she did my brother got so bored of typing the same thing that he typed new things, and then education could slip in.
So I’m not worried about being stuck, because I will definitely gnaw my own leg off just to feel something if that happens. And it’s unclear if I can speed up the process by pushing myself outside faster, because leaving comfort zone ring n too early delays getting bored of it (although done judiciously it might speed up the boredom).
I’ll be at EAGxVirtual this weekend. My primary goal is to talk about my work on epistemics and truthseeking within EA, and especially get the kind of feedback that doesn’t happen in public. If you’re interested, you can find me on the usual channels.
Sounds cool, thanks for attending!
I’m pretty sure you can’t have consequentialist arguments for deceptions of allies or self, because consequentialism relies on accurate data. If you’ve blinded yourself then you can have the best utility function in the world and it will do you no good because you’re applying it to gibberish.