I currently lead EA funds.
Before that, I worked on improving epistemics in the EA community at CEA (as a contractor), as a research assistant at the Global Priorities Institute, on community building, and Global Health Policy.
Unless explicitly stated otherwise, opinions are my own, not my employer’s.
You can give me positive and negative feedback here.
I’m very sorry that you had such a bad experience here. Whilst I would disagree with some of the details here I do think that our communication was worse than I would have liked and I am very sorry for any hardship that you experienced. It sounds like a stressful process which could have been made much better if we had communicated more often and more quickly.
In my last email (March 4th), I said that we were exploring making this grant, but it’s legally challenging. Grants for mental health support are complicated, in general, as we have to show that there is a pure public benefit. We have an open thread with our legal counsel, and I’m cautiously optimistic about getting a decision on this relatively soon.
In general, I don’t think I made promises or hard commitments to get back in a certain time frame; instead, I said that we aim to get back by a certain time. I believe I am at fault for not making this distinction appropriately clear, and I am upset that this mismatch of expectations resulted in hardship.
I’d also like to quickly clarify that many of the errors here were mine (as opposed to the wider EA Funds team). I should have been more realistic about the time frame for a grant of this nature.
I think I agree with the general thrust of your post (that mental health may deserve more attention amongst neartermist EAs), but I don’t think the anecdote you chose highlights much of a tension.
> I asked them how they could be so sceptical of mental health as a global priority when they had literally just been talking to me about it as a very serious issue for EAs.
I am excited about improving the mental health of EAs, primarily because I think that many EAs are doing valuable work that improves the lives of others and good mental health is going to help them be more productive (I do also care about EAs being happy as much as I care about anyone being happy, but I expect that value produced from this to be much less that the value produced from the EAs actions).
I care much less about the productivity benefits that we’d see from improving the mental health of people outside of the EA community (although of course I do think their mental health matters for other reasons).
So the above claim seems pretty reasonable to me.
As an illustration, I can care about EAs having good laptops much more than I care about random people having good laptops, I am much more sceptical about giving random people good laptops producing impact than giving EAs good laptops.
Some quick thoughts on AI consciousness work, I may write up something more rigorous later.
Normally when people have criticisms of the EA movement they talk about its culture or point at community health concerns.
I think aspects of EA that make me more sad is that there seems to be a few extremely important issues on an impartial welfarist view that don’t seem to get much attention at all, despite having been identified at some point by some EAs. I do think that ea has done a decent job of pointing at the most important issues relative to basically every other social movement that I’m aware of but I’m going to complain about one of it’s shortcomings anyway.
It looks to me like we could build advanced ai systems in the next few years and in most worlds we have little idea of what’s actually going on inside them. The systems may tell us they are conscious, or say that they don’t like the tasks we tell them to do but right now we can’t really trust their self reports. There’ll be a clear economic incentive to ignore self reports that create a moral obligation to using the systems in less useful/efficient ways. I expect the number of deployed systems to be very large and that it’ll be plausible that we lock in the suffering of these systems in a similar way to factory farming. I think there are stronger arguments for the topic’s importance that I won’t dive into right now but the simplest case is just the “big if true-ness” of this area seems very high.
My impression is that our wider society and community is not orienting in a sane way to this topic. I don’t remember ever coming across a junior EA seriously considering directing their career to work in this area. 80k has a podcast with Rob Long and a very brief problem profile (that seems kind of reasonable), ai consciousness (iirc) doesn’t feature in ea virtual programs or any intro fellowship that I’m aware of, there haven’t been many (or any?) talks about it at eag in the last year. I do think that most organisations could turn around and ask “well what concrete action do you actually want our audience to take” and my answers are kind of vague and unsatisfying right now—I think we were at a similar point with alignment a few years ago and my impression is that it had to be on the communities mind for a while before we were able to pour substantial resources into it (though the field of alignment feels pretty sub-optimal to me and I’m interested in working out how to do a better job this time round).
I get that there aren’t shovel ready directions to push people to work on right now, but in so far as our community and its organisations brand themselves substantially as the groups identifying and prioritising the worlds most pressing problems it sure does feel to me like more people should have this topic on their minds.
There are some people I know of dedicating some of their resources to making progress in this area, and I am pretty optimistic about the people involved—the ones that I know of seem especially smart and thoughtful.
I don’t want all of the EA to jump into this rn, and I’m optimistic about having a research agenda in this space that I’m excited about and maybe even a vague plan about what one might do about all this by the end of this year—after which I think we’ll be better positioned to do field building. I am excited about people who feel especially well placed moving into this area—in particular people with some familiarity with both mainstream theories of consciousness and ml research (particularly designing and running empirical experiments). Feel free to reach out to me or apply for funding at the ltff.
I’m excited that Zach is stepping into this role. Zach seems substantially better than my expectations for the new CEA CEO, and I expect the CEO hiring committee + Ben + the EV board had a lot to do with that (and probably lots of other people at CEA that I don’t know about)!
Most CEA users and EA community members probably don’t know Zach, so I thought it would be helpful to share some of my thoughts on them and this position (though I don’t know Zach especially well, and these are just quick subjective takes). Thanks to @Ben_West for the nudge to do this.
Quick takes on my impression of Zach and his fit for this role
Zach seems very strong on typical management consultant skills, e.g. communication skills, professionalism, creative problem-solving in typical professional environments, and having difficult conversations.
One aptitude that I would bet Zach is strong on that I think is very neglected in EA organisations is developing and mentoring mid-level staff. Many EA orgs have bright, agentic, competent young people in fairly high-responsibility roles. While you can learn a lot from these roles that you might not learn in a traditional organisation, I worry (particularly for myself) that there might be a lot of gains from being in a more typical management/organisational structure. I’m pretty excited for people like Jessica McCurdy, who runs the EA groups team, to have a solid manager like Zach (Edit: I didn’t mean to imply here that leadership at the time of writing weren’t providing good management; just that relative to people I thought might end up in this role, I expect ~Zach to be quite strong on managing mid-level people). I’d guess that CEA will become a substantially more attractive place to work (for senior people) because of Zach.
While I don’t have much insight into Zach’s vision for CEA, I remember thinking that Zach seemed sharp, thoughtful, and reasonable in conversations about EA and CEA. I also got the sense that he has thought about some of EA’s more intellectual/philosophical parts—it’s imo fairly rare to find people who can both do the philosophy part of EA and the execution part of EA, but both parts seem important for this role.
I do have some reservations about Zach entering this role related to the professional relationships/responsibilities that Zach holds.[1]
Zach previously worked at Open Phil; this relationship seems particularly important for the future of CEA as that is where they get most of their funding from. I think it’s reasonable for people to be increasingly concerned about the epistemic influence of Open Phil on EA, and having an ex-senior Open Phil employee, who is probably still good friends with Open Phil leadership, meaningfully reduces the epistemic independence of CEA. It also could make it hard for CEA or the EV board to push Zach out if he turns out to be a bad fit for CEA—and given CEA’s history, I think this is worth bearing in mind (though I’d guess overall that Zach is net positive for CEA governance).
Zach is on Anthropic’s Long-Term Benefit Trust. It’s not super clear what this means, particularly in light of recent events with the Open AI board, but I am a bit concerned about the way that EA views Anthropic, and that the CEO of CEA being affiliated with Anthropic could make it more difficult for people within EA to speak out against Anthropic. Managing social/professional relationships within EA is challenging,[2] and I’d guess that overall this cost is worth paying to have a CEO of Zach’s calibre—but I think it’s a meaningful cost that people should be tracking.
In an ideal world, I would prefer that Zach [3]was less strongly connected to Open Phil (weak confidence) and also less strongly connected to Anthropic (medium/high confidence).
CEAs future strategy
I don’t have many thoughts at this time on what changes to CEA’s strategy I’d like to see. To me it seems like CEA is at a bit of a crossroads in both a strategic and organisational sense:
Organisational—various teams seem to want to spin out and be their own thing where the project leads get more of a say over the direction of their project, and they have less baggage from being part of CEA. I’d be excited about Zach working out whether this is the right move for individual projects and increasing the value that projects get from being part of CEA.
Strategic—many CEA staff seem to have become most concerned about risk from AI and naturally want their work to focus on this topic. At the same time, it seems like relatively little money is available to the non-AI parts of EA for meta work.
I am not sure what (if anything) should change on the funding side, but on the CEA side I’d be excited about:
Zach/CEA figuring out a coherent vision for CEA that is transparent about its motivations and outcomes, CEA staff are excited about and doesn’t leave various parts of the EA community feeling isolated.
Zach figuring out how to increase the value that CEA’s projects get from being part of CEA, or helping them spin out if that’s what they want to do.
Zach/CEA figuring out how to leverage and improve the CEA brand so that it doesn’t restrict the actions of various projects (and ideally is an asset) and doesn’t create negative externalities for organisations outside of CEA.
FYI, I didn’t run this by Zach, but as it’s not really a criticism that could affect his reputation and is mostly just pointing at publicly available information, it didn’t seem warranted to me.
For example, I live with two people who work at Anthropic, and in general, living with people probably has substantive epistemic effects.
Edit 2024-Apr-16: I meant the CEO of EV, as opposed to Zach specifically.
I agree in principle with this argument but ….
Here are some of my concrete candidates for most interesting work: infinite ethics, theoretical AI safety, rationality techniques, and writing high-level critiques of EA[1].
I really don’t think there are many people at all putting substantial resources into any of these areas.
Theoretical AIS work seems really important and depending on your definition of ‘theoretical’ there are probably 20-100 FTE working on this per year. I would happily have at least 10-100x this amount of work if the level of quality can be at least maintained.
Who is actually working on infinite ethics?? I don’t think this has pulled a lot of EA talent, I’d guess fewer than 5 FTE.
Rat techniques—CFAR basically doesn’t exist and has fewer than 5 FTE, I’d guess most of this work is done on the side now but it certainly does not seem to be over-done by EAs.
Criticisms—I think this is more reasonable, I don’t think lots of time is spent here but a lot of attention is and I generally haven’t found criticism posts to be particularly insight generating (although this one is more interesting).
My suspicion is that this “people are generally overprioritising interesting things” claim sounds nice but won’t hold up to empirical investigation (at least on my world view).
One thing to note is that at the end of January, we rejected the original grant (which I believed that we wouldn’t be able to show a clear public benefit for), and then said we were interested in a different version of the grant that seemed more defensible to me (subject to legal review). Since then, we have been working out whether we can make this alternate grant.
I didn’t realise that Igor stopped taking clients completely, and I regret that I didn’t make a stronger effort to understand the consequences of the unclear situation whilst we tried to understand the legal implications of making the grant.
The 80k job board has too much variance.
(Quickly written, will probably edit at some point in future)
Jobs in the main 80k job board can range from (in my estimation) negligible value to amongst the best opportunities I’m aware of. I have also seen a few jobs that I think are probably actively harmful (e.g. token alignment orgs who are trying to build AGI where the founders haven’t thought carefully about alignment—based on my conversations with them).
I think a helpful orientation towards jobs on the jobs board is, at least one person with EA values who happens to work at 80k thinks it’s worth signal boosting. And NOT EA/80k endorses all of these jobs without a lot more thought from potential applicants.
Jobs are also on the board for a few different reasons e.g. building career cap Vs direct impact Vs … And there’s isn’t lots of info about why the job is there in the first place.
I think 80k does try to to give more of this vibe than people get. I don’t mean to imply that they are falling short in an obvious way.
I also think that the jobs board is more influential than 80k thinks. Explicit endorsements of organisations from core EA orgs are pretty rare and I think they’d be surprised how many young EAs overupdate on their suggestions (but only medium confidence about it being pretty influential).
My concrete improvement would be to seperate jobs into a few different boards to the degree that they endorse the organisation.
One thing I find slightly frustrating is the response that I have heard from 80k staff to this is that the main reason they don’t do this is around managing relationships with the organisations (which could be valid). Idk if it’s the right call but I think it’s a little sus, I think people are too quick to jump to the nice thing that doesn’t make them feel uncomfortable over the impact maximising thing (pin to write more about this in future).
One error that I think I’m making is criticising an org for doing a thing that is probably much better than not doing the thing even if it think it’s leaving some value on the table, I think that this is kind of unhealthy and incentives inaction. I’m not sure what to do about this other than flag that I think 80k is great as is most of the stuff they do and I’d rather orgs had a policy of occasionally producing things that I feel moderately about if this helps them do a bunch of cool stuff, than underperform and not get much done (pin to write more about this in future).
I think that you could have emailed the Atlas fellowship at the email listed here with this question. I suspect this would have better achieved your intention of not causing controversy.
I feel a bit confused about how much I should be donating.
On the one hand there’s just a straight forward case that donating could help many sentient beings to a greater degree than it helps me. On the other hand, donating 10% for me feels like it’s coming from a place of fitting in with the EA consensus, gaining a certain kind of status and feeling good rather than believing it’s the best thing for me to do.
I’m also confused about whether I’m already donating a substantial fraction of my income.
I’m pretty confident that I’m taking at least a 10% pay-cut in my current role. If nothing else my salary right now is not adjusted for inflation which was ~8% last year so it feels like I’m at least underpaid by that amount (though it’s possible they were overpaying me before). Many of my friends earn more than twice as much as I do and I think if I negotiated hard for a 100% salary increase the board would likely comply.
So how much of my lost salary should I consider to be a donation? I think numbers between 0% and 100% are plausible. −50% also isn’t insane to me as my salary does funge with other peoples donations to charities.
One solution is that I should just negotiate for my salary from a non-altruistic perspective, and then decide how much I want to donate back to my organisation after that. This seems a bit inefficient though and I think we should be able to do better.
One reason I don’t donate ~50% of my salary is that I genuinely believe it’s more cost-effective for me to build runway than donate right now. I quite like the idea of discussing this with someone who strongly disagrees with me and I admire and see if they come round to my position. It feels a bit too easy to find reasons not to give, and I’m very aware of my own selfishness in many parts of my life.
In fairness, I was prevented from posting a bunch of stuff and spent a long time (like tens of hours) workshopping text until legal council were happy with it. At least in one case I didn’t end up posting the thing because it didn’t feel useful after the various edits and it had been by then a long time since the event the post was about.
I think in hindsight the response (with the information I think the board had) was probably reasonable—but if similar actions were to be taken by EV when writing a post about Anthropic I’d be pretty upset about that. I wouldn’t use the word censoring in the real ftx case—but idk in the fictional Anthropic case I might?
Thanks for writing this post. It looks like it took a lot of effort that could have been spent on much more enjoyable activities, including your mainline work.
This isn’t a comment on the accuracy of the post (though it was a moderate update for me). I could imagine nonlinear providing compelling counter evidence over the next few days and I’d of course try to correct my beliefs in light of new evidence.
Posts like this one are a public good. I don’t think anyone is particularly incentivised to write them, and they seem pretty uncomfortable and effortful, but I believe they serve an important function in the community by helping to root out harmful actors and disincentivising harmful acts in the first place.