I’m CEO at 80,000 Hours. Before that, I was in various roles at 80k, most recently Chief of Staff.
I was also the initial program officer for global catastrophic risk at Open Philanthropy. Comments here are my own views only, not my present or past employers’, unless otherwise specified.
I think the people responsible for EA Global admissions (including Amy Labenz, Eli Nathan, and others) have added a bunch of value to me over the years by making it more likely that a conversation or meeting with somebody at EA Global who I don’t already know will end up being productive. Making admissions decisions at EAG (and being the public face of an exclusive admissions policy) sounds like a really thankless job and I know a bunch of the people involved end up having to make decisions that make them pretty sad because they think it’s best for the world. I mostly just wanted to express some appreciation for them and to mention that I’ve benefitted from it since it feels uncomfortable to say out loud so is probably under expressed.
One positive effect of selective admissions that I don’t often see discussed is that it makes me more likely to take meetings with folks I don’t already know. I’d guess that this increases the accessibility of EA leaders to a bunch of folks in the community.
Fwiw, I’ve sometimes gotten overambitious with the number of meetings I take at EAG and ended up socially exhausted enough to be noticeably less productive for several days afterwards. This is a big enough cost that I’ve skipped some years. So, I think in the past I’ve probably been on the margin where if the people at EAG had not been selected for being folks I could be helpful to, I’d have been less likely to go.
I’m curious whether there’s any answer AI experts could have given that would be a reasonably big update for you.
For example is there any level of consensus against ~AGI by 2070 (or some other date) that would be strong enough to move your forecast by 10 percentage points?
I definitely agree that takeaway would be a mistake. I think my view is more like “if the specifics of what MT says on a particular topic don’t feel like they really fit your organisation, you should not feel bound to them. Especially if you’re a small organisation with an unusual culture or if their advice seems to clash with conventional wisdom from other sources, especially in silicon valley.
I’d endorse their book as useful for managers at any org. A lot of the basic takeaways (especially having consistent one on ones) seem pretty robust and it would be surprising if you shouldn’t do them at all.
Agree with a lot of this post. I lived in DC from 2008-2010 and various short periods before and after and overall I liked it (though I’d probably like it a bit less today and expect a lot of EAs to like it less than I did).
The features of DC that most affected me: -DC felt like a company town. This had advantages. I liked having tons of friends who were think tank analysts or worked on the Hill and were trying to change the world (though I suspect polarization has made the vibe a bit worse). It also had disadvantages. Relative to NYC (which I knew best at the time) I knew relatively few people living in DC because they wanted to make DC great and this meant things like a worse music scene (despite the fact that I grew up on DC punk music). -Lots of people, especially young people, only stay for a couple of years so it was hard to maintain a friend group. I think this was a big deal. -DC is small relative to a place like NY. Overall this felt like a disadvantage to me though I expect it would be a feature to some other people. DC felt like more of a bubble and there were fewer places to explore. There was a concert I’d be interested in ~once a week instead of a couple per night. On the other hand, several houses full of friends and co-workers lived within a five minute walk which was great. That said, it’s still one of the biggest metro areas in the US. -I thought it was cool/exciting to live in a city where policy and politics were happening (though I think I’d enjoy less today). -I think there were some disadvantages to everybody being very networky and the culture being kind of conservative.
“I don’t think they would put out material that fails to apply to them.”
I think we mostly agree but I don’t think that’s necessarily true. My impression is that they mainly study what’s useful to their clients and from what I can glean from their book, those clients are mostly big and corporate. I think they might fall outside of their main target audience.
+1 to Paul grahams essays.
[Unfortunately didn’t have time to read this whole post but thought it was worth chiming in with a narrow point.]
I like Manager Tools and have recommended it but my impression is that some of their advice is better optimized for big, somewhat corporate organizations than for small startups and small nonprofits with an unusual amount of trust among staff. I’d usually recommend somebody pair MT with a source of advice targeted at startups (e.g. CEO Within though the topics only partially overlap) so you know when the advice differs and can pick between them.
Just making sure you saw Eli Nathan’s comment saying that this year plus next year they didn’t/won’t hit venue capacity so you’re not taking anybody’s spot
No worries!
tl;dr I wouldn’t put too much weight on my tweet saying I think I probably wouldn’t be working on x-risk if I knew the world would end in 1,000 years and I don’t think my (wild) guess at the tractability of x-risk mitigation is particularly pessimistic.
***
Nice post. I agree with the overall message of as well as much of Ben’s comment on it. In particular, I think emphasizing the significance of future generations, and not just reducing x-risk, might end up as a crux for how much you care about: a) how much an intervention reduces x-risk v. GCRs that are unlikely to (directly?) lead to existential catastrophe; b) whether civilization just manages to avoid x-risk v. ends up on track to flourish as much as possible and last a lot longer than (e.g.) the typical mammalian species.
***
That said, I mostly came here to quickly caution against putting too much weight on this:
Alyssa Vance’s tweet about whether the longtermism debate is academic
Howie’s response is interesting to me, as it implies a fairly pessimistic assessment of tractability of x-risks given that 1,000 years would shift the calculations presented here by over an OOM (>10 generations).
That’s mostly for the general reason that I put approximately one Reply Tweet’s worth of effort into it. But here are some specific reasons not to put too much weight on it and also that I don’t think it implies a particularly pessimistic assessment of the tractability of x-risk.[1]
I’m not sure I endorse the Tweet on reflection mostly because of the next point.
I’m not sure if my tweet was accounting for the (expected) size of future generations. A claim I’d feel better about would be “I probably wouldn’t be working on x-risk reduction if I knew there would only be ~10X more beings in the future than are alive today or if I thought the value of future generations was only ~10X more than the present.” My views on the importance of the next 1,000 years depend a lot on whether generations in the coming century are order(s) of magnitude bigger than the current generation (which seems possible if there’s lots of morally relevant digital minds). [2]
I haven’t thought hard about this but I think my estimates of the cost-effectiveness of the top non-longtermist opportunities are probably higher than implied by your table.
I think I put more weight on the badness of being in a factory farm and (probably?) the significance of chickens than implied by Thomas’s estimate.
I think the very best global health interventions are probably more leveraged than giving to GiveWell.
I find animal welfare and global poverty more intuitively motivating than working on x-risk, so the case for working on x-risk had to be pretty strong to get me to spend my career on it. (Partly for reasons I endorse, partly for reasons I don’t.)
I think the experience I had at the time I switched the focus of my career was probably more relevant to global health and animal welfare than x-risk reduction.
My claim was about what I would in fact be doing, not about what I ought to be doing.
[1] Actual view: wildly uncertain and it’s been a while since I last thought about this but something like the numbers from Ben’s newsletter or what’s implied by the 0.01% fund seem within the realm of plausibility to me. Note that, as Ben says, this is my guess for the marginal dollar. I’d guess the cost effectiveness of the average dollar is higher and I might say something different if you caught me on a different day.
[2] Otoh, conditional on the world ending in 1,000 years maybe it’s a lot less likely that we ended up with lots of digital minds?
I agree with Caleb that theoretical AIS, infinite ethics, and rationality techniques don’t currently seem to be overprioritized. I don’t think there are all that many people working full-time on theoretical AIS (I would have guessed less than 20). I’d guess less than 1 FTE on infinite ethics. And not a ton on rationality, either.
Maybe your point is more about academic or theoretical research in general? I think FHI and MIRI have both gotten smaller over the last couple of years and CSER’s work seems less theoretical. But you might still think there’s too much overall?
My impression is that there’s much more of a supply of empirical AI safety research and, maybe, theoretical AI safety research written by part-time researchers on LessWrong. My impression is that this isn’t the kind of thing you’re talking about though.
There’s a nearby claim I agree with, which is that object level work on specific cause areas seems undervalued relative to “meta” work.
Academic-like research into interesting areas of AI risk is far easier to get funded by many funders than direct research into, say, vaccine production pipelines.
My guess is that this has less to do with valuing theory or interestingness over practical work, and more to do with funders prioritizing AI over bio. Curious if you disagree.
Know that other people have gone through the disillusionment pipeline, including (especially!) very smart, dedicated, caring, independent-minded people who felt strong affinity for EA. Including people who you may have seen give talks at EA Global or who have held prestigious jobs at EA orgs.
Also, I think even people like this who haven’t gone through the disillusionment pipeline are often a lot more uncertain about many (though not all) things than most newcomers would guess.
Thanks for writing this post. I think it improved my understanding of this phenomenon and I’ve recommended reading it to others.
Hopefully this doesn’t feel nitpicky but if you’d be up for sharing, I’d be pretty interested in roughly how many people you’re thinking of:
“I know at least a handful of people who have experienced this (and I’m sure there are many more I don’t know)—people who I think are incredibly smart, thoughtful, caring, and hard-working, as well as being independent thinkers. In other words, exactly the kind of people EA needs. Typically, they throw themselves into EA, invest years of their life and tons of their energy into the movement, but gradually become disillusioned and then fade away without having the energy or motivation to articulate why.”
I’m just wondering whether I should update toward this being much more prevalent than I already thought it was.
“My best guess is that I don’t think we would have a strong connection to Hanson without Eliezer”
Fwiw, I found Eliezer through Robin Hanson.
111% of deaths averted?
Agree they have a bunch of very obnoxious business practices. Just fyi you can change a seeing so nobody can see whose pages you look at.
I think Open Philanthropy has done some of this. For example:
The Open Philanthropy technical reports I’ve relied on have had significant external expert review. Machine learning researchers reviewed Bio Anchors; neuroscientists reviewed Brain Computation; economists reviewed Explosive Growth; academics focused on relevant topics in uncertainty and/or probability reviewed Semi-informative Priors.2 (Some of these reviews had significant points of disagreement, but none of these points seemed to be cases where the reports contradicted a clear consensus of experts or literature.)
Was this in the deleted tweet? The tweet I see is just him tagging someone with an exclamation point. I don’t really think it would be accurate to characterise that as “Torres supports the ‘voluntary human extinction’ movement”
Yeah that does sell me a bit more on delegating choice.
I think that’s an improvement though “delegating” sounds a bit formal and it’s usually the authority doing the delegating. Would “deferring on views” vs “deferring on decisions” get what you want?
Thanks for this! It was really useful and will save 80,000 Hours a lot of time.