When Julian Hazell wrote a previous version of this post, I asked “Can you explain what the bottleneck is to having there be more AI grantmakers? It seems like there are always many bright young people who want to work for EA charities - what prevents the charities that need more grantmakers from hiring some of them?”
He answered that grantmakers needed to be “Relatively senior (6+ years of work experience?) AI-safety-focused people.”
If this is true, I think it changes the calculus enough that first of all, I’m no longer surprised you’re having trouble finding them, and second of all, I think you should make it clear in posts like this (and in the job description/ad). Otherwise, you get the failure mode where EA orgs say “Help, we so desperately need more people,” and then thousands of very smart people put in lots of effort to apply, and the org says “Oh no, not any of you guys”. This is the main reason I no longer respond to posts like this by urging my seemingly-suitable friends to apply for positions.
But also, I’m confused by the failure mode of “because grant investigation capacity is tight, hiring one fewer grantmaker usually means those millions will just sit in an account for another year rather than being deployed to useful ends”. I can name right now five organizations that can absorb an extra $10 million in ways that I think are net positive in expectation from within CG’s world-model (for example, not really thought through, just as proof of concept, Palisade, Apollo, METR, MATS, Tarbell).
Or to put it another way, SFF temp-hires people who have been in the community a while, but maybe don’t have six years AI safety experience, to be temporary grantmakers (disclosure: sometimes including me). They investigate the situation for a month or two and move millions of dollars to various orgs like the five mentioned above based on common-sense models like that they seem good and underfunded. It seems like either this works and is easy (in which case CG should be able to do it) or hard (in which case SFF is net negative and should stop—if this were true I’d like to know it so I could stop supporting them). I still can’t figure out the world-model in which SFF exists and is net positive, CG is begging for more grantmakers, and all these organizations still aren’t fully funded.
I hadn’t seen Julian’s comment and I don’t fully agree with it.
Our needs vary over time and between roles. Some roles ~require several years of prior work experience (the more senior roles, specialist roles like infosec expert or China expert), but generalist grantmaker roles are often suitable for people straight out of undergrad or during/after grad school — e.g. on my team, that describes Trevor, Julian, Nick, and Catherine when we hired them, and they all moved tons of money, contributed to team-level strategy, etc. in their first 18mo. That said, there are only so many early-career folks we can hire before we need additional more-experienced management capacity to take them on.
Re: grantees. I’ll skip commenting on specific groups here, but here are some reasons that orgs can still have funding gaps even if we at CG agree they’re performing well and could probably spend more money productively: (a) they’re trying to diversify funders, and non-CG money is scarcer than CG money, (b) they could spend money in net-positive ways, but not above our ROI bar on the current margin, (c) they acquired ‘room for more funding’ recently, but we funded them not too long ago and can’t afford to investigate new RFMF claims for every grantee every few months, (d) they’re in our queue to investigate for renewal/expansion, but we don’t have staff capacity to do it anytime soon. In some cases the issue might also be “Scott heard they could spend more money but actually they just got approved for a huge CG grant and that info hasn’t propagated yet.”
Re: SFF. I’m not familiar with many of the grantees in the latest batch. Many are previous CG grantees we’re happy to fund generously, and I’m guessing a lot of those were cases of (a)-(d) above. In some cases I know of specific information that makes me bearish, and I wonder if grantmakers with more time and context would make the same decision.
But another point is that probably a majority of the FTE we acquire from this hiring round will (at least for my team) go into a mix of strategy development (for specific areas, like AI character or infosecurity or strategic communications) and “active grantmaking” (identify key gaps from the strategy work, outline and org/project that would address the gap, headhunt a founder for it) which usually takes a ton more time and context than “opportunitistic grantmaking” (responding to applications that come your way, which afaik is ~all SFF grants).
It strikes me that this implies it would be very useful if you were able to share some information with SFF about which bucket various applications fell into.
Maybe you can be more specific going forward, then? It’s easy to interpret your original post as a plea for smart people to help, but it sounds like your audience for these posts are much narrower.
It seems like either this works and is easy (in which case CG should be able to do it) or hard (in which case SFF is net negative and should stop—if this were true I’d like to know it so I could stop supporting them). I still can’t figure out the world-model in which SFF exists and is net positive, CG is begging for more grantmakers, and all these organizations still aren’t fully funded.
To be fair, CG’s grant to OpenAI looks fairly terrible with the benefit of hindsight. That does seem to suggest that AI grantmaking should be done with care, or something of that nature. But I would say that problem would be better fixed through public reflection on what went wrong with the OpenAI grant, as opposed to just trying to monopolize more talent. It would really suck if CG is monopolizing talent for an organization that is fundamentally broken at a structural incentives level, e.g. excessive focus on prestige-maxxing.
He answered that grantmakers needed to be “Relatively senior (6+ years of work experience?) AI-safety-focused people.”
FWIW I don’t actually agree with this interpretation, but I see how my comment could have been confusing.
Let me try to clarify:
I don’t think grantmakers need to be senior. As Luke points out, CG has hired a bunch of people out of undergrad/grad school who ended up contributing pretty quickly.
In any case, I do think senior people are much more of a bottleneck, for reasons including what Luke mentioned here about management capacity being scarce.
I still agree with the claim that there are “many bright young people who want to work for EA charities”, but I should have clarified in my comment that I don’t think being a bright young person is sufficient.
If you look at the “Call to action” section of my post, I list a few characteristics that I think are important for being a good grantmaker. In my experience when hiring grantmakers, there are a lot of people who I would consider bright and impact-focused that nevertheless lack a number of these important traits.
Yes, in fact, what they really need are individuals who can offer profound insights, conceptual frameworks, and solutions—not necessarily as hired employees, but potentially as collaborators.
>Palisade, Apollo, METR, MATS, Tarbell For other readers who thought, “Wait, are those not getting CG funding?” Almost all of those orgs have been funded by CG, most within the last year. Maybe not at the level Scott’s talking about though.
thousands of very smart people put in lots of effort to apply, and the org says “Oh no, not any of you guys”
(I’ll note that I’m qualified to be a grantmaker for the area of human intelligence amplification, which many leaders in AI safety view as a crucial second or third priority behind “stop AI” and “at least try to solve alignment”. But it seems like a waste of my time to apply to be a grantmaker at CG without some indication that they’d be open to this. I did message a couple people hoping for a quick “nah” or “worth applying” but didn’t hear back.)
When Julian Hazell wrote a previous version of this post, I asked “Can you explain what the bottleneck is to having there be more AI grantmakers? It seems like there are always many bright young people who want to work for EA charities - what prevents the charities that need more grantmakers from hiring some of them?”
He answered that grantmakers needed to be “Relatively senior (6+ years of work experience?) AI-safety-focused people.”
If this is true, I think it changes the calculus enough that first of all, I’m no longer surprised you’re having trouble finding them, and second of all, I think you should make it clear in posts like this (and in the job description/ad). Otherwise, you get the failure mode where EA orgs say “Help, we so desperately need more people,” and then thousands of very smart people put in lots of effort to apply, and the org says “Oh no, not any of you guys”. This is the main reason I no longer respond to posts like this by urging my seemingly-suitable friends to apply for positions.
But also, I’m confused by the failure mode of “because grant investigation capacity is tight, hiring one fewer grantmaker usually means those millions will just sit in an account for another year rather than being deployed to useful ends”. I can name right now five organizations that can absorb an extra $10 million in ways that I think are net positive in expectation from within CG’s world-model (for example, not really thought through, just as proof of concept, Palisade, Apollo, METR, MATS, Tarbell).
Or to put it another way, SFF temp-hires people who have been in the community a while, but maybe don’t have six years AI safety experience, to be temporary grantmakers (disclosure: sometimes including me). They investigate the situation for a month or two and move millions of dollars to various orgs like the five mentioned above based on common-sense models like that they seem good and underfunded. It seems like either this works and is easy (in which case CG should be able to do it) or hard (in which case SFF is net negative and should stop—if this were true I’d like to know it so I could stop supporting them). I still can’t figure out the world-model in which SFF exists and is net positive, CG is begging for more grantmakers, and all these organizations still aren’t fully funded.
I hadn’t seen Julian’s comment and I don’t fully agree with it.
Our needs vary over time and between roles. Some roles ~require several years of prior work experience (the more senior roles, specialist roles like infosec expert or China expert), but generalist grantmaker roles are often suitable for people straight out of undergrad or during/after grad school — e.g. on my team, that describes Trevor, Julian, Nick, and Catherine when we hired them, and they all moved tons of money, contributed to team-level strategy, etc. in their first 18mo. That said, there are only so many early-career folks we can hire before we need additional more-experienced management capacity to take them on.
Re: grantees. I’ll skip commenting on specific groups here, but here are some reasons that orgs can still have funding gaps even if we at CG agree they’re performing well and could probably spend more money productively: (a) they’re trying to diversify funders, and non-CG money is scarcer than CG money, (b) they could spend money in net-positive ways, but not above our ROI bar on the current margin, (c) they acquired ‘room for more funding’ recently, but we funded them not too long ago and can’t afford to investigate new RFMF claims for every grantee every few months, (d) they’re in our queue to investigate for renewal/expansion, but we don’t have staff capacity to do it anytime soon. In some cases the issue might also be “Scott heard they could spend more money but actually they just got approved for a huge CG grant and that info hasn’t propagated yet.”
Re: SFF. I’m not familiar with many of the grantees in the latest batch. Many are previous CG grantees we’re happy to fund generously, and I’m guessing a lot of those were cases of (a)-(d) above. In some cases I know of specific information that makes me bearish, and I wonder if grantmakers with more time and context would make the same decision.
But another point is that probably a majority of the FTE we acquire from this hiring round will (at least for my team) go into a mix of strategy development (for specific areas, like AI character or infosecurity or strategic communications) and “active grantmaking” (identify key gaps from the strategy work, outline and org/project that would address the gap, headhunt a founder for it) which usually takes a ton more time and context than “opportunitistic grantmaking” (responding to applications that come your way, which afaik is ~all SFF grants).
Thanks for sharing your thoughts.
It strikes me that this implies it would be very useful if you were able to share some information with SFF about which bucket various applications fell into.
Maybe you can be more specific going forward, then? It’s easy to interpret your original post as a plea for smart people to help, but it sounds like your audience for these posts are much narrower.
To be fair, CG’s grant to OpenAI looks fairly terrible with the benefit of hindsight. That does seem to suggest that AI grantmaking should be done with care, or something of that nature. But I would say that problem would be better fixed through public reflection on what went wrong with the OpenAI grant, as opposed to just trying to monopolize more talent. It would really suck if CG is monopolizing talent for an organization that is fundamentally broken at a structural incentives level, e.g. excessive focus on prestige-maxxing.
FWIW I don’t actually agree with this interpretation, but I see how my comment could have been confusing.
Let me try to clarify:
I don’t think grantmakers need to be senior. As Luke points out, CG has hired a bunch of people out of undergrad/grad school who ended up contributing pretty quickly.
In any case, I do think senior people are much more of a bottleneck, for reasons including what Luke mentioned here about management capacity being scarce.
I still agree with the claim that there are “many bright young people who want to work for EA charities”, but I should have clarified in my comment that I don’t think being a bright young person is sufficient.
If you look at the “Call to action” section of my post, I list a few characteristics that I think are important for being a good grantmaker. In my experience when hiring grantmakers, there are a lot of people who I would consider bright and impact-focused that nevertheless lack a number of these important traits.
Yes, in fact, what they really need are individuals who can offer profound insights, conceptual frameworks, and solutions—not necessarily as hired employees, but potentially as collaborators.
>Palisade, Apollo, METR, MATS, Tarbell
For other readers who thought, “Wait, are those not getting CG funding?” Almost all of those orgs have been funded by CG, most within the last year. Maybe not at the level Scott’s talking about though.
(I’ll note that I’m qualified to be a grantmaker for the area of human intelligence amplification, which many leaders in AI safety view as a crucial second or third priority behind “stop AI” and “at least try to solve alignment”. But it seems like a waste of my time to apply to be a grantmaker at CG without some indication that they’d be open to this. I did message a couple people hoping for a quick “nah” or “worth applying” but didn’t hear back.)
fyi @lukeprog wrote this elsewhere in the comments, though it doesn’t adequately address your (excellent) points