(I work as a grantmaker at CG, but I’m speaking for myself not Luke here)
FWIW, I’d expect ~1400 FTE to be an underestimate for “how many FTE work on AI governance/policy or technical AI safety”.
I don’t have a more up-to-date estimate, sorry. I think Steven’s estimates were off in 2025 (e.g. undercounting lab staff, missing researchers in academia doing AIS-relevant work), and I think his model is weird, so I don’t really trust estimates based on it.
I think that the ratio of grantmakers:people in AIS isn’t that informative for answering the question, “is the number of grantmakers a bottleneck”. (The better ratio is presumably something like, “people who could be working in AI safety/governance if they got funding: grantmakers”).
Lots of our grantmaking, and especially strategy-driven creation of new grantees (Luke’s #3 item on what we fund), pulls talent into AI safety/AI governance, increasing the number of FTEs doing direct work. This is one of our top priority lines of work.
We currently feel more constrained by evaluating or creating new funding opportunities than directing $ to them.
Some kinds of grantmaking are very time-intensive, especially strategy-driven creation of new grantees.
Some renewals can be (and are!) made very quickly, but in other cases we think it’s useful for grantmakers to spend longer working with grantees.
Grantmakers can give feedback on grantees’ plans, encourage them to expand into new areas or be more ambitious, and help new initiatives go better (e.g., by helping them make key hires, advising on object-level decisions, having conversations about their strategy, and connecting them to relevant people). So renewals aren’t merely a passive “accept/reject” process.
One example of this kind of work is my colleague Abbey working closely with Heron AI Security on a shortlist of pressing problems in infosecurity, and Heron’s new programs addressing this.
Since somebody nudged me to reply to this. I didn’t find this reply very convincing.
First, I agree that work pulling people into AI safety is/will be more time-consuming. Other than that, though,
I think that the ratio of grantmakers:people in AIS isn’t that informative for answering the question, “is the number of grantmakers a bottleneck”
This seems off to me. Surely you’d agree that if the ratio were 1:1 or something, we would say “ok, some people who are currently grantmaking need to be doing direct work”. Not everyone can just be funding things.
The better ratio is presumably something like, “people who could be working in AI safety/governance if they got funding: grantmakers”
I mean, maybe that’s another one to consider, but it still feels less relevant. A lot of people think that the denominator in this case is infinite/(nearly) all humans.
We currently feel more constrained by evaluating or creating new funding opportunities than directing $ to them
I actually don’t think this necessarily implies that we should get a lot more people into grantmaking. This could just mean that there is a lot of money available and that grantmakers should move into roles on the ground (direct work)
(I work as a grantmaker at CG, but I’m speaking for myself not Luke here)
FWIW, I’d expect ~1400 FTE to be an underestimate for “how many FTE work on AI governance/policy or technical AI safety”.
I don’t have a more up-to-date estimate, sorry. I think Steven’s estimates were off in 2025 (e.g. undercounting lab staff, missing researchers in academia doing AIS-relevant work), and I think his model is weird, so I don’t really trust estimates based on it.
I think that the ratio of grantmakers:people in AIS isn’t that informative for answering the question, “is the number of grantmakers a bottleneck”. (The better ratio is presumably something like, “people who could be working in AI safety/governance if they got funding: grantmakers”).
Lots of our grantmaking, and especially strategy-driven creation of new grantees (Luke’s #3 item on what we fund), pulls talent into AI safety/AI governance, increasing the number of FTEs doing direct work. This is one of our top priority lines of work.
We currently feel more constrained by evaluating or creating new funding opportunities than directing $ to them.
Some kinds of grantmaking are very time-intensive, especially strategy-driven creation of new grantees.
Some renewals can be (and are!) made very quickly, but in other cases we think it’s useful for grantmakers to spend longer working with grantees.
Grantmakers can give feedback on grantees’ plans, encourage them to expand into new areas or be more ambitious, and help new initiatives go better (e.g., by helping them make key hires, advising on object-level decisions, having conversations about their strategy, and connecting them to relevant people). So renewals aren’t merely a passive “accept/reject” process.
One example of this kind of work is my colleague Abbey working closely with Heron AI Security on a shortlist of pressing problems in infosecurity, and Heron’s new programs addressing this.
Since somebody nudged me to reply to this. I didn’t find this reply very convincing.
First, I agree that work pulling people into AI safety is/will be more time-consuming. Other than that, though,
This seems off to me. Surely you’d agree that if the ratio were 1:1 or something, we would say “ok, some people who are currently grantmaking need to be doing direct work”. Not everyone can just be funding things.
I mean, maybe that’s another one to consider, but it still feels less relevant. A lot of people think that the denominator in this case is infinite/(nearly) all humans.
I actually don’t think this necessarily implies that we should get a lot more people into grantmaking. This could just mean that there is a lot of money available and that grantmakers should move into roles on the ground (direct work)