Quantifying the impact of grantmaking career paths
How impactful is it to try to become a grantmaker focused on pressing world problems? Encouraged by Nuno’s tweet, I spent a couple of hours trying to find out.
TL;DR
I estimate the expected x-risk basis points reduced for the prospective marginal grantmaker to be 0.12 (with enormous error bars and caveats).
Background
Epistemic status: I have zero grantmaking experience and haven’t even double-checked my numbers; please do not use this post for anything that relies on it not containing serious errors.
My estimand is something like: “expected x-risk basis points reduced for the prospective marginal grantmaker doing their best to benefit the most pressing problems at an Open Philanthropy-equivalent organization.”
BOTEC
Chance of getting a job
I make guesses of
the number of employers that might be possible to work at,
the Open Philanthropy-equivalentness of employers, and
the probability of acceptance for any particular job application.
Multiplying these factors together gives me a probability of succeeding in getting an Open Philanthropy-equivalent grantmaking job offer at (at least) one of the Open Philanthropy-equivalent organizations of 7% (on average). Seems not-crazy.
Impact from grantmaking
I used recent data from these charts to guess the total longtermist grants given per year by Open Philanthropy-equivalent organizations going forwards.
The fraction of grants a prospective grantmaker is responsible for is given by the reciprocal of my guess of the number of grantmaker-equivalents at Open Philanthropy-equivalent organizations. (I apologize for phrasing...)
Only a fraction of the grants that a grantmaker is responsible for can be counterfactually attributed to them — some grants would make for such obvious decisions that the counterfactual grantmaker would have acted identically. I convert the vibes from this post into a fraction of grants worth deliberating over (on average, 13%).
I set the ratio of skill vs. a replacement grantmaker to be mean 1.2 with mass either side of 1. This reflects the fact grantmakers may be better or worse than their counterfactual replacement, but also that on average their being hired implies that they are expected to perform better.
Taken together, I get a mean estimate of $5.7m for the Open Philanthropy-equivalent resources counterfactually moved by grantmaking activities of the prospective marginal grantmaker, conditional on job offer.
Impact from other paths
Linch claims that some fraction of grantmakers’ impact comes from things that look like ‘improving all grantmaking processes at their organization,’ ‘improving the work/selection of their grantees,’ and other paths that I did not attempt to categorize. I add the categorized paths in to my estimate of impact.
My mean estimate for the Open Philanthropy-equivalent impact from non-grantmaking activities sums to around $30m (equivalent). ~70% of this comes from improving all grantmaking, ~30% from improving grantees.
Units
In order to calculate impact for the whole career path, I multiply annual impact by the number of years employed as a grantmaker.
Finally, to convert impact into terms of x-risk basis points, I divide by USD per x-risk basis point. Estimates for this quantity come from this comment.
Putting everything together
All considered, I get an estimate of the x-risk basis points reduced by the prospective marginal grantmaker of 0.12 in expectation. There is a ~27% chance of having a negative counterfactual impact (vs. replacement grantmaker) and a ~5% chance of having an impact greater than expected.
For full details, see my code in the appendix.
Limitations
There might be over- or under-counting issues. For instance, I guess that the average number of years employed will be 11. This corresponds to me excluding the impact of the grantmaker’s post-grantmaking career from my definition of career path. Several possible implementation choices seem valid here.
It goes without saying that personal factors (chance of getting jobs, grantmaker skill, etc.) could change the numbers significantly. As could cause area, changes in funding landscape, changes in Open Philanthropy-equivalentness of other organizations, etc..
I’ve ignored the Shapley values debate.
Several of the distributions that I have made guesses for could quite easily come from actual data instead — e.g. the chance of getting any particular grantmaking job, and Open Philanthropy grant amounts.
Appendix
Here’s the code! You can mess around with it in squiggle playground.
// chance of success
n_organisations_available = truncateLeft((3 to 30) - 1, 0)
organisation_op_equivalent = 0.01 to 1
n_op_equivalent = 1 + (n_organisations_available * organisation_op_equivalent)
probability_get_particular_job = truncate(0.001 to 0.1, 0, 1)
probability_get_any_job = 1 - (1 - probability_get_particular_job)^n_op_equivalent
// share of grants receive credit for
op_annual_grants = mx(normal(200M, 50M), 100M to 1B, 30M to 3B, [0.5, 0.2, 0.3])
fraction_of_annual_grants = 1 / truncateLeft(mx([1 to 100, 10 to 100, 1 to 1000], [0.2, 0.5, 0.3]), 1)
// impact from grantmaking
fraction_worth_deliberating = truncate(normal(0.1, 0.1), 0, 1)
skill_vs_replacement_grantmaker = 0.3 to 3
impact_grantmaking = op_annual_grants *
fraction_of_annual_grants *
fraction_worth_deliberating *
skill_vs_replacement_grantmaker
// other paths to impact
improve_grantmaking_process = op_annual_grants *
(mx([normal(1, 0.003), 1 to 3, 1 to 10], [0.98, 0.01, 0.01]) - 1)
improve_grantees = op_annual_grants *
fraction_of_annual_grants *
(mx(normal(1, 0.003), 0.3 to 3, 0.3 to 10, [0.7, 0.2, 0.1]) - 1)
impact_other_paths = improve_grantmaking_process + improve_grantees
// expected years employed, ignoring exogenous x-risk
years_employed = truncateRight(1 to 50, 60)
// conversion to basis points of existential risk reduced
usd_per_basis_point_xrisk = truncate(mx(normal(300M, 300M), 30M to 1B), 30M, 3B)
// final estimate
impact = probability_get_any_job *
(impact_grantmaking + impact_other_paths) *
years_employed /
usd_per_basis_point_xrisk
It would be interesting to see estimates that compare the impact of grantmaking with the impact of researching or entrepreneurship. Because many factors are hard to estimate at an absolute level, but could be held fixed across those cases, such as “your overall level of talent”, “the value of X units of x-risk reduction research”, and so on. And besides, a comparison would be more decision-relevant for people.
I think this would be valuable, but doing this across various careers might be too much for any one person. But posts like this are a good first step which makes that later comparison step easier.
Agreed! Especially since Charity Entrepreneurship is starting efforts in health security, and I’d expect increasing longtermist entrepreneurship efforts over the next few years.
Love seeing this type of thinking on the Forum. Thanks for writing.
Thanks for the post! In addition to the caveats you mention, I’m a bit worried that many of your numbers came from me, especially as I’m a fairly junior (and also part-time) grantmaker. Not that I’m saying I have much better numbers now (otherwise I’d just update towards them). But just wanted to flag a potential issue with deference cascades.
Thank you for flagging!
For interested onlookers, note that my use of Linch’s numbers leads to:
~6x increase in grantmaker impact (due to non-grantmaking activities). Probably tonnes of room for disagreement.
~0.13x multiplier due to the proportion of grants where grantmakers can make a counterfactual difference. Probably not much more than 0.5-1 orders of magnitude available for disagreement.
~300M $/x-risk basis point estimate. Probably not much more than 1-2 order of magnitude available for disagreement.
Ok, so here is how I am thinking about this.
What Joel is doing
You can divide your estimate between:
The impact per year of an Open Philanthropy-equivalent grantmaking role.
The chance you can get one such, and the number of years.
To estimate 1., you look at:
The number of funds you would be responsible for
The share of the impact that those funds cause that can be attributed to you, e.g., vs your counterfactual
Other benefits from being a good grantmaker.
Their conversion between funds affected and basis points of x-risk.
To estimate 2., you look at:
The number of organizations, and their relative value vs Open Philanthropy
The probability that you would get hired by each
Nitpicks:
I am not sure whether you are considering that if you arrive at a smaller org, you will probably be responsible for a greater % of their funding.
Probabilities of getting hired are not independent, e.g., a “desirable grantmaker” would probably be
Probability of getting hired probably should be a beta, rather than I lognormal? Feels right, though I don’t really have many intuitions for the beta
What I would do to improve this.
I think I’d first simplify the problem and look at only 1., and leave 2. (probability of getting a job, number of such jobs available) for later.
Then, I’d want to:
Have a really clear model, which can be very well maintained.
Get better estimates of how much money each grantmaker at Open Philanthropy is moving
Have the model output all of:
Dollars of money the grantmaker will move
Dollars the grantmaker would be responsible for (probably including Shapley values)
Expected reduction in x-risk, but based on looking at past OP grants, rather than relying on Linch’s estimates
“What I would do to improve this” should read “what I would do to build upon this”. I think that most of the value of this particular estimation lies in the scenario where I/others use it to build a selection of impacts for various career paths, which allows for relative value comparisons between career paths.
Thank you for feedback, Nuno!
I totally agree that part 2 is an unhelpful distraction at this stage. (Agreed with your other points at time of writing!)
Good post (I think, without checking numbers).
probability_get_any_job depends a lot on the person. Also note that if you attempt to do grantmaking but don’t get a job, you still get most of the impact of your default career. So the “try to do grantmaking” action is worth a lot more than P(become a grantmaker)*EV(grantmaking).
I suspect that usd_per_basis_point_xrisk is doing a lot of work, and different people will have pretty different beliefs about it, and maybe in particular the best grants are extremely likely to be made without the marginal grantmaker.
Thank you! On your points:
Probability of getting any Open Philanthropy-equivalent job
Totally agree! For reference, here’s what impact looks like if you set
probability_get_any_jobto 0.01or 0.3
Broader value due to trying for grantmaking jobs
Yes! This is what I am trying to get at by: “There might be over- or under-counting issues. For instance, I guess that the average number of years employed will be 11. This corresponds to me excluding the impact of the grantmaker’s post-grantmaking career from my definition of career path.” But modeling out the value of career paths attempted instead and their respective probabilities sounds awful :P
Should be fine as long as you don’t (1) compare different career paths using very dissimilar career path lengths, and (2) think exit options have different EV ex ante, conditional on the person’s characteristics. (2) looks fishy—I’m sure there’s much room for improvement there.
Sensitivity to $/x-risk basis point
Agree again! Although I am more pessimistic! I think that we can’t disagree about this parameter by much more than 1 order of magnitude, whereas there might be other parameters that have even wider scope for disagreement (or error).
For reference, here’s what impact looks like if you set
usd_per_basis_point_xriskto 100Mor 1B
I guess my preferred solution to the probability_get_any_job issue is generally to just effectively set it to 1. That way you can answer the question “impact of a grantmaker” rather than “impact of trying to be a grantmaker,” and the counterfactual impact of trying to become a grantmaker is P(become a grantmaker)*EV(grantmaking) with opportunity cost depending on your circumstances.
Makes sense! Would add another 0.5 orders of magnitude vs. the unconditional estimate, so ~2 x-risk basis points.
“I estimate the expected x-risk basis points reduced for the prospective marginal grantmaker to be 0.12 (with enormous error bars and caveats).”
I think it would be more understandable to present this as 1 out of X. Also, is this relative or absolute risk reduction?
A basis point is 0.01% absolute risk, so 0.12 basis points corresponds to 0.0012% change in absolute risk, or 1 in (1 / (0.0012/100)) = 1 in 83,000 change in absolute risk.
Thanks for writing this! I’m glad to see more concrete modeling like this. A few concerns/questions:
A 7% probability of getting an OP-equivalent grantmaking job seems high to me, but I might not have a good sense of what roles are out there.
How many (impact-weighted?) roles are there, and at what orgs?
I’d also worry about self-selection and invitations (to apply) to become a grantmaker skewing things.
Answering in turn:
It comes from this number of relevant organizations
this level of Open Philanthropy-ness of those organizations
and this probability of succeeding in any particular job application
I’m sure all of these are wrong/up for debate!
Impact weighting is done at the org-level, above. The number of applicants is the reciprocal of the fraction of grants the grantmaker is responsible for (so, on average, 1⁄0.082 = 12, although this is driven by 25% of Open Philanthropy-equivalent organizations with smaller effective staffs).
Lots to debate here too! You can look into this yourself by copy-pasting the code into squiggle playground.
“Skewing” undersells the issue! It would totally change the calculations. You can see more in the first part of this comment.