I emailed CEA with some questions about the LTFF and EAIF, and Michael Aird (MichaelA on the forum) responded about the EAIF. He said that I could post his email here. Some of the questions overlap with the contents of this AMA (among other things), but I included everything. My questions are formatted as quotes, and the unquoted passages below were written by Michael.
Here are some things I’ve heard about LTFF and EAIF (please correct any misapprehensions):
You can apply for a grant anytime, and a decision will be made within a few weeks.
Basically correct. Though some decisions take longer, mainly for unusually complicated, risky, and/or large grants, or grants where the applicant decides in response to our questions that they need to revisit their plans and get back to us later. And many decisions are faster.
The application process is meant to be low-effort, with the application requiring no more than a few hours’ work.
Basically correct, though bear in mind that that doesn’t necessarily include the time spent actually doing the planning. We basically just don’t want people to spend >2 hours on actually writing the application, but it’ll often make sense to spend >2 hours, sometimes much more than 2 hours, on actual planning.
The funds don’t put many resources into evaluation, which is ad hoc and focuses on the most-controversial grants—the goal is to decide whether to make more such grants in the future. (Question: how do you decide whether a controversial grant was successful?) [Author’s note: I was unclear here—I was asking about post-hoc evaluation, but Michael’s answer is about evaluating grant applications.]
These statements seem somewhat fuzzy so it’s hard to say if I’d agree. Here’s what I’d say:
My understanding is that we tend to spend something like 1 hour per $10k of grants made. (I haven’t actually checked this, but I’m pretty sure it’d be the right order of magnitude at least.)
When I joined EAIF, I was surprised by that and felt kind-of anxious or uncomfortable about it, but overall I do think it makes sense.
We tend to spend more time on grants that are larger, have at first glance higher upside potential plus higher downside risk, and/or are harder to evaluate for some reason (e.g., they’re in areas the fund managers are less familiar with, or the plan is pretty complex).
I don’t think I’d say that what grants we focus more time on is driven by deciding whether to make more grants of that type in the future.
The typical grant is small and one-off (more money requires a new application), and made to an individual. Grants are also made to organizations, and these might be a little bigger but still on the small side (probably not more than $300k).
I guess this is about right, but:
“small” is ambiguous. Some specific numbers: Grants I’ve been involved in evaluating have ranged (if I recall correctly) from ~$5k to ~$400k, and there are two ~$250k grants I recommended and that were made. People can definitely apply for larger grants, but often it’d make more sense for another funder to evaluate and fund those.
We do make quite a few grants to organizations.
You could compile info on individuals vs orgs and on grant sizes from the public payout reports.
Your specific questions:
How many grants come through channels other than people applying unbidden (e.g., referrals/nominations by third parties or active grantmaking by fund managers)? What’s the most common such channel?
I don’t have these numbers (possibly someone else does), but I’d fairly confidently guess at least 10% of applicants whose applicants are approved had at an earlier point had someone (whether a fund manager or not) specifically encourage them to apply.
I’m not sure your question uses a useful way of carving up the space of possibilities. E.g., many people seem to apply in response to fund managers publicly or semi-publicly encouraging people in general to apply, e.g. via Forum posts or via posting in relevant Slack workspaces. E.g., many people presumably apply after 80k advisors or community builders encourage them to. I guess I mean it seems likely that some active promotion effort was involved in the vast majority of grants received, but the active promotion effort can vary a lot in terms of how targeted it is, who it’s from, etc.
The LTFF’s fund managers all have backgrounds in AI or CS. Is the process for evaluating grants in areas outside the managers’ areas of expertise any different?
I don’t know since I’m on the EAIF, but I’m also not sure this is quite the right question to ask. I don’t think it’s really like there’s a set of three different pre-specified processes that are engaged under different conditions; it’s more ad hoc than that. And there could be many AI/CS projects that are outside their area of expertise and many non-AI/CS projects inside their area of expertise (e.g., my understanding is that Oliver and Evan both have experience trying to do things like building research talent pipelines / infrastructure / mentorship structures, so they’d have some expertise relevant to projects focused on doing that for non-AI issues).
Another thing to note is that some guest fund managers earlier this year had other backgrounds.
I do think it can be problematic to have all fund managers have too narrow a range of areas of expertise and interest, and I think EA Funds sometimes arguably has that. But I also think this is mostly an unfortunate result of talent constraints. And I also think the guest manager system has helped mitigate this, and that the existing permanent fund manager’s areas of expertise isn’t super overlapping.
What’s the role of the advisers to the LTFF and EAIF listed on the website? Do managers commonly discuss grants with people not listed on the website (e.g., experts at other nonprofits)?
Advisors other than Nicole Ross are only involved in maybe something like 10% of grant evaluations, and usually just for quite quick input. They’re also sometimes involved in higher-level strategic questions, and sometimes they proactively suggest things (e.g., maybe we should reach out to X to ask if they want to apply to EA Funds or to ask if a larger grant would be useful since they seem to be relying on volunteers).
Nicole Ross checks recommended grants for possible issues of various kinds before the grant is actually made. I think it’s pretty rare that this actually changes a grant decision, but sometimes it results in further discussion with applicants that helps double-check or mitigate the potential issues.
Fund managers veryoften discuss grants with specific people not listed on the website. I’d guess that an average of ~3 external people are asked for input on each grant that ends up being approved. (Sometimes 0, often >5.) This is done in an ad hoc way based on the key uncertainties about that particular grant. We also explicitly ask that these consulted people keep the fact the applicant applied confidential.
What’s the process for a grant’s being approved or rejected? E.g., can a primary grant evaluator unilaterally reject a grant? Do grants have to be unanimously approved by all managers? Do all mangers have a say in all grants?
By default, all fund managers on a given Fund get 5 days in which to vote after a grant is put up for vote by the primary evaluator. Then the final decision is based on whether the average of the votes exceeds a particular threshold. On the EAIF, this average is a weighted average, with the primary evaluator having a weight of 2 by default and everyone else having a weight of 1 by default.
Usually only ~2 people actually give a vote, in my experience.
Usually the final decision is the decision the PI recommended.
Sometimes the voting period is shortened if a grant is time-sensitive.
Sometimes a given fund manager recuses themselves due to possible conflicts of interest, in which case they don’t vote and may also be removed from the doc with notes and such.
What are the motivations for having guest managers—increased capacity, identifying or training promising grantmakers, diversity of viewpoints?
This is discussed in some recent AMAs, if I recall correctly
We also now have an assistant fund manager on the EAIF, helping Buck with his evaluations. I personally think this is a great move, for all 3 reasons you mentioned, just as I think the guest fund manager role was a good thing to have created.
I know that sometimes you give feedback to unsuccessful grant recipients. What does this feedback look like—e.g., is it a 3-sentence email, or an arbitrarily long phone conversation with the primary evaluator?
Basically, either, or anything in between, though I think “arbitrarily long” seems unlikely—I’d guess it’s rarely or never been a >1 hour phone call.
What processes do you have to learn from mistakes or sub-optimal decisions?
We get reports from grantees on their progress etc. - though I don’t think we actually heavily use this to improve over time
I personally make forecasts relevant to most grants I recommend before the grants are made, and I plan to look back at them later to see how calibrated I was and what I can learn from that. I think some other people do this as well, but I think most don’t, and unfortunately I’ve come to feel that that’s reasonable given time constraints. (I think this is a shame, and that more capacity such that we could do that and various other things would be good, but there’s a severe talent constraint.)
There are various ad hoc and/or individual-level things
There may be things Jonas, fund chairs, and/or permanent fund managers do that I’m aware of
We’ve discussed whether and how to better evaluate our performance and improve over time, what we’d want to learn, etc. I think this is something people will continue to think more about. I personally expect there’s more we should be doing, but it’s not super obvious that that’s the case (there are also many other good things we could do if we were willing to spend extra hours on something new), nor precisely what it’d be best to do.
I emailed CEA with some questions about the LTFF and EAIF, and Michael Aird (MichaelA on the forum) responded about the EAIF. He said that I could post his email here. Some of the questions overlap with the contents of this AMA (among other things), but I included everything. My questions are formatted as quotes, and the unquoted passages below were written by Michael.
Basically correct. Though some decisions take longer, mainly for unusually complicated, risky, and/or large grants, or grants where the applicant decides in response to our questions that they need to revisit their plans and get back to us later. And many decisions are faster.
Basically correct, though bear in mind that that doesn’t necessarily include the time spent actually doing the planning. We basically just don’t want people to spend >2 hours on actually writing the application, but it’ll often make sense to spend >2 hours, sometimes much more than 2 hours, on actual planning.
These statements seem somewhat fuzzy so it’s hard to say if I’d agree. Here’s what I’d say:
My understanding is that we tend to spend something like 1 hour per $10k of grants made. (I haven’t actually checked this, but I’m pretty sure it’d be the right order of magnitude at least.)
When I joined EAIF, I was surprised by that and felt kind-of anxious or uncomfortable about it, but overall I do think it makes sense.
We tend to spend more time on grants that are larger, have at first glance higher upside potential plus higher downside risk, and/or are harder to evaluate for some reason (e.g., they’re in areas the fund managers are less familiar with, or the plan is pretty complex).
I don’t think I’d say that what grants we focus more time on is driven by deciding whether to make more grants of that type in the future.
I guess this is about right, but:
“small” is ambiguous. Some specific numbers: Grants I’ve been involved in evaluating have ranged (if I recall correctly) from ~$5k to ~$400k, and there are two ~$250k grants I recommended and that were made. People can definitely apply for larger grants, but often it’d make more sense for another funder to evaluate and fund those.
We do make quite a few grants to organizations.
You could compile info on individuals vs orgs and on grant sizes from the public payout reports.
Your specific questions:
I don’t have these numbers (possibly someone else does), but I’d fairly confidently guess at least 10% of applicants whose applicants are approved had at an earlier point had someone (whether a fund manager or not) specifically encourage them to apply.
I’m not sure your question uses a useful way of carving up the space of possibilities. E.g., many people seem to apply in response to fund managers publicly or semi-publicly encouraging people in general to apply, e.g. via Forum posts or via posting in relevant Slack workspaces. E.g., many people presumably apply after 80k advisors or community builders encourage them to. I guess I mean it seems likely that some active promotion effort was involved in the vast majority of grants received, but the active promotion effort can vary a lot in terms of how targeted it is, who it’s from, etc.
I don’t know since I’m on the EAIF, but I’m also not sure this is quite the right question to ask. I don’t think it’s really like there’s a set of three different pre-specified processes that are engaged under different conditions; it’s more ad hoc than that. And there could be many AI/CS projects that are outside their area of expertise and many non-AI/CS projects inside their area of expertise (e.g., my understanding is that Oliver and Evan both have experience trying to do things like building research talent pipelines / infrastructure / mentorship structures, so they’d have some expertise relevant to projects focused on doing that for non-AI issues).
Another thing to note is that some guest fund managers earlier this year had other backgrounds.
I do think it can be problematic to have all fund managers have too narrow a range of areas of expertise and interest, and I think EA Funds sometimes arguably has that. But I also think this is mostly an unfortunate result of talent constraints. And I also think the guest manager system has helped mitigate this, and that the existing permanent fund manager’s areas of expertise isn’t super overlapping.
Advisors other than Nicole Ross are only involved in maybe something like 10% of grant evaluations, and usually just for quite quick input. They’re also sometimes involved in higher-level strategic questions, and sometimes they proactively suggest things (e.g., maybe we should reach out to X to ask if they want to apply to EA Funds or to ask if a larger grant would be useful since they seem to be relying on volunteers).
Nicole Ross checks recommended grants for possible issues of various kinds before the grant is actually made. I think it’s pretty rare that this actually changes a grant decision, but sometimes it results in further discussion with applicants that helps double-check or mitigate the potential issues.
Fund managers very often discuss grants with specific people not listed on the website. I’d guess that an average of ~3 external people are asked for input on each grant that ends up being approved. (Sometimes 0, often >5.) This is done in an ad hoc way based on the key uncertainties about that particular grant. We also explicitly ask that these consulted people keep the fact the applicant applied confidential.
By default, all fund managers on a given Fund get 5 days in which to vote after a grant is put up for vote by the primary evaluator. Then the final decision is based on whether the average of the votes exceeds a particular threshold. On the EAIF, this average is a weighted average, with the primary evaluator having a weight of 2 by default and everyone else having a weight of 1 by default.
Usually only ~2 people actually give a vote, in my experience.
Usually the final decision is the decision the PI recommended.
Sometimes the voting period is shortened if a grant is time-sensitive.
Sometimes a given fund manager recuses themselves due to possible conflicts of interest, in which case they don’t vote and may also be removed from the doc with notes and such.
This is discussed in some recent AMAs, if I recall correctly
We also now have an assistant fund manager on the EAIF, helping Buck with his evaluations. I personally think this is a great move, for all 3 reasons you mentioned, just as I think the guest fund manager role was a good thing to have created.
Basically, either, or anything in between, though I think “arbitrarily long” seems unlikely—I’d guess it’s rarely or never been a >1 hour phone call.
We get reports from grantees on their progress etc. - though I don’t think we actually heavily use this to improve over time
I personally make forecasts relevant to most grants I recommend before the grants are made, and I plan to look back at them later to see how calibrated I was and what I can learn from that. I think some other people do this as well, but I think most don’t, and unfortunately I’ve come to feel that that’s reasonable given time constraints. (I think this is a shame, and that more capacity such that we could do that and various other things would be good, but there’s a severe talent constraint.)
There are various ad hoc and/or individual-level things
There may be things Jonas, fund chairs, and/or permanent fund managers do that I’m aware of
We’ve discussed whether and how to better evaluate our performance and improve over time, what we’d want to learn, etc. I think this is something people will continue to think more about. I personally expect there’s more we should be doing, but it’s not super obvious that that’s the case (there are also many other good things we could do if we were willing to spend extra hours on something new), nor precisely what it’d be best to do.