I emailed CEA with some questions about the LTFF and EAIF, and Michael Aird (MichaelA on the forum) responded about the EAIF. He said that I could post his email here. Some of the questions overlap with the contents of this AMA (among other things), but I included everything. My questions are formatted as quotes, and the unquoted passages below were written by Michael.
Here are some things Iāve heard about LTFF and EAIF (please correct any misapprehensions):
You can apply for a grant anytime, and a decision will be made within a few weeks.
Basically correct. Though some decisions take longer, mainly for unusually complicated, risky, and/āor large grants, or grants where the applicant decides in response to our questions that they need to revisit their plans and get back to us later. And many decisions are faster.
The application process is meant to be low-effort, with the application requiring no more than a few hoursā work.
Basically correct, though bear in mind that that doesnāt necessarily include the time spent actually doing the planning. We basically just donāt want people to spend >2 hours on actually writing the application, but itāll often make sense to spend >2 hours, sometimes much more than 2 hours, on actual planning.
The funds donāt put many resources into evaluation, which is ad hoc and focuses on the most-controversial grantsāthe goal is to decide whether to make more such grants in the future. (Question: how do you decide whether a controversial grant was successful?) [Authorās note: I was unclear hereāI was asking about post-hoc evaluation, but Michaelās answer is about evaluating grant applications.]
These statements seem somewhat fuzzy so itās hard to say if Iād agree. Hereās what Iād say:
My understanding is that we tend to spend something like 1 hour per $10k of grants made. (I havenāt actually checked this, but Iām pretty sure itād be the right order of magnitude at least.)
When I joined EAIF, I was surprised by that and felt kind-of anxious or uncomfortable about it, but overall I do think it makes sense.
We tend to spend more time on grants that are larger, have at first glance higher upside potential plus higher downside risk, and/āor are harder to evaluate for some reason (e.g., theyāre in areas the fund managers are less familiar with, or the plan is pretty complex).
I donāt think Iād say that what grants we focus more time on is driven by deciding whether to make more grants of that type in the future.
The typical grant is small and one-off (more money requires a new application), and made to an individual. Grants are also made to organizations, and these might be a little bigger but still on the small side (probably not more than $300k).
I guess this is about right, but:
āsmallā is ambiguous. Some specific numbers: Grants Iāve been involved in evaluating have ranged (if I recall correctly) from ~$5k to ~$400k, and there are two ~$250k grants I recommended and that were made. People can definitely apply for larger grants, but often itād make more sense for another funder to evaluate and fund those.
We do make quite a few grants to organizations.
You could compile info on individuals vs orgs and on grant sizes from the public payout reports.
Your specific questions:
How many grants come through channels other than people applying unbidden (e.g., referrals/ānominations by third parties or active grantmaking by fund managers)? Whatās the most common such channel?
I donāt have these numbers (possibly someone else does), but Iād fairly confidently guess at least 10% of applicants whose applicants are approved had at an earlier point had someone (whether a fund manager or not) specifically encourage them to apply.
Iām not sure your question uses a useful way of carving up the space of possibilities. E.g., many people seem to apply in response to fund managers publicly or semi-publicly encouraging people in general to apply, e.g. via Forum posts or via posting in relevant Slack workspaces. E.g., many people presumably apply after 80k advisors or community builders encourage them to. I guess I mean it seems likely that some active promotion effort was involved in the vast majority of grants received, but the active promotion effort can vary a lot in terms of how targeted it is, who itās from, etc.
The LTFFās fund managers all have backgrounds in AI or CS. Is the process for evaluating grants in areas outside the managersā areas of expertise any different?
I donāt know since Iām on the EAIF, but Iām also not sure this is quite the right question to ask. I donāt think itās really like thereās a set of three different pre-specified processes that are engaged under different conditions; itās more ad hoc than that. And there could be many AI/āCS projects that are outside their area of expertise and many non-AI/āCS projects inside their area of expertise (e.g., my understanding is that Oliver and Evan both have experience trying to do things like building research talent pipelines /ā infrastructure /ā mentorship structures, so theyād have some expertise relevant to projects focused on doing that for non-AI issues).
Another thing to note is that some guest fund managers earlier this year had other backgrounds.
I do think it can be problematic to have all fund managers have too narrow a range of areas of expertise and interest, and I think EA Funds sometimes arguably has that. But I also think this is mostly an unfortunate result of talent constraints. And I also think the guest manager system has helped mitigate this, and that the existing permanent fund managerās areas of expertise isnāt super overlapping.
Whatās the role of the advisers to the LTFF and EAIF listed on the website? Do managers commonly discuss grants with people not listed on the website (e.g., experts at other nonprofits)?
Advisors other than Nicole Ross are only involved in maybe something like 10% of grant evaluations, and usually just for quite quick input. Theyāre also sometimes involved in higher-level strategic questions, and sometimes they proactively suggest things (e.g., maybe we should reach out to X to ask if they want to apply to EA Funds or to ask if a larger grant would be useful since they seem to be relying on volunteers).
Nicole Ross checks recommended grants for possible issues of various kinds before the grant is actually made. I think itās pretty rare that this actually changes a grant decision, but sometimes it results in further discussion with applicants that helps double-check or mitigate the potential issues.
Fund managers veryoften discuss grants with specific people not listed on the website. Iād guess that an average of ~3 external people are asked for input on each grant that ends up being approved. (Sometimes 0, often >5.) This is done in an ad hoc way based on the key uncertainties about that particular grant. We also explicitly ask that these consulted people keep the fact the applicant applied confidential.
Whatās the process for a grantās being approved or rejected? E.g., can a primary grant evaluator unilaterally reject a grant? Do grants have to be unanimously approved by all managers? Do all mangers have a say in all grants?
By default, all fund managers on a given Fund get 5 days in which to vote after a grant is put up for vote by the primary evaluator. Then the final decision is based on whether the average of the votes exceeds a particular threshold. On the EAIF, this average is a weighted average, with the primary evaluator having a weight of 2 by default and everyone else having a weight of 1 by default.
Usually only ~2 people actually give a vote, in my experience.
Usually the final decision is the decision the PI recommended.
Sometimes the voting period is shortened if a grant is time-sensitive.
Sometimes a given fund manager recuses themselves due to possible conflicts of interest, in which case they donāt vote and may also be removed from the doc with notes and such.
What are the motivations for having guest managersāincreased capacity, identifying or training promising grantmakers, diversity of viewpoints?
This is discussed in some recent AMAs, if I recall correctly
We also now have an assistant fund manager on the EAIF, helping Buck with his evaluations. I personally think this is a great move, for all 3 reasons you mentioned, just as I think the guest fund manager role was a good thing to have created.
I know that sometimes you give feedback to unsuccessful grant recipients. What does this feedback look likeāe.g., is it a 3-sentence email, or an arbitrarily long phone conversation with the primary evaluator?
Basically, either, or anything in between, though I think āarbitrarily longā seems unlikelyāIād guess itās rarely or never been a >1 hour phone call.
What processes do you have to learn from mistakes or sub-optimal decisions?
We get reports from grantees on their progress etc. - though I donāt think we actually heavily use this to improve over time
I personally make forecasts relevant to most grants I recommend before the grants are made, and I plan to look back at them later to see how calibrated I was and what I can learn from that. I think some other people do this as well, but I think most donāt, and unfortunately Iāve come to feel that thatās reasonable given time constraints. (I think this is a shame, and that more capacity such that we could do that and various other things would be good, but thereās a severe talent constraint.)
There are various ad hoc and/āor individual-level things
There may be things Jonas, fund chairs, and/āor permanent fund managers do that Iām aware of
Weāve discussed whether and how to better evaluate our performance and improve over time, what weād want to learn, etc. I think this is something people will continue to think more about. I personally expect thereās more we should be doing, but itās not super obvious that thatās the case (there are also many other good things we could do if we were willing to spend extra hours on something new), nor precisely what itād be best to do.
I emailed CEA with some questions about the LTFF and EAIF, and Michael Aird (MichaelA on the forum) responded about the EAIF. He said that I could post his email here. Some of the questions overlap with the contents of this AMA (among other things), but I included everything. My questions are formatted as quotes, and the unquoted passages below were written by Michael.
Basically correct. Though some decisions take longer, mainly for unusually complicated, risky, and/āor large grants, or grants where the applicant decides in response to our questions that they need to revisit their plans and get back to us later. And many decisions are faster.
Basically correct, though bear in mind that that doesnāt necessarily include the time spent actually doing the planning. We basically just donāt want people to spend >2 hours on actually writing the application, but itāll often make sense to spend >2 hours, sometimes much more than 2 hours, on actual planning.
These statements seem somewhat fuzzy so itās hard to say if Iād agree. Hereās what Iād say:
My understanding is that we tend to spend something like 1 hour per $10k of grants made. (I havenāt actually checked this, but Iām pretty sure itād be the right order of magnitude at least.)
When I joined EAIF, I was surprised by that and felt kind-of anxious or uncomfortable about it, but overall I do think it makes sense.
We tend to spend more time on grants that are larger, have at first glance higher upside potential plus higher downside risk, and/āor are harder to evaluate for some reason (e.g., theyāre in areas the fund managers are less familiar with, or the plan is pretty complex).
I donāt think Iād say that what grants we focus more time on is driven by deciding whether to make more grants of that type in the future.
I guess this is about right, but:
āsmallā is ambiguous. Some specific numbers: Grants Iāve been involved in evaluating have ranged (if I recall correctly) from ~$5k to ~$400k, and there are two ~$250k grants I recommended and that were made. People can definitely apply for larger grants, but often itād make more sense for another funder to evaluate and fund those.
We do make quite a few grants to organizations.
You could compile info on individuals vs orgs and on grant sizes from the public payout reports.
Your specific questions:
I donāt have these numbers (possibly someone else does), but Iād fairly confidently guess at least 10% of applicants whose applicants are approved had at an earlier point had someone (whether a fund manager or not) specifically encourage them to apply.
Iām not sure your question uses a useful way of carving up the space of possibilities. E.g., many people seem to apply in response to fund managers publicly or semi-publicly encouraging people in general to apply, e.g. via Forum posts or via posting in relevant Slack workspaces. E.g., many people presumably apply after 80k advisors or community builders encourage them to. I guess I mean it seems likely that some active promotion effort was involved in the vast majority of grants received, but the active promotion effort can vary a lot in terms of how targeted it is, who itās from, etc.
I donāt know since Iām on the EAIF, but Iām also not sure this is quite the right question to ask. I donāt think itās really like thereās a set of three different pre-specified processes that are engaged under different conditions; itās more ad hoc than that. And there could be many AI/āCS projects that are outside their area of expertise and many non-AI/āCS projects inside their area of expertise (e.g., my understanding is that Oliver and Evan both have experience trying to do things like building research talent pipelines /ā infrastructure /ā mentorship structures, so theyād have some expertise relevant to projects focused on doing that for non-AI issues).
Another thing to note is that some guest fund managers earlier this year had other backgrounds.
I do think it can be problematic to have all fund managers have too narrow a range of areas of expertise and interest, and I think EA Funds sometimes arguably has that. But I also think this is mostly an unfortunate result of talent constraints. And I also think the guest manager system has helped mitigate this, and that the existing permanent fund managerās areas of expertise isnāt super overlapping.
Advisors other than Nicole Ross are only involved in maybe something like 10% of grant evaluations, and usually just for quite quick input. Theyāre also sometimes involved in higher-level strategic questions, and sometimes they proactively suggest things (e.g., maybe we should reach out to X to ask if they want to apply to EA Funds or to ask if a larger grant would be useful since they seem to be relying on volunteers).
Nicole Ross checks recommended grants for possible issues of various kinds before the grant is actually made. I think itās pretty rare that this actually changes a grant decision, but sometimes it results in further discussion with applicants that helps double-check or mitigate the potential issues.
Fund managers very often discuss grants with specific people not listed on the website. Iād guess that an average of ~3 external people are asked for input on each grant that ends up being approved. (Sometimes 0, often >5.) This is done in an ad hoc way based on the key uncertainties about that particular grant. We also explicitly ask that these consulted people keep the fact the applicant applied confidential.
By default, all fund managers on a given Fund get 5 days in which to vote after a grant is put up for vote by the primary evaluator. Then the final decision is based on whether the average of the votes exceeds a particular threshold. On the EAIF, this average is a weighted average, with the primary evaluator having a weight of 2 by default and everyone else having a weight of 1 by default.
Usually only ~2 people actually give a vote, in my experience.
Usually the final decision is the decision the PI recommended.
Sometimes the voting period is shortened if a grant is time-sensitive.
Sometimes a given fund manager recuses themselves due to possible conflicts of interest, in which case they donāt vote and may also be removed from the doc with notes and such.
This is discussed in some recent AMAs, if I recall correctly
We also now have an assistant fund manager on the EAIF, helping Buck with his evaluations. I personally think this is a great move, for all 3 reasons you mentioned, just as I think the guest fund manager role was a good thing to have created.
Basically, either, or anything in between, though I think āarbitrarily longā seems unlikelyāIād guess itās rarely or never been a >1 hour phone call.
We get reports from grantees on their progress etc. - though I donāt think we actually heavily use this to improve over time
I personally make forecasts relevant to most grants I recommend before the grants are made, and I plan to look back at them later to see how calibrated I was and what I can learn from that. I think some other people do this as well, but I think most donāt, and unfortunately Iāve come to feel that thatās reasonable given time constraints. (I think this is a shame, and that more capacity such that we could do that and various other things would be good, but thereās a severe talent constraint.)
There are various ad hoc and/āor individual-level things
There may be things Jonas, fund chairs, and/āor permanent fund managers do that Iām aware of
Weāve discussed whether and how to better evaluate our performance and improve over time, what weād want to learn, etc. I think this is something people will continue to think more about. I personally expect thereās more we should be doing, but itās not super obvious that thatās the case (there are also many other good things we could do if we were willing to spend extra hours on something new), nor precisely what itād be best to do.