I currently lead EA funds.
Before that, I worked on improving epistemics in the EA community at CEA (as a contractor), as a research assistant at the Global Priorities Institute, on community building, and Global Health Policy.
Unless explicitly stated otherwise, opinions are my own, not my employer’s.
You can give me positive and negative feedback here.
Sorry, it wasn’t clear. The reference class I had in mind was cause prio focussed resources on the EA forum.
I think people/orgs do some amount of this, but it’s kind of a pain to share them publicly. I prefer to share this kind of stuff with specific people in Google Docs, in in-person conversations, or on Slack.
I also worry somewhat about people deferring to random cause prio posts, and I’d guess that on the current margin, more cause prio posts that are around the current median in quality make the situation worse rather than better (though I could see it going either way).
Thanks for writing this.
I disagree with quite a few points in the total utilitarianism section, but zooming out slightly, I think that total utilitarians should generally still support alignment work (and potentially an AI pause/slow down) to preserve option value. If it turns out that AIs are moral patients and that it would be good for them to spread into the universe optimising for values that don’t look particularly human, we can still (in principle) do that. This is compatible with thinking that alignment from a total utilitarian perspective is ~neutral—but it’s not clear that you agree with this from the post.
I’m interested in examples of this if you have them.
oops, fixed—thank you
Oh, I thought you might have suggested the live thing before, my mistake. Maybe I should have just given the 90-day figure above.
(That approach seems reasonable to me)
I answered the first questions above in an edit of the original comment. I’m pretty sure when I re-ran the analysis with decided in last 30 days it didn’t change the results significantly (though I’ll try and recheck this later this week—in our current setup it’s a bit more complicated to work out than the stats I gave above).
I also checked to make sure that only looking at resolved applications and only looking at open applications didn’t make a large difference to the numbers I gave above (in general, the differences were 0-10 days).
Oh, right—I was counting “never receiving a decision but letting us know” as a decision. In this case, the number we’d give is days until the application was withdrawn.
We don’t track the reason for withdrawals in our KPIs, but I am pretty sure that process length is a reason for a withdrawal 0-5% of the time.
I might be missing why this is important, I would have thought that if we were making an error it would overestimate those times—not underestimate them.
I’m not sure sorry, I don’t have that stat in front of me. I may be able to find it in a few days.
Empirically, I don’t think that this has happened very much. We have a “withdrawn by applicant status”, which would include this, but the status is very rarely used.
In any case, the numbers above will factor those applications in, but I would guess that if we didn’t, the numbers would decrease by less than a day.
We do have a few processes that are designed to do this (some of which are doing some of the things you mentioned above). Most of the long delays are fairly uncorrelated (e.g. complicated legal issue, a bug in our application tracker …).
I think it could be good to put these number on our site. I liked your past suggestion of having live data, though it’s a bit technically challenging to implement—but the obvious MVP (as you point out) is to have a bunch of stats on our site. I’ll make a note to add some stats (though maintaining this kind of information can be quite costly, so I don’t want to commit to doing this).
In the meantime, here are a few numbers that I quickly put together (across all of our funds).
Grant decision turnaround times (mean, median):
applied in the last 30 days = 14 days, 15 days
this is pretty volatile as it includes applications that haven’t yet closed
applied in the last 60 days = 23 days, 20 days
applied in the last 90 days = 25 days, 20 days
When I last checked our (anonymous) feedback form, the average score for [satisfaction of evaluation process] (I can’t quite remember the exact question) was ~4.5/5.
(edit: just found the stats—these are all out of 5)
Overall satisfaction with application process: 4.67
Overall satisfaction with processing time: 4.58
Evaluation time: 4.3
Communications with evaluators: 4.7
I’m not sure that these stats tell the whole story. There are cases where we (or applicants) miss emails or miscommunicate—but the frequency of events like this is difficult to report quickly and also accounts for the majority of negative experiences (according to our feedback form and my own analysis).
On (b), I really would like us to be quicker—and more importantly, more reliable. A few very long-tail applications make the general grantee experience much worse. The general stages in our application process are:
Applicant submits application → application is assigned to a fund manager → fund manager evaluates the application (which often involves back and forth with the applicant, checking references etc.) → other fund managers vote on the application → fund chair reviews evaluation → application is reviewed by external advisors → fund chair gives decision to grantee (pending legal review)
There’s also a really high volume of grants and increasingly few “obvious” rejections. E.g. the LTFF right now has over > 100 applications in its pipeline, and in the last 30 days < 10% of applications were obvious rejections).
If it’s okay with you, I’d prefer not to have screenshots of my emails posted right now; I’m happy to rethink this in a few days when it feels a bit lower pressure.
I generally don’t write emails, assuming that they will be posted to a public place like the forum.
I believe our application form says
The Animal Welfare Fund, Long-Term Future Fund and EA Infrastructure Fund aim to respond to all applications in 2 months and most applications in 3 weeks. However, due to an unprecedentedly high load, we are currently unable to achieve our desired speedy turnarounds. If you need to hear back sooner (e.g., within a few weeks), you can let us know in the application form, and we will see what we can do. Please note that: EA Funds is low on capacity and may not be able to get back to you by either your stated deadline or the above aims—we encourage you to apply to other funders as well if you have a time-sensitive ask.
In other parts of the site we say things like “usually we get a decision within 21 days”. It’s possible that the application form said something different when Igor applied, we edit it fairly regularly though I’m not aware of changing that specific like recently.
One thing to note is that at the end of January, we rejected the original grant (which I believed that we wouldn’t be able to show a clear public benefit for), and then said we were interested in a different version of the grant that seemed more defensible to me (subject to legal review). Since then, we have been working out whether we can make this alternate grant.
I didn’t realise that Igor stopped taking clients completely, and I regret that I didn’t make a stronger effort to understand the consequences of the unclear situation whilst we tried to understand the legal implications of making the grant.
I’m very sorry that you had such a bad experience here. Whilst I would disagree with some of the details here I do think that our communication was worse than I would have liked and I am very sorry for any hardship that you experienced. It sounds like a stressful process which could have been made much better if we had communicated more often and more quickly.
In my last email (March 4th), I said that we were exploring making this grant, but it’s legally challenging. Grants for mental health support are complicated, in general, as we have to show that there is a pure public benefit. We have an open thread with our legal counsel, and I’m cautiously optimistic about getting a decision on this relatively soon.
In general, I don’t think I made promises or hard commitments to get back in a certain time frame; instead, I said that we aim to get back by a certain time. I believe I am at fault for not making this distinction appropriately clear, and I am upset that this mismatch of expectations resulted in hardship.
I’d also like to quickly clarify that many of the errors here were mine (as opposed to the wider EA Funds team). I should have been more realistic about the time frame for a grant of this nature.
If the funders get a nontrivial portion of the impact for early-stage projects then I think the AWF (inc. its donors) is very plausible.
Here are some very brief takes on the CCM web app now that RP has had a chance to iron out any initial bugs. I’m happy to elaborate more on any of these comments.
Some praise
This is an extremely ambitious project, and it’s very surprising that this is the first unified model of this type I’ve seen (though I’m sure various people have their own private models).
I have a bunch of quantitative models on cause prio sub-questions, but I don’t like to share these publicly because of the amount of context that’s required to interpret them (and because the methodology is often pretty unrefined) - props to RP for sharing theirs!
I could see this product being pretty useful to new funders who have a lot of flexibility over where donations go.
I think the intra-worldview models (e.g. comparing animal welfare interventions) seem reasonable to me (though I only gave these a quick glance)
I see this as a solid contribution to cause prioritisation efforts and I admire the focus on trying to do work that people might actually use—rather than just producing a paper with no accompanying tool.
Some critiques
I think RP underrates the extent to which their default values will end up being the defaults for model users (particularly some of the users they most want to influence)
I think the default values are (in my personal view) pretty far from my values or the mean of values for people who have thought hard about these topics in the EA community.
The x-risk model in particular seems to make bake-in quite conservative assumptions (medium-high confidence)
I found it difficult to provide very large numbers on future population per star—I think with current rates of economic and compute growth, the number of digital people could be extremely high very quickly.
I think some x-risk interventions could plausibly have very long run effects on x-risk (e.g. by building an aligned super intelligence)
The x-risk model seems to confuse existential risk and extinction risk (medium confidence—maybe this was explained somewhere, and I missed it)
Using the model felt clunky to me, it didn’t handle extremely large values well and it made iterating on values difficult, it’s not the kind of thing that you can “play with” imo.
Some improvements I’d like to see
I’d be interested in seeing RP commission some default values from researchers/EAs who can explain their suggested values well.
I would like for the overall app to feel more polished/responsive/usable—idk how much this would cost. I’d guess it’s at least a month’s work for a competent dev, maybe more.
I could see people upvoting this post because they think it should be more like −10 than −21. I personally don’t see it as concerning that it’s “only” on −21.