2018-19 Donor Lottery Report, pt. 2
This is the second in a series of reports on my decision-making process and decisions in allocating the $500k funding pool from the January 2019 CEA donor lottery. This writeup on my phase-2 grant recommendations is released simultaneously with my writeup of phase 1, which also provides a broader introduction to my personal background, philosophical foundation, and initial process.
While the decision-making process for phase 1 was largely completed prior to the widespread understanding of the scope of the Covid-19 pandemic, phase-2 grantmaking began in March 2020 and specifically focused on neglected responses to the pandemic. (Recommendations were issued in late May and June.) This writeup outlines what I can reconstruct of my process and opinions at the time, and discusses my thoughts on room for further funding.
As with the previous report, this writeup represents independent work and is not coauthored or endorsed by CEA, the organizations or individuals mentioned, or my employer. Grantee organizations and individuals were given one week to review a draft of relevant segments (except for COVID-END, who I neglected to contact before publication through my own mistake), though final editorial decisions were mine.
Overall summary
In phase 2, CEA accepted my recommendations to grant (in decreasing order of grant size):
$100k to Fast Grants, for rapid regranting to academic research projects related to Covid-19.
$80k to COVID-END, to reduce duplication in Covid-19-related research and accelerate evidence-based policy responses worldwide.
$15k to the Bottleneck Fund, for regranting to global interventions in water and sanitation to prevent the spread of <span class=together>Covid-19</āspan>.
$6.4k to Ian David Moss, for consulting work supporting the FRAPPE donor group.
Between these grants and those from phase 1, a total of $366.4k has been granted from the original pool of $500k. I expect the remaining funding pool to be fully distributed in one or two further phases in the next year.
This post has top-level sections on:
General considerations and approach
I should begin this section by saying that I have mostly not followedāor been involved inādebates in non-public social media spaces within the EA community on the relative importance of Covid-19 work.
Instead, one of the very first discussions of Covid-19 EA research that I saw was the post Prioritizing COVID-19 interventions & individual donations, posted to the EA Forum by Catherine Olsson and Ian David Moss in early May. I reached out to them, and they helpfully shared additional unpublished thoughts and research material that had not been suitable for publication at the time. At their invitation, I joined the private chat for the FRAPPE donor group during its final weeks of operation.
At the same time, I was attempting to form my own back-of-the-envelope estimate of whether the Covid-19 pandemic would be an important and neglected issue that EAs should be working on. I expected (with high confidence) that most Western institutions would fail to react to the situation with sufficient flexibility and operational tempo (i.āe., āspeed and intensity of [actions] relative to the speed and intensity of unfolding events in the operational environmentā) to pick even a substantial fraction of the low-hanging and easily-identifiable intervention opportunities.
I further believed (with medium confidence) that widespread lockdowns were politically inevitable, and that (with medium confidence) the economic and human costs of those widespread lockdowns would make Covid-19 interventions plausibly among the most effective interventions for making the world better, at least outside of far-future interventions. (I did not seriously re-evaluate the feasibility of finding neglected and effective far-future interventions, for similar reasons to before.)
[ry: Iām not going to quantify the claims above about the pandemicās absolute importance, lest this post take several times longer to research and write.]
Relative opinions on Covid-19 interventions
Furthermore, I knew that most EA donors were used to acting under the significant evidential certainty that the work of Givewell et al. provideāand anticipated that they would (understandably) be reluctant and slow to act quickly under the massive uncertainty here.
Many EAs, I thought, would be used to āeffectiveā being synonymous with āevidence-backedā, and I believed that none of the most promising Covid-19 interventions would be truly evidence-backed. (I was less certain that these effects would affect large, thoughtful EA donors, but had trouble predicting their actions, and with medium-weak confidence did not expect them to fully fund the promising Covid-19 interventions available.)
In the language of relative opinions, I expected that novel, non-evidence-backed approachesāthat were promising mostly based on outside-view analysis and intuitive prior beliefsāwould be materially under-supported by both the EA community and non-EA institutions. Fortunately, I had the chance to direct discretionary funding due to the donor lottery, and looked for ways to deploy it for effect.
Research approach
I started by considering FRAPPEās longlist of Covid-19 interventions (both the public recommendations, and others noted in unpublished research). I also solicited suggestions for other organizations and interventions from individuals I trust, though no candidates from those conversations ended up working out. (I was already independently aware of Fast Grants, though I was not seriously considering funding it on EA grounds before it was written up in Ian and Catherineās post.)
My subsequent approach was mostly unstructured and punctuated by personal and professional interruptions. At most points, I felt like I faced a strategic compromise between a desire for thoroughness and the premium to acting quickly.
Given the constraints of the exercise, Iām actually quite grateful for my professional experience in trading and applied research, which was useful in helping me to realize when I should stop analyzing further and act on the information I had. Iām near-certain that if I had tried to conduct all of the research at a uniformly high and ārespectableā level of rigor, I would have issued grants months later than I did (and been much less effective).
In the end, Iām glad that I was able to direct the grants that I did, though in hindsight I wish I had started sooner, issued grants with less delay once I already thought they were likely to be a good idea, and aimed to exhaust the remaining funding pool by, say, the middle of the year.
Fast Grants
Summary
Fast Grants was a rapid-turnaround funding mechanism for research on Covid-19, mostly but not exclusively biomedical in nature. In my cursory (~1hr) review of projects they had funded and announced on their website at the time, roughly one-half of the projects they supported were related to vaccine development, one-quarter related to clinical treatment, and one-quarter related to testing or epidemiological modeling.
In the grant rounds that were running at the time, grant applications were reviewed and responses issued on a rolling 48-hour basis, and focus was on projects with a six-month-or-less timescale. The initial round of grants distributed $20mln in the first month after launch, and at the time I was considering a grant, Fast Grants was soliciting first-round grantees to nominate peersā projects for the second round of grants.
(By the point of this postās publication, Fast Grants had distributed more than $40mln; their pace of grants decelerated significantly in the second half of 2020.)
While Fast Grantsās lead principal, Tyler Cowen, is not an epidemiologist or research biologist, I was able to get a sense of the panel of scientists who were actually reviewing grant applications. To my cursory (<2hr) review of their online presences, they appeared to be competent research scientists.
My perspective
While many actors recognized the Covid-19 pandemic as an unusually important moment for action, most organizations in academia, industry, and government demonstrated an unwillingness or inability to change their operating rules for how research was done and resources were allocated. At the time, Fast Grants was one of the few organizations I saw operating to advance Covid-19 research with urgency appropriate to the situation.
My positive opinion of Fast Grants was further due in part to my pre-existing positive opinion of Tyler Cowen, whose writings and nonprofit work I have followed with interest for more than five years. While I do not completely understand Tylerās perspective on how to choose or prioritize projects, I expect him to be extremely capable at work that requires broad context, flexible thinking, and an operating awareness of expected value.
Given that Fast Grants was a project in a high-value cause area suited to a project manager with traits such as Tylerās, I found Fast Grants to be an especially promising funding opportunity. I recommended an initial grant of $100k, with the intention of recommending an additional grant if further research did not reveal other promising funding opportunities.
In hindsight
In hindsight, I now believe I should have recommended a larger grant to Fast Grants at the time, exhausting the entire funding pool not distributed in other phase-1 or phase-2 grants. I expect that the marginal funding would have enabled Fast Grants to further support high-expected-value, funding-constrained, Covid-19-related researchāand in practice, I did not resolve my further research in time to issue grants elsewhere, or a follow-up grant to Fast Grants.
At the present time, however, Fast Grants is no longer soliciting funding. In any case, I no longer believe that it represents a particularly compelling funding opportunity (see āFurther pandemic interventionsā below).
COVID-END
Summary
COVID-END is a collaboration of research-synthesis groups that was, at the time, working to reduce duplication of effort by policymakers reviewing evidence on questions of Covid-19-related policy. (Iām less familiar with the evolution of their work since then, though I understand it to be essentially similar.) Their major initiatives at the time were providing on-demand rapid evidence syntheses and coordinating the sharing of existing evidence reviews between policymakers and other stakeholders.
My perspective
I found it relatively difficult to evaluate the specific concrete actions of the different organs of COVID-END, but I thought there was substantial promise in the fundamental value proposition of reducing duplication in Covid-19 evidence synthesis. I reviewed a selection of rapid evidence reviews and a non-public document about their near-term plans, and I spoke at length with Ian David Moss (from the FRAPPE donor group) about his thoughts about their potential impact.
The evidence that I could evaluate, plus secondhand recommendations from Ian, left me happy to recommend a $80k grant to fill a substantial portion of their then-identified funding gap.
In hindsight
I did not prioritize conducting a follow-up review of COVID-END, as I found Fast Grants to be a more promising candidate for a follow-up grant at the time. I have not tried to evaluate the counterfactual effects of the grant on COVID-ENDās 2020 operations, and do not think it is currently a high priority to do so.
As the Covid-19 information environment matured, it became more plausible to me that the most promising marginal opportunities would not be in further evidence synthesis, and I do not currently believe that funding COVID-END represents a particularly compelling opportunity (see āFurther pandemic interventionsā below).
The Bottleneck Fund
Summary /ā My perspective
The Bottleneck Fund was an ad-hoc project by Kristen Tonga, re-granting operating funds to small-scale ($1k-$10k) interventions in water and sanitation to prevent the spread of Covid-19 in developing regions. I spoke briefly with Kristen as part of my work with the FRAPPE donor group and reviewed a list of the projects she was considering for support. Ultimately, I decided to opportunistically fill the funding gap corresponding to what I felt were the highest-value marginal opportunities. This grant increased the fundās total operating budget by about 60%.
At the time, the Covid-19 information environment was fast-moving, most institutional actors did not act with sufficient operational tempo, and many positive-expected-value interventions were going neglected. It was clear to me that there would be small-scale opportunities to direct funding based on individualsā prior expertise. Kristenās work to opportunistically identify and fund small-scale interventions related to her experience in the water/āsanitation space seemed promising along those lines.
In hindsight
I did not conduct follow-up research on the ultimate deployment of Bottleneck Fund funds. I have reviewed an interim report on the uses of the funds (as I expected, these primarily included installing handwashing stations, distributing liquid soap/āsanitizer/ādetergent, distributing personal protective equipment, and providing Covid-19 education in developing regions), and I understand that follow-up reporting is ongoing.
I continue to be excited in general about the concept of small-scale, opportunistic regranting funds as an option for small- and medium-sized donors to partner with EA community members with narrow but relevant expertise.
Ian David Moss (consulting work for FRAPPE)
Summary
Ian David Moss operates an independent consulting practice offering strategy and research services to donors, foundations, and nonprofits. He co-authored the āPrioritizing Covid-19 interventions & individual donationsā EA Forum post with Catherine Olsson, and I spoke with him multiple times as part of my work with the FRAPPE donor group. At the time, he was contributing a substantial amount of research to the group, with his work partially pro bono and partially at a specially-negotiated consulting rate funded by the FRAPPE donors.
After evaluating his work and the groupās funds remaining, I decided to fund him for a fixed portion of additional research work, with a goal of producing medium-depth evaluations of additional Covid-19 interventions that had seemed marginally promising in initial evaluation.
My perspective
I appreciate the pro bono research work that Ian conducted and shared with the groupāand the larger communityāin this effort. I also support his decision to provide additional work to this community effort at a negotiated consulting rate. I believe that itās important for the long-term economic health and effectiveness of the EA community to support individuals contributing services to the community at market-comparable rates, and I think Ianās work here was no exception.
At the time, we discussed specific projects that he could progress if he diverted marginal time away from his private consulting practice towards Covid-19-related research work, and I thought that the medium-depth investigations would be worthwhile in terms of the potential value of further donations diverted to more-valuable interventions.
In hindsight
Ian used his support from this grant to monitor FRAPPEās previous recommendations for a month longer than originally planned, vet additional potential grantees, and investigate interventions related to combating COVID in Latin America and fighting COVID-related disinformation. Ultimately, I thought none of the new recommendations that resulted from this engagement seemed promising enough to merit granting or further investigation.
I still believe that the expected value of the attempt made the initial grant a good idea at the time. However, I donāt think thereās scope for a similarly promising engagement at this point.
Covid-19 state of play (December 2020)
Further pandemic interventions
Two Covid-19 vaccines have demonstrated a ~95% efficacy rate, and are widely expected to receive US regulatory approval within the month. If they do, and the efficacy numbers are anywhere close to correct, I expect that widespread global vaccination will be the most economically efficient response to the pandemic. If I knew of cost-effective ways of accelerating production or deployment of vaccines (or of encouraging more efficiently-targeted deployment), I believe such interventions would be massively important.
However, my impression is that most of vaccine-deployment policy in the US will be effectively dictated by government beaurocracy, and that influencing that policy will involve shifting a major partisan political dispute. Itās not at all clear to me at this point whether there are effective, funding-limited ways to accelerate vaccine deployment abroad, but if such opportunities existed, it seems plausible that they could be among the most cost-effective interventions targeting the lives of humans living today.
These priorities are materially different than the ones that motivated my grants to Fast Grants and COVID-END. I donāt have any particular belief that either organization has good ways of improving vaccine production or deployment, and so I donāt believe that they (or any of the other grants from phase 2) now represent particularly compelling opportunities for further funding.
Finally, for an outside perspective: GiveWell, in their November recommendations for 2020 giving, claims they did not find unfunded Covid-19 interventions that were more effective than their (non-pandemic-related) top charities. I have not independently reviewed their research.
The 2020 funding environment
Lockdowns and other economic disruptions have impaired economic activity worldwide for much of 2020, and I expect (with medium-high confidence) that donations (including from the EA community) will be negatively impacted. At a baseline, I expect (with medium-high confidence) that funding-constrained donation opportunities will face weaker fundraising from the EA community in 2020 than they did in 2019. If so, some organizationsā marginal opportunities to use resources may be correspondingly stronger than they would be otherwise.
I hope that the standard set of charity evaluators will help to clarify their expectations for the funding environment, at least in the organizations they track. However, I havenāt yet seen a clear summary of such research out of GiveWell or Animal Charity Evaluators. (If youāre aware of any public literature along these lines, please share it in the comments!)
In the absence of solid reviews of the funding environment, I think it is worthwhile for donors of all sizes to consider whether they should adjust their personal giving in 2020. At a minimum, it seems to me that donors idiosyncratically less negatively affected should consider donating more than they historically have, and that donors idiosyncratically more affected should consider donating less than their historical patterns (consistent with the commitments they hold themselves to).
[ry: I hope to write up my own personal giving for 2020āand further thoughts along these linesāon my personal blog soon, and Iāll post a link here when I do.]
What next?
As of December 2020, I have not yet begun phase-3 decision-making in earnest. I donāt expect to start before the new year.
For now, Iāll open a public call for proposals for phase-3 grantsāideally, specific opportunities for grants of size $30k-$200k amenable to a few hours of shallow review, and a few tens of hours of final investigation. (In hindsight, I think I should have done this immediately after the resolution of the donor lottery, or earlier this year.) Proposals can be submitted as comments here, and Iāll reply with any follow-up questions I have. Iāll aim to start shallow reviews of the proposals I find most promising shortly after the new year.
NB: I am open to proposals in the far-future and EA-meta cause areas, subject to the concerns I discussed in āConsiderations on cause areasā in my phase-1 writeup. Iām not opposed to considering organizations that are already high-profileāthough I expect to consider them in the framework described in āRelative opinionsā in my phase-1 writeup, so proposals should indicate the specific feature of the organization that they believe to be undervalued by the donor/āevaluator consensus.
In addition to a phase-3 writeup, I expect to conclude this series with a post of general reflections and post-mortem (no pun intended) on the process.
Finally, CEAās 2020-21 donor lotteries are now open, if youāre interested in entering. (And after this exercise, I still am!)
Here is a number of EA funding requests
I would love for the new Improving Institutional Decision-Making working group to be considered for funding. You can find a description of our planned activities for 2021 at the link, and this comment includes a more detailed explanation of why funding would be helpful. Happy to provide additional information on request.
Funding proposal: AI Safety Support
Our goal is to enable aspiring AI Safety researchers to do the things they are trying to achieve. We provide operational and community support to early career and transitioning researchers to fill gaps in the AI Safety career pipeline. (For more info, see this blogpost)
Suggested donation: Anything in the range $30k - $60k.
We would not turn away smaller amounts, since we are not trying to get fully funded from a single donation anyway. But you suggested $30k as a lower limit.
RegardingāRelative opinionsā, Iām happy todiscussthat in a private, if you want.Edit: I donāt think this reasoning applies to us anyway. Though Iām happy to talk anyway.