CatherineHollander

Karma: 112

CatherineHollander 23 Feb 2017 0:07 UTC
8 points
0 ∶ 0
on: GiveWell and the problem of partial funding
Hi Ben,

Thanks for the post! I wanted to reply to a couple ideas you raised for GiveWell:

(1) Assess outcomes.

Many of the points you raised, such as making empirical after-the-fact estimates, relate to the question of why GiveWell isn’t putting more effort into collecting and examining post-hoc data demonstrating the impacts of our top charities.

We provide an estimate of the impact of a donation to each of our top charities, in humanitarian terms, here: http://www.givewell.org/charities/top-charities/impact. As you note, this is based on the expected impact of donations made to the charity today, rather than a look back at the impact of past work by the charity. We don’t currently collect outcomes data of the type you’re describing (we have generally focused on collecting data, for example, to show that children received deworming treatments, rather than that recipients’ income later improved; we rely on previously conducted studies to estimate the connection between the two).

Our impression is that post-hoc outcomes data of the type you’re describing—on lives saved, health outcomes, future test scores, or income—is largely not available from charities, even our top charities, such that we’d need to fund or collect it ourselves. We are taking some steps to do this in our work with IDinsight, an organization that supports and conducts rigorous evaluations of development interventions. As part of GiveWell Incubation Grants, Good Ventures supported the creation of an IDinsight embedded team at GiveWell to improve monitoring of GiveWell top charities and support the development of potential future GiveWell recommendations. Our work with IDinsight has so far furthered our impression that this information is quite challenging to collect, even with additional funding and effort, such that we’d most likely want to take a judicious approach and complete such work in conjunction with specialized, third-party organizations. In general, we expect significant time and resources are needed to gather data of the type you’re referring to. We’re willing to put these resources in when they seem likely to further our understanding, as demonstrated by our work with IDinsight, but expect a high ratio of resource-input to better-information output.

I’m unsure whether the GiveDirectly study remains underfunded; the footnote you cited referenced a conversation from 2014, and it appears from looking at our subsequent communications that the study has gone forward. But to your point about whether we should fund studies like these: We agree that we should strongly consider doing so, and decisions to do so will depend on room for more funding, as well as our expectation that the study will have sufficient power to be informative for our recommendations.

(Continued in next comment)

**Comment was edited to clarify impact of deworming treatment.

CatherineHollander 23 Feb 2017 0:08 UTC
7 points
0 ∶ 0
in reply to: CatherineHollander’s comment on: GiveWell and the problem of partial funding
(Continued from previous comment)

(2) Market humbly.

We agree that not everyone has an accurate view of GiveWell’s work, and that we should continue to improve our communications around the kinds of opportunities we recommend. Publishing information about our reasoning and goals on our website and blog is one way we aim to do this, as is speaking with the media and donors who use our research, but we agree there is room for improvement. In my experience working on GiveWell’s outreach, it has been particularly challenging to effectively communicate around the following:

a) The uncertainty associated with deworming research. b) The limitations of our cost-effectiveness analyses. c) The type of opportunities GiveWell considers as potential top charities, and why.

We think we can continue to improve in our written and verbal communications around these topics. A goal on our website and in our own communications is to provide the most accurate picture we can at any given level of detail, within reasonable bounds of staff capacity and time. Someone who only reads a headline on our website should have the most accurate picture it’s possible to have after reading only a headline; someone who only reads our page listing top charities should have the most accurate picture associated with that level of detail, and so on.

Going forward, we think we can improve by more proactively reaching out when we become aware of a misimpression of GiveWell. We’ve had internal discussions about this, prompted by this post, and plan to more proactively communicate about mistakes or misunderstandings of our work when we become aware of them, including emailing the media with clarifications. We did not do this in the case of the quote you cite from The Atlantic article, and on reflection think this is something we should do in the future.

On the name of “top charities”: We’d be interested in whether there is a term that you feel would more succinctly and accurately convey our views on our recommendations than “top charities.” We want to avoid projecting overconfidence, but we also don’t want to suggest we’re less confident or think these are less good options for most donors than we believe they are. A concern with applying a restricted category, such as “top charities within global health and development,” would be suggesting that we hadn’t considered opportunities that fall outside of this category or that we chose this category arbitrarily, neither of which is true: GiveWell focuses on global health and development because our initial research led us to believe that the charities most likely to succeed by GiveWell’s criteria of cost-effectiveness and a strong, generalizable evidence base work in global health and development. (More on this here and here.) Similarly, a concern with referring to GiveWell’s top charities as something like “reasonably good options” could lead donors to be confused about whether we were recommending them—which, in the case of most low-time, low-trust donors (more below; this is the group we believe makes up most of our donor base), we do think they likely represent the best options.

GiveWell and the Open Philanthropy Project, the “last dollar,” and whether GiveWell’s top charities are better giving opportunities

Here, it’s helpful to distinguish between GiveWell and the Open Philanthropy Project, which are currently part of the same organization, but which we plan to legally separate this year. GiveWell—referring to our longtime work to find and recommend top charities, as described on www.givewell.org—does not have an organizational position on the “last dollar” question for Good Ventures, because GiveWell’s mission is to recommend and move money to the charities that meet its criteria; it does not have an organizational mission to consider Good Ventures’ overall or long-term budget. However, staff members serving the Open Philanthropy Project do, since theirs is a long-term partnership with Good Ventures and they are actively weighing tradeoffs between grantmaking opportunities that Good Ventures will have resources to fund in the short- and long-run. The Open Philanthropy Project views the “last dollar” discussion you refer to above (http://www.openphilanthropy.org/blog/good-ventures-and-giving-now-vs-later-2016-update) as unstable, and thinks it would be a mistake to view its recent writeup as high confidence that Open Philanthropy Project opportunities are better—in some sort of absolute sense—than GiveWell top charities.

GiveWell believes the following is true:
- We do think GiveWell’s top charities represent the best giving opportunities we’re aware of for donors who have limited time to spend on their decision. That’s due to the strength of the evidence base, cost-effectiveness, and their transparency and ability to be vetted and spot-checked by donors with even a low degree of trust in GiveWell. As noted on GiveWell’s top charities page, “They represent the best opportunities we’re aware of to help low-income people with relatively high confidence and relatively short time horizons.”
- We also think that it’s possible that more cost-effective or otherwise ‘better’ giving opportunities exist, but that we a) haven’t found them yet, and/or b) don’t consider them as potential top charities because they fail to meet our criteria (e.g., not having a strong evidence base or not needing additional funding), which were designed to serve low-time donors and produce the types of recommendations described above.
- Some donors—who have a high degree of trust in a particular person or organization and want to outsource their thinking about giving to that person or organization, or who have a large amount of time with which to spend identifying and assessing giving opportunities—might identify other opportunities they feel offer the best bang for their buck, such as those identified by the Open Philanthropy Project.
This is discussed on GiveWell’s top charities page: www.givewell.org/charities/top-charities#Proscons.

We hope that the separation of the Open Philanthropy Project from GiveWell this year clarifies the difference in our approaches, which we believe is also a source of confusion.

Thanks again for your thoughtful post on our work.
What links here?
- Holden Karnofsky's comment on Some Thoughts on Public Discourse by Holden Karnofsky (7 Mar 2017 2:42 UTC; 7 points)

CatherineHollander 15 Feb 2019 17:13 UTC
10 points
0 ∶ 0
in reply to: Raemon’s comment on: How GiveWell’s Research is Evolving
Hi Raemon,
I work at GiveWell; thanks for your question. There are a few key differences with the Open Philanthropy Project:
1. Approach. While this work is still new and we’re unsure exactly how we’ll approach it, we continue to see our core activity as intensive critical assessment of specific giving opportunities to maximize the return of the funds we direct. By contrast, the core activity of the Open Philanthropy Project is intensive cause selection followed by intensive selection of Program Officers to lead grantmaking in those causes, and heavy reliance on the judgment of those Program Officers to identify hits.
  
  In the post above, we aim to highlight how our approach to conducting intensive critical assessment of giving opportunities (our research process) has changed. GiveWell traditionally relied on empirical, quantitative evidence (often randomized controlled trials and academic papers) as the primary input into this assessment. In the last few years, we began to expand the scope of the information we rely in our assessments. Our recent write-ups of the case for a grant to the Innovation in Government Initiative and our evaluation of Phase 1 of our grant to Results for Development are examples of how we have assessed new types of information in recent years.
2. Transparency. Transparency remains a core GiveWell value. We continue to be committed to publishing the full details of our research and recommendations as we evolve our research process so that our donors and others who rely on our recommendations can follow our logic and evaluate our evidence to make their own determinations about how to allocate their charitable giving. The Open Philanthropy Project’s mission is to give as effectively as it can and share its findings openly so that anyone can build on its work, but it does not aim for comprehensive information sharing such that the reasoning behind every decision can be understood and critiqued. The Open Philanthropy Project describes its approach to transparency here.
3. Focus area. GiveWell is focused on improving as much as possible the lives of humans today, and we focus on global health and development to do so. Even as our approach to research changes, we do not currently expect to look to areas focused explicitly on the long-term future or animal welfare, which are among the areas in which the Open Philanthropy Project makes grants. (The Open Philanthropy Project also supports global health and development, such as by supporting GiveWell’s top charities and GiveWell Incubation Grants.)

CatherineHollander 29 Jan 2020 22:51 UTC
50 points
0 ∶ 0
in reply to: Peter Wildeford’s comment on: Growth and the case against randomista development
Hi Peter,
Catherine from GiveWell here. We appreciate the dialogue this piece has generated. We agree that economic growth is an important area to consider evaluating, due to its potential for significant and positive impacts on well-being.
Today, our top charities list comprises charities implementing programs that have been studied via randomized controlled trials (RCTs). By pointing to these trials (and the monitoring conducted by our charities), we can serve our donors by making a public, vettable case for our recommendations and demonstrating their likely impact. We believe these are excellent, cost-effective opportunities for donors to help people alive today.
As John and Hauke note, GiveWell is not just focused on RCTs. We’ve expanded GiveWell’s focus to include new areas that may be more challenging to measure than the programs our current top charities implement,and we will therefore consider potential top charities that don’t have RCTs of their work. Our goal in expanding our focus is to identify programs that are more cost-effective than our current top charities (which we believe are highly cost-effective and difficult to beat). We wrote a blog post in February 2019 outlining our early plans for this work: https://blog.givewell.org/2019/02/07/how-givewells-research-is-evolving/.
We plan to expand our focus gradually, starting with areas in which we think we can make significant progress. We’re looking into health policy interventions—like alcohol control, ambient air pollution, micronutrient fortification in India, pesticide regulation, and lead paint regulation—based on our understanding of the existing research within these areas and our experience evaluating health interventions. From an institutional and research perspective, we think this is the right starting point for our expansion.
That doesn’t mean we’ll stop there. In that February 2019 blog post, one of the areas we listed as under consideration for future research was “Increasing economic growth and redistribution.” We hope to be able to deepen our understanding of this topic soon, although we don’t expect to do so in the very near future, so unfortunately don’t have substantive additions to the above discussion at this time.
Preliminarily, we guess that it might be particularly difficult to analyze giving opportunities in “economic growth” broadly because we perceive that growth is the result of a complex interplay between many different areas one could make grants in. These areas include infrastructure development, fiscal policy, monetary policy, industrial policy, peace and stability, individually-targeted programs, health, and so on. We haven’t yet done substantial work to map this space, but we expect that considering the more granular cause areas within the broad economic growth space will help us make progress on prioritizing further research.
We look forward to following the research that is done in this space and we are excited that other researchers are focusing on international development, as we think this will improve our research and recommendations in the long term.

CatherineHollander 12 Jan 2021 20:36 UTC
18 points
0 ∶ 0
on: How You Can Counterfactually Send Millions of Dollars to EA Charities
Hi Brendon and Tharun,
Catherine from GiveWell here. Thanks for your post—it’s generated some good discussion!
First, I want to confirm a few aspects of our banking that have been discussed here:
- A large proportion of the funding that we hold at any point in time is for making grants to our recommended charities. We typically grant those funds within 2-5 months of receiving them. Therefore, our balance fluctuates throughout the year as funds are received and granted.
- The vast majority of donations to GiveWell are made in December, which means that both our end-of-year and beginning-of-year balances are significantly higher than the typical balance on any given day of the year. Our January balance is higher because we have not yet granted the December donations, and because some year-end donations are processed in January, such as checks (which tend to be larger donations) that are mailed to us on December 31.
- Beyond funding for granting, we also maintain a balance of operating expenses.
We agree that consideration of where to hold and invest funds is important. Last year, we began work to revisit our banking practices. We also adopted an investment policy in December 2020 that enables us to invest a portion of our available cash balances in financial vehicles, with the following goals: preserving capital, meeting our liquidity requirements, minimizing volatility, and maximizing our after-tax return.
We are continuing our project to revisit our banking this year. In addition to the interest rates offered on accounts, our decisions about where to hold funds will also factor in our additional needs, such as strong customer service, security, understanding of non-profit vs. for-profit needs, user interface, connection with our various tools and platforms, and international operability. We plan to take this work forward in 2021.
What links here?
- Brendon_Wong's comment on Brendon_Wong’s Quick takes by Brendon_Wong (30 Aug 2022 5:57 UTC; 1 point)

CatherineHollander 26 Mar 2021 22:48 UTC
3 points
0 ∶ 0
in reply to: Jonas V’s comment on: EA Funds has appointed new fund managers
Thanks for raising these questions! I work at GiveWell, and we’re planning to update the EA Global Health and Development Fund page to make the distinction between it and the Maximum Impact Fund clearer—we think we can do better to explain the difference.
Here’s a quick summary:
- The Maximum Impact Fund is granted regularly to the highest-value funding opportunities we see among GiveWell’s recommended charities. This is a great option for donors who want to support GiveWell’s top charities and are open to their funding being used wherever it can do the most good among them.
  
  You can read more about the Maximum Impact Fund, and see our past allocations, here.
- The Effective Altruism Global Health and Development Fund has a broader remit, including programs that are new and about which little is known, as well as policy advocacy, public health regulation, technical assistance, and direct delivery programs, including GiveWell’s recommended charities.
  
  There’s no minimum requirement for evidence for grantees’ work. The Fund Manager (Elie) takes an expected value approach to calculating the potential impact of grants from the Global Health and Development Fund. He’ll recommend higher-risk grants when they are more effective (in expectation) than GiveWell top charities. He’ll fund GiveWell top charities if no such higher-expected-value grant opportunity exists.
  
  The list of past grants shows the breadth of programs that have received Global Health and Development Fund support. Here’s a grant for an RCT on the effects of mask-wearing on COVID-19, a grant for a policy advocacy organization, and a grant to GiveWell top charities.
  
  This is a great option for donors who are open to taking calculated risks when the expected value appears higher than GiveWell’s top charities, but are happy to support GiveWell’s top charities if not.
We don’t expect to change the overall portfolio of opportunities that we pursue in response to the allocation of donations between these two funds. In other words, as with most giving, there is fungibility. For example, if there are insufficient funds in the Global Health and Development Fund to support a high-leverage opportunity we want to fund, we would expect to seek funding for that opportunity from Open Philanthropy, with whom we work closely, or another donor.
I hope that helps clarify!

CatherineHollander 12 Oct 2021 23:54 UTC
5 points
0 ∶ 0
in reply to: freedomandutility’s comment on: Initial thoughts on malaria vaccine approval
Thanks for your question! I work at GiveWell.

The initial calculation we shared in the blog post is a simple one, intended to give a rough sense of the cost-effectiveness of each opportunity given the current limited investigation we’ve done of RTS,S. You’re correct that it doesn’t account for how RTS,S might interact with LLINs and SMC and the funding needs for those interventions; it’s possible that interventions will be layered atop one another, rather than an “either/or” situation.
We expect we would estimate the marginal value if we were deeply investigating an opportunity to fund RTS,S or if it seemed likely that the rollout of RTS,S was going to impact the cost-effectiveness of LLINs or SMC. Going forward, we’re also incorporating lower expected malaria rates in some locations in our SMC cost-effectiveness analysis due to the expectation of higher LLIN coverage in the future than the past. We’re doing the same where we’re funding LLINs to be delivered in areas where SMC is expanding.