Note: Before the launch of the Open Philanthropy Project Blog, this post appeared on the GiveWell Blog. Uses of “we” and “our” in the below post may refer to the Open Philanthropy Project or to GiveWell as an organization. Additional comments may be available at the original post.
When we first started GiveWell, we wondered why major staffed foundations didn’t write more about the thinking behind their giving (and the results of it), in order to share their knowledge and influence others. We’ve tried to counterbalance normal practice by making transparency one of our core values.
Our commitment to transparency is as strong as it’s ever been; we derive major benefits from it, and we believe there’s far too little public information and discussion about giving. At the same time, we’ve learned a lot about just why transparency in philanthropy is so difficult, and we no longer find it mysterious that it is so rare. This post summarizes what we see as the biggest challenges of being public and open about giving decisions.
Summary:
Everything we publish can help or hurt our brand. We put substantial effort into the accuracy, clarity and tone of our public content.
In most cases, writing about our thinking and our results also means writing about other organizations (the organizations we recommend and support, both via our traditional work and via the Open Philanthropy Project). We don’t want to hurt or upset other organizations, and we put substantial effort into making our public content both (a) amenable to the organizations we write about and (b) fair and complete in its characterization of our views.
The level of transparency we seek is unusual, meaning it often takes substantial effort to communicate our expectations and processes to the organizations we recommend and support.
The interaction of the above challenges can make it extremely difficult and time-consuming to write publicly about grants, recommendations, and grantee progress. In addition, it can be the cause of major delays between drafting and publication: much of our content takes weeks or months to go from draft to published piece, as we solicit feedback from parties who might be affected by the content.
The costs of transparency are significant, but we continue to feel they are outweighed by the benefits. Public writeups help clarify and improve our thinking; they play a major role in our credibility with our audience; and they represent a step toward a world in which there is far more, and better, information available to help donors give well.
We don’t think it necessarily makes sense for all philanthropic organizations to put as much effort into transparency as we do. Rather, we see transparency as one of the core areas in which we are trying to experiment, innovate, and challenge the status quo.
Challenge 1: protecting our brand
Because of the work we’ve put into explaining and defending our positions in the past, we benefit substantially from our reputation and from word-of-mouth. Nobody checks every statement and footnote on our site; even our closest readers rely often on the idea that content under the GiveWell name has a certain degree of thoroughness, reliability and clarity. (We believe a common way of approaching GiveWell content is to spot-check the occasional claim, rather than checking all claims or no claims; in order for our content to perform well under arbitrary spot-checks, our content needs to have fairly consistently high quality.)
Somewhat ironically, this dynamic means we’re hesitant to publish content that we haven’t thought through, checked out, and worded carefully (in order to say what we feel is important and defensible, and no more). We feel that poorly researched or poorly worded content could erode the brand we’ve built up, and could make people feel that they have to choose between checking everything we write themselves and simply placing less weight on our claims. (In general, most of our busy audience would likely choose the latter in this case.)
Giving decisions are generally impossible to justify purely with appeals to facts and logic; there are many judgment calls and a great deal of guesswork even in the most seemingly straightforward decisions. This makes it particularly challenging to write about them while preserving a basic level of credibility and a strong brand, and we don’t know of clear role models for this endeavor. (A funder once told me that s/he didn’t want to publish the reasoning behind giving decisions because this reasoning wasn’t up to academic standards, and so would not be perceived as reasonable or credible.)
Rather than aim to write only what we can back with hard evidence, and rather than write everything we believe regardless of the level of support, we put a great deal of effort into being clear about why we believe what we believe—whether it is because of solid evidence or simply a guess. (Phrases such as “we would guess that” are common in our content.) This allows us to share a good deal of our thinking (not just the parts of it that are strongly supported) while still maintaining credibility. But it requires a careful, thoughtful, and somewhat distinctive writing style that has been an ongoing challenge to develop and maintain.
As our brand becomes stronger, our audience becomes broader and our staff grows, the challenges of maintaining the appropriate style—and backing up our statements appropriately—intensifies. For example, we now put most public pages through a “vet”—in which a staff member who was not involved in writing the page goes carefully through its statements, making sure that each is appropriately supported—before publication. (We do not do this for all pages, and we generally do not do it for blog posts, which are more informal.)
Challenge 2: information about us is information about grantees
We seek to be highly open about the lessons we’ve learned and the results we’ve seen from our work—including developments that contradict our expectations and reflect poorly on our earlier decisions (which are often particularly valuable for learning). Because our function is to recommend other organizations for donations and grants, being open about our performance almost always means being open about another organization’s performance as well. (For simplicity, the rest of this section will refer to GiveWell-recommended organizations, as well as Open Philanthropy Project grantees, as “grantees.”)
While we want to be open, we don’t want to create a dynamic in which working with us creates significant risks for grantees. (This could lead good organizations to avoid working with us.) So we’ve had to find ways of balancing the goal of openness with the goal of making it “safe” for an organization to work with us. Doing so has been a major challenge and the subject of many long-running discussions, both internally and with grantees.
Things we’ve done to strike the right balance include:
Putting serious effort into communicating expectations up front. Simply saying “we value transparency” is not enough to communicate to a grantee what sorts of things we might want to write in the future. We generally try to send examples of past things we’ve written (such as our 2013 updates on Against Malaria Foundation and Schistosomiasis Control Initiative), and we often try to agree on an initial writeup before going forward with a grant or recommendation.
Giving grantees ample opportunity to comment on pending writeups that discuss them. There have been cases in which a writeup has been the subject of weeks, or even months, of discussion and negotiation.
Giving grantees a standing opportunity to retract non-public information, including even the fact that they’ve participated in our process. (Organizations considered as potential top charities have often been given the option to withdraw from our process and have us publish a page simply saying “Organization X declined to participate in our process”; this option has sometimes been invoked quite early in the process and has sometimes been invoked quite late, after a draft writeup has been produced and shared with the organization.)
Being generally hesitant to run a writeup that a grantee is highly uncomfortable with. We’re often willing to put substantial effort into working on a writeup’s language, until it both (a) communicates the important aspects of our views and (b) minimizes grantees’ concerns about giving misleading impressions.
In general, unusual goals are harder to achieve than common goals, because the rest of the world isn’t already set up to help with unusual goals. When we ask for budgets, project plans, confidentiality agreements, proof of 501(c)(3) status, etc., people immediately know what we’re seeking and are ready to provide it. When we bring up transparency, people are often surprised, confused, and cautious. In some cases people underestimate how much we plan to write, which could lead to problems later; in other cases people fear that we will disclose information carelessly and indiscriminately, leading them to be be highly wary. Discussions about transparency often involve extensive communication between senior staff at both organizations, in order to ensure that everyone is clear on what is being requested and expected.
We believe that we could achieve the same level of transparency with far less effort if our practices were even moderately common and familiar.
The difficulty of writing about grants
The interaction of the above challenges can make it extremely difficult and time-consuming to write publicly about grants, recommendations, and grantee progress. We can’t simply “open-source” our process: each piece of public content needs to simultaneously express our views, maintain our credibility, and be as amenable as possible to other organizations discussed therein. Much of our content takes weeks or months between drafting and publication.
With this in mind, we no longer find it puzzling that existing foundations tend to do little substantive public discussion of their work.
Benefits of transparency
The costs of transparency are significant, but we continue to feel they are outweighed by the benefits.
First, the process of drafting and refining public writeups is often valuable for our own thinking and reflection. In the process of discussing and negotiating content with grantees, we often become corrected on key points and gain better understanding of the situation. Writing about our work takes a lot of time, but much of that time is best classified as “refining and checking our thinking” rather than simply “making our thinking public.”
Second, transparency continues to be important for our credibility. This isn’t because all of our readers check all of our claims (in fact, we doubt that any of our readers check the majority of our claims). Rather, it’s because people are able to spot-check our reasoning. Our blog generally tries to summarize the big picture of why our priorities and recommendations are what they are; it links to pages that go into more detail, and these pages in turn use footnotes to provide yet more detail. A reader can pick any claim that seems unlikely, or is in tension with the reader’s background views, or is otherwise striking, and click through until they understand the reasoning behind the claim. This process often takes place in conversation rather than merely online—for example, see our research discussions. For these discussions, we rely on the fact that we’ve previously reached agreement with grantees on acceptable public formulations of our views and reasoning. Some readers do a lot of “spot-checking,” some do a little, and some merely rely on the endorsements of others. But without extensive public documentation of why we believe what we believe, we think we would have much more trouble being credible to all such people.
Finally, we believe that there is currently very little substantive public discussion of philanthropy, and that a new donor’s quest to learn about good giving is unnecessarily difficult. Work on the history of philanthropy is sparse, and doing new work in this area is challenging. Intellectuals tend to focus their thoughts and discussions on questions about public policy rather than philanthropy, making it hard to find good sources of ideas and arguments; we believe this is at least partly because of the dearth of public information about philanthropy.
We don’t think philanthropic transparency is easy, and we certainly don’t believe it’s something that foundations can jump into overnight. We don’t think it necessarily makes sense for all philanthropic organizations to put as much effort into transparency as we do. Rather, we see transparency as one of the core areas in which we are trying to experiment, innovate, and challenge the status quo.
In doing so, we hope to continue refining the processes necessary to achieve transparency, encouraging future (as well as present) foundations to adopt them, and making it easier for future organizations to be transparent than it currently is for us, so that one day there will be rich and abundant information available about how to give well.
Our ultimate goal is to do as much good as possible, and if we ever believe we might accomplish this better by dropping the emphasis on transparency, we will give serious consideration to the possibility. But at this time, the chance to promote philanthropic transparency is a major part of the case for GiveWell’s future impact, and we plan to retain transparency as a costly but essential goal.
Challenges of Transparency
Link post
Note: Before the launch of the Open Philanthropy Project Blog, this post appeared on the GiveWell Blog. Uses of “we” and “our” in the below post may refer to the Open Philanthropy Project or to GiveWell as an organization. Additional comments may be available at the original post.
When we first started GiveWell, we wondered why major staffed foundations didn’t write more about the thinking behind their giving (and the results of it), in order to share their knowledge and influence others. We’ve tried to counterbalance normal practice by making transparency one of our core values.
Our commitment to transparency is as strong as it’s ever been; we derive major benefits from it, and we believe there’s far too little public information and discussion about giving. At the same time, we’ve learned a lot about just why transparency in philanthropy is so difficult, and we no longer find it mysterious that it is so rare. This post summarizes what we see as the biggest challenges of being public and open about giving decisions.
Summary:
Everything we publish can help or hurt our brand. We put substantial effort into the accuracy, clarity and tone of our public content.
In most cases, writing about our thinking and our results also means writing about other organizations (the organizations we recommend and support, both via our traditional work and via the Open Philanthropy Project). We don’t want to hurt or upset other organizations, and we put substantial effort into making our public content both (a) amenable to the organizations we write about and (b) fair and complete in its characterization of our views.
The level of transparency we seek is unusual, meaning it often takes substantial effort to communicate our expectations and processes to the organizations we recommend and support.
The interaction of the above challenges can make it extremely difficult and time-consuming to write publicly about grants, recommendations, and grantee progress. In addition, it can be the cause of major delays between drafting and publication: much of our content takes weeks or months to go from draft to published piece, as we solicit feedback from parties who might be affected by the content.
The costs of transparency are significant, but we continue to feel they are outweighed by the benefits. Public writeups help clarify and improve our thinking; they play a major role in our credibility with our audience; and they represent a step toward a world in which there is far more, and better, information available to help donors give well.
We don’t think it necessarily makes sense for all philanthropic organizations to put as much effort into transparency as we do. Rather, we see transparency as one of the core areas in which we are trying to experiment, innovate, and challenge the status quo.
Challenge 1: protecting our brand
Because of the work we’ve put into explaining and defending our positions in the past, we benefit substantially from our reputation and from word-of-mouth. Nobody checks every statement and footnote on our site; even our closest readers rely often on the idea that content under the GiveWell name has a certain degree of thoroughness, reliability and clarity. (We believe a common way of approaching GiveWell content is to spot-check the occasional claim, rather than checking all claims or no claims; in order for our content to perform well under arbitrary spot-checks, our content needs to have fairly consistently high quality.)
Somewhat ironically, this dynamic means we’re hesitant to publish content that we haven’t thought through, checked out, and worded carefully (in order to say what we feel is important and defensible, and no more). We feel that poorly researched or poorly worded content could erode the brand we’ve built up, and could make people feel that they have to choose between checking everything we write themselves and simply placing less weight on our claims. (In general, most of our busy audience would likely choose the latter in this case.)
Giving decisions are generally impossible to justify purely with appeals to facts and logic; there are many judgment calls and a great deal of guesswork even in the most seemingly straightforward decisions. This makes it particularly challenging to write about them while preserving a basic level of credibility and a strong brand, and we don’t know of clear role models for this endeavor. (A funder once told me that s/he didn’t want to publish the reasoning behind giving decisions because this reasoning wasn’t up to academic standards, and so would not be perceived as reasonable or credible.)
Rather than aim to write only what we can back with hard evidence, and rather than write everything we believe regardless of the level of support, we put a great deal of effort into being clear about why we believe what we believe—whether it is because of solid evidence or simply a guess. (Phrases such as “we would guess that” are common in our content.) This allows us to share a good deal of our thinking (not just the parts of it that are strongly supported) while still maintaining credibility. But it requires a careful, thoughtful, and somewhat distinctive writing style that has been an ongoing challenge to develop and maintain.
As our brand becomes stronger, our audience becomes broader and our staff grows, the challenges of maintaining the appropriate style—and backing up our statements appropriately—intensifies. For example, we now put most public pages through a “vet”—in which a staff member who was not involved in writing the page goes carefully through its statements, making sure that each is appropriately supported—before publication. (We do not do this for all pages, and we generally do not do it for blog posts, which are more informal.)
Challenge 2: information about us is information about grantees
We seek to be highly open about the lessons we’ve learned and the results we’ve seen from our work—including developments that contradict our expectations and reflect poorly on our earlier decisions (which are often particularly valuable for learning). Because our function is to recommend other organizations for donations and grants, being open about our performance almost always means being open about another organization’s performance as well. (For simplicity, the rest of this section will refer to GiveWell-recommended organizations, as well as Open Philanthropy Project grantees, as “grantees.”)
While we want to be open, we don’t want to create a dynamic in which working with us creates significant risks for grantees. (This could lead good organizations to avoid working with us.) So we’ve had to find ways of balancing the goal of openness with the goal of making it “safe” for an organization to work with us. Doing so has been a major challenge and the subject of many long-running discussions, both internally and with grantees.
Things we’ve done to strike the right balance include:
Putting serious effort into communicating expectations up front. Simply saying “we value transparency” is not enough to communicate to a grantee what sorts of things we might want to write in the future. We generally try to send examples of past things we’ve written (such as our 2013 updates on Against Malaria Foundation and Schistosomiasis Control Initiative), and we often try to agree on an initial writeup before going forward with a grant or recommendation.
Giving grantees ample opportunity to comment on pending writeups that discuss them. There have been cases in which a writeup has been the subject of weeks, or even months, of discussion and negotiation.
Giving grantees a standing opportunity to retract non-public information, including even the fact that they’ve participated in our process. (Organizations considered as potential top charities have often been given the option to withdraw from our process and have us publish a page simply saying “Organization X declined to participate in our process”; this option has sometimes been invoked quite early in the process and has sometimes been invoked quite late, after a draft writeup has been produced and shared with the organization.)
Being generally hesitant to run a writeup that a grantee is highly uncomfortable with. We’re often willing to put substantial effort into working on a writeup’s language, until it both (a) communicates the important aspects of our views and (b) minimizes grantees’ concerns about giving misleading impressions.
We are in the process of creating a more formal process for negotiating about transparency with grantees up front. This process will draw on the agreement we negotiated with The Pew Charitable Trusts.
Challenge 3: transparency is unusual
In general, unusual goals are harder to achieve than common goals, because the rest of the world isn’t already set up to help with unusual goals. When we ask for budgets, project plans, confidentiality agreements, proof of 501(c)(3) status, etc., people immediately know what we’re seeking and are ready to provide it. When we bring up transparency, people are often surprised, confused, and cautious. In some cases people underestimate how much we plan to write, which could lead to problems later; in other cases people fear that we will disclose information carelessly and indiscriminately, leading them to be be highly wary. Discussions about transparency often involve extensive communication between senior staff at both organizations, in order to ensure that everyone is clear on what is being requested and expected.
We believe that we could achieve the same level of transparency with far less effort if our practices were even moderately common and familiar.
The difficulty of writing about grants
The interaction of the above challenges can make it extremely difficult and time-consuming to write publicly about grants, recommendations, and grantee progress. We can’t simply “open-source” our process: each piece of public content needs to simultaneously express our views, maintain our credibility, and be as amenable as possible to other organizations discussed therein. Much of our content takes weeks or months between drafting and publication.
With this in mind, we no longer find it puzzling that existing foundations tend to do little substantive public discussion of their work.
Benefits of transparency
The costs of transparency are significant, but we continue to feel they are outweighed by the benefits.
First, the process of drafting and refining public writeups is often valuable for our own thinking and reflection. In the process of discussing and negotiating content with grantees, we often become corrected on key points and gain better understanding of the situation. Writing about our work takes a lot of time, but much of that time is best classified as “refining and checking our thinking” rather than simply “making our thinking public.”
Second, transparency continues to be important for our credibility. This isn’t because all of our readers check all of our claims (in fact, we doubt that any of our readers check the majority of our claims). Rather, it’s because people are able to spot-check our reasoning. Our blog generally tries to summarize the big picture of why our priorities and recommendations are what they are; it links to pages that go into more detail, and these pages in turn use footnotes to provide yet more detail. A reader can pick any claim that seems unlikely, or is in tension with the reader’s background views, or is otherwise striking, and click through until they understand the reasoning behind the claim. This process often takes place in conversation rather than merely online—for example, see our research discussions. For these discussions, we rely on the fact that we’ve previously reached agreement with grantees on acceptable public formulations of our views and reasoning. Some readers do a lot of “spot-checking,” some do a little, and some merely rely on the endorsements of others. But without extensive public documentation of why we believe what we believe, we think we would have much more trouble being credible to all such people.
Finally, we believe that there is currently very little substantive public discussion of philanthropy, and that a new donor’s quest to learn about good giving is unnecessarily difficult. Work on the history of philanthropy is sparse, and doing new work in this area is challenging. Intellectuals tend to focus their thoughts and discussions on questions about public policy rather than philanthropy, making it hard to find good sources of ideas and arguments; we believe this is at least partly because of the dearth of public information about philanthropy.
We don’t think philanthropic transparency is easy, and we certainly don’t believe it’s something that foundations can jump into overnight. We don’t think it necessarily makes sense for all philanthropic organizations to put as much effort into transparency as we do. Rather, we see transparency as one of the core areas in which we are trying to experiment, innovate, and challenge the status quo.
In doing so, we hope to continue refining the processes necessary to achieve transparency, encouraging future (as well as present) foundations to adopt them, and making it easier for future organizations to be transparent than it currently is for us, so that one day there will be rich and abundant information available about how to give well.
Our ultimate goal is to do as much good as possible, and if we ever believe we might accomplish this better by dropping the emphasis on transparency, we will give serious consideration to the possibility. But at this time, the chance to promote philanthropic transparency is a major part of the case for GiveWell’s future impact, and we plan to retain transparency as a costly but essential goal.