It’s a good question, and one that we ask ourselves a lot. If we thought we were worse than AMF and that wasn’t likely to change, we would close up shop. I am fairly confident that we produce more value than AMF, partly because our activities raise more for AMF than they take away. However, I think it’s right to be uncertain about this and Owen makes some good points.
In addition, I think most of the value of CEA’s activities comes from long term potential of our projects and EA as a whole—as Ben discusses here.
Our positive effect on AMF is clearest at Giving What We Can which has a return of roughly 100:1 in high-value donations (counterfactually adjusted and time-discounted, but not all to AMF). Even if you assume that not a single member of GWWC gives another penny ever, the ratio is still 5:1. It is unclear if the marginal return on a donation to GWWC is higher or lower than the average return. It would be higher if we thought that GWWC could still realise increasing economies of scale. It would be lower if we thought most of the value comes from the idea itself and not execution on it. I tend to think marginal funds are more effective than average funds, but I’m very uncertain. A fuller discussion is here (http://effective-altruism.com/ea/ql/giving_what_we_can_needs_your_help_this_christmas/)
At 80k, the metrics are less directly comparable. At the last review we estimated it cost £1,670 to achieve a significant plan change (and these costs have been coming down every review cycle, indicating we are getting more cost-effective). It’s unclear how much each plan change is worth—but it seems very likely that getting someone to earn-to-give or move to do valuable direct work will be worth far more than £1,670 to AMF even within one year.
GPP impact is extremely hard to estimate because idea change and policy-work are chaotic and complex. In order to get a lower bound, we can focus on just one policy that we advocated which was successfully implemented—increasing the research budget for treatment and vaccines for malaria, TB, and NTDs and pandemic prevention by £2.5bn over 5 years. If our calculations are correct this move was worth $1.5bn-$30bn in donations to AMF. Even if we are only responsible for a very small part of this, it isn’t hard to imagine our 2015 budget outperformed a donation to AMF. (See discussion here http://globalprioritiesproject.org/2015/12/new-uk-aid-strategy-prioritising-research-and-crisis-response/)
EA Outreach is probably hardest to compare directly against AMF-type charities because much of our estimate of its value depends on the fact that we think effective altruism and its ideas have huge upside potential. Any attempt to calculate the direct impact within the first year of its running in terms of money to AMF would short-change the value of the work.
Because most of the money that goes to CEA has a huge counterfactual positive impact on funding for AMF, I’m quite confident in recommending giving to CEA.
With respect to your question about growth in costs—I think Owen has some good thoughts here. It seems, however, that the unit costs of CEA outputs are stable or decreasing so the growth in costs represents expanding outputs rather than decreasing marginal returns.
Our positive effect on AMF is clearest at Giving What We Can which has a return of roughly 100:1 in high-value donations (counterfactually adjusted and time-discounted, but not all to AMF). Even if you assume that not a single member of GWWC gives another penny ever, the ratio is still 5:1.
That’s precisely what’s at issue. For one I don’t find it all convincing, having talked with people who have been experienced with the organisation. And prima facie it’s implausibly profitable. So it needs more justification than the prospectus gives.
This statement is interesting, because it suggests at least some of the disagreement is about priors/reference classes. My prior for the ratio achievable was really quite broad. It sounds like you had a much tighter prior, which would decrease the extent to which you want to update on evidence.
I don’t know whether that disagreement is resolvable, but here are some of the thoughts that inform my prior:
You talk about ‘profitable’, which suggests businesses as a reference class. I agree that that kind of ratio is implausible for businesses, but I think that’s a function of competition—if it were available, someone would have been doing it already and got rich as a result. Monopolists can get much more profitable than non-monopolists. I think there are quite a lot of analogies with a business, so it’s not ridiculous to consider them as a reference class, but I also think we understand the basic mechanism which stops them getting too profitable and it doesn’t apply here, so we should not weigh this that strongly.
A closer reference class seems to be fundraising for charities. The institute of fundraising estimates median returns for different kind of fundraising activity to vary between around 1.5:1 and around 30:1, depending on the activity (link). The ratio for campaigns to encourage committed giving to the charity running the campaign is around 6:1. Note that these numbers are sustained despite what is probably some competition between charities (I’m mildly surprised by this).
The activity that GWWC is engaging in is not fundraising for itself, but encouraging people to give (and give effectively). Compared to charities fundraising for themselves, there is less competition, and the approach is also more novel: both of these could support more of the low-hanging fruit still being available. Moreover it may be easier to persuade people to give when there is no obvious conflict-of-interest of the charity receiving funds being the same as the people trying to persuade you.
Thanks, this is helpful (though as you predict not by itself not enough to resolve the issue). Fundraising seems a good reference class—not too broad (like ‘all businesses’ would be) and not too narrow. One comment/question, at least for now:
The activity that GWWC is engaging in is not fundraising for itself, but encouraging people to give (and give effectively). Compared to charities fundraising for themselves, there is less competition, and the approach is also more novel: both of these could support more of the low-hanging fruit still being available. Moreover it may be easier to persuade people to give when there is no obvious conflict-of-interest of the charity receiving funds being the same as the people trying to persuade you.
This seems the main reason that could account for your fundraising being so much more profitable than normal. The lack of conflict of interest could help, and I’ve read Charity Science use the same argument somewhere. But it has very limited strength, there are many independent people who fundraise for charities they’re passionate about, and it’s hard to see why it’d drive up fundraising profitability that much. That would take a novel approach in an enviroment of low hanging fruit (because low competetition). What exactly is GWWC’s approach of this sort? I’m still not clear what you will do with the staff time our money buys to churn out a hundred dollars per dollar.
First is the focus on effective giving. This makes the case for giving much stronger, to those who are convinced by the arguments. Your giving is really saving lives.
Related, the case is supported by analytical arguments, which really appeals to a certain type of person, who often isn’t engaged by existing charity.
Second is the size of the ask. Most charity fundraising focuses on small donations. GWWC focuses on a 10% lifetime pledge. This is much harder to get, but results in much more money. It seems like the extra difficultly doesn’t fully offset the extra money (at least when combined with the first point).
Now GWWC also has the advantages of a strong community, lots of experience, credibility, a large audience etc. which make it easier and easier to get more pledges on the margin.
Another way that Giving What We Can can beat normal fundraisers and faces little competition is this:
The vast majority of fundraising is done by organisations raising money for themselves. And it’s very rare someone will be willing to make a lifetime commitment to a specific organisation, because they want to stay flexible in where they donate.
As a result, for most fundraising organisations there’s no sense in them pushing for such a major long-term commitment. It won’t work. They could in theory ask for a lifetime commitment to any organisation, not just themselves, but in addition to being very weird, that doesn’t help them achieve their fundraising goals when people move on to other groups.
So one reason this approach is very neglected, is it’s only sensible for those who are fundraising for other groups, or a very general cause like ‘the common good’. And that’s a very niche and little explored approach.
A possible exception is churches/temples/mosques which people expect to be involved with for their entire life, but then they did have tithing and collected enormous sums that way.
One thing to bear in mind is that there will naturally be quite a bit of variance in fundraising ratios. There was a factor-of-20 difference between median returns from standard methods, and I’m sure quite a bit of variance for each method according to implementation. I think the GWWC team is quite talented and it would be hard for an arbitrary charity to duplicate it at the same salaries, which might make you think they’d be in the positive tail for the method chosen.
However, I think you’re right that “no conflict-of-interest” probably doesn’t carry you so far by itself.
I think the main new thing that GWWC has been doing is asking people to donate a large amount on an ongoing basis. Compared to normal ongoing giving which might be £5/month for five years, the GWWC pledge may be two or three orders of magnitude larger. The question is how much lower the rate of getting people is. My prior uncertainty on this would be extremely large—it’s obvious that it will be lower, but unclear whether just a bit or 6+ orders of magnitude.
I think the big ask means people thinking seriously about their lives, rather than making an in-the-moment decision (as I think most charitable donations are). I think it’s also much less plausible for individual charities to get this commitment from people than a general encouragement to give more—this is the proper force of my previous “no conflict-of-interest” point, but it’s broader than that because the GWWC pledge also doesn’t involve binding your future judgements about which charities actually are best.
Now, I’m not sure this is entirely new. People have tried to persuade others to be generous before. But it’s plausible that such efforts have in fact always been very effective. Because of a lack of data, and because no individual charity could scale this up to get large income for themselves, it’s not clear that the market would have been saturated even if it always were a great activity.
You say something like this every time Giving What We Can’s multiple gets discussed, but never point out specific problems with the estimates, or provide alternative estimates.
I agree that if you’d asked me five years ago what one could expect in a fundraising ratio I would have been surprised by estimates like 100:1. Most charitable fundraising is in the ballpark of 10:1. Nevertheless, the folks at GWWC are very methodical about gathering huge amounts of data and processing it carefully and transparently. If you have any specific suggestions for the methodology I’d be very open to exploring them.
Break down which activities have led to which members in as much detail as possible.
Justify the “Counter-factual donation rate” more deeply. Use a graduate volunteer’s time to dig into it and present multiple explorations of it, some of which don’t rely on people’s subjective estimates of it when asked by GWWC at the time they’re pledging to it. Include some in-depth exploration of the counter-factual rate for a few members.
The GWWC fundraising prospectus sets out in quite extensive detail the observations and assumptions that underlie the figures, as well as providing the spreadsheets to let you explore how your own probability estimates would change them.
What further information do you think should be included?
There are indeed great questions there with extensive responses, many of which point to information that was already publicly available.
I think it’s awesome for people to ask questions: in the GWWC fundraising post there’s been a really productive discussion. But here, as on a previous occasion, you seem to be suggesting there is some deception going on. You’ve suggested in another post that you see these responses as ‘punching up’, but by keeping it vague it also looks a lot like mud-slinging (as opposed to an airing of your concerns, which I hope everybody would be keen to have happen).
Hopefully you’ll have time to elaborate on your concerns soon.
Hey Impala,
You can find our entire methodology for those on our impact page, and we’ve been answering detailed questions and providing additional requested data on this forum post. We didn’t put all of it in the prospectus this time partly because we didn’t want to swamp people and partly because we think that this is not that useful for evaluating whether to give to us, compared to looking at our overall plans and team (see this helpful post from Ben Todd). It is more like a check to make sure we are continuing to plenty of money to effective charities compared to the money we are spending than the precise value we expect our leverage ratio to stably be going into the future. On the other hand, having a high leverage ratio so far does seem to be a good sign overall of how things are going. Finding it prima facie implausible seems like a good reason to think that it may decrease quite substantially in the future, but not a good reason to avoid updating on it at all.
Could you elaborate on what you think we might be doing wrong, or what you think we could improve on? That way I might be able to provide more insight how why we act as we do, or improve our workings.
I wanted to add that Givewell, in their recent board meeting mentioned that they moved more than $20 Million to top charities excluding GoodVentures money in 2015. They said that Effective Altruism becoming more popular was the primary driver of increased web traffic this year based on analytics (note that this a non-verbatim summary).
source: http://www.givewell.org/about/official-records/board-meeting-31
Thus, I think there’s a very good case to be made for EA outreach being very valuable.
There should however be significant question, that the EA Outreach team should answer, as to counterfactual growth, as well as whether the growth activities are putting EA in a better or worse position reputationally (and thus adding to or detracting from long-term growth potential).
We should distinguish between EA Outreach the organization and EA outreach the activity. I agree that the activity of EA outreach is very likely to be valuable, but I agree with Josh that EA Outreach the organization may not be effective despite this.
Unfortunately we’re not in a great position to know the counterfactuals on people added to the movement from our activity. Upper bound on new EAs added through EA Global is 110 people. I expect the actual number to be lower but I’m not sure by how much.
We also sold 17,000 books so far, but we don’t know the conversion rate from books to becoming involved in the EA community and we don’t know how many counterfactual sales we were responsible for.
In terms of the long-term potential of the movement, I feel quite confident defending that our 2016 plan will put the EA movement in a much better position in the future.
It’s a good question, and one that we ask ourselves a lot. If we thought we were worse than AMF and that wasn’t likely to change, we would close up shop. I am fairly confident that we produce more value than AMF, partly because our activities raise more for AMF than they take away. However, I think it’s right to be uncertain about this and Owen makes some good points.
In addition, I think most of the value of CEA’s activities comes from long term potential of our projects and EA as a whole—as Ben discusses here.
Our positive effect on AMF is clearest at Giving What We Can which has a return of roughly 100:1 in high-value donations (counterfactually adjusted and time-discounted, but not all to AMF). Even if you assume that not a single member of GWWC gives another penny ever, the ratio is still 5:1. It is unclear if the marginal return on a donation to GWWC is higher or lower than the average return. It would be higher if we thought that GWWC could still realise increasing economies of scale. It would be lower if we thought most of the value comes from the idea itself and not execution on it. I tend to think marginal funds are more effective than average funds, but I’m very uncertain. A fuller discussion is here (http://effective-altruism.com/ea/ql/giving_what_we_can_needs_your_help_this_christmas/)
At 80k, the metrics are less directly comparable. At the last review we estimated it cost £1,670 to achieve a significant plan change (and these costs have been coming down every review cycle, indicating we are getting more cost-effective). It’s unclear how much each plan change is worth—but it seems very likely that getting someone to earn-to-give or move to do valuable direct work will be worth far more than £1,670 to AMF even within one year.
GPP impact is extremely hard to estimate because idea change and policy-work are chaotic and complex. In order to get a lower bound, we can focus on just one policy that we advocated which was successfully implemented—increasing the research budget for treatment and vaccines for malaria, TB, and NTDs and pandemic prevention by £2.5bn over 5 years. If our calculations are correct this move was worth $1.5bn-$30bn in donations to AMF. Even if we are only responsible for a very small part of this, it isn’t hard to imagine our 2015 budget outperformed a donation to AMF. (See discussion here http://globalprioritiesproject.org/2015/12/new-uk-aid-strategy-prioritising-research-and-crisis-response/)
EA Outreach is probably hardest to compare directly against AMF-type charities because much of our estimate of its value depends on the fact that we think effective altruism and its ideas have huge upside potential. Any attempt to calculate the direct impact within the first year of its running in terms of money to AMF would short-change the value of the work.
Because most of the money that goes to CEA has a huge counterfactual positive impact on funding for AMF, I’m quite confident in recommending giving to CEA.
With respect to your question about growth in costs—I think Owen has some good thoughts here. It seems, however, that the unit costs of CEA outputs are stable or decreasing so the growth in costs represents expanding outputs rather than decreasing marginal returns.
That’s precisely what’s at issue. For one I don’t find it all convincing, having talked with people who have been experienced with the organisation. And prima facie it’s implausibly profitable. So it needs more justification than the prospectus gives.
This statement is interesting, because it suggests at least some of the disagreement is about priors/reference classes. My prior for the ratio achievable was really quite broad. It sounds like you had a much tighter prior, which would decrease the extent to which you want to update on evidence.
I don’t know whether that disagreement is resolvable, but here are some of the thoughts that inform my prior:
You talk about ‘profitable’, which suggests businesses as a reference class. I agree that that kind of ratio is implausible for businesses, but I think that’s a function of competition—if it were available, someone would have been doing it already and got rich as a result. Monopolists can get much more profitable than non-monopolists. I think there are quite a lot of analogies with a business, so it’s not ridiculous to consider them as a reference class, but I also think we understand the basic mechanism which stops them getting too profitable and it doesn’t apply here, so we should not weigh this that strongly.
A closer reference class seems to be fundraising for charities. The institute of fundraising estimates median returns for different kind of fundraising activity to vary between around 1.5:1 and around 30:1, depending on the activity (link). The ratio for campaigns to encourage committed giving to the charity running the campaign is around 6:1. Note that these numbers are sustained despite what is probably some competition between charities (I’m mildly surprised by this).
The activity that GWWC is engaging in is not fundraising for itself, but encouraging people to give (and give effectively). Compared to charities fundraising for themselves, there is less competition, and the approach is also more novel: both of these could support more of the low-hanging fruit still being available. Moreover it may be easier to persuade people to give when there is no obvious conflict-of-interest of the charity receiving funds being the same as the people trying to persuade you.
Thanks for elaborating, I found that very helpful. I also felt like the ratio was implausibly high a priori.
Thanks, this is helpful (though as you predict not by itself not enough to resolve the issue). Fundraising seems a good reference class—not too broad (like ‘all businesses’ would be) and not too narrow. One comment/question, at least for now:
This seems the main reason that could account for your fundraising being so much more profitable than normal. The lack of conflict of interest could help, and I’ve read Charity Science use the same argument somewhere. But it has very limited strength, there are many independent people who fundraise for charities they’re passionate about, and it’s hard to see why it’d drive up fundraising profitability that much. That would take a novel approach in an enviroment of low hanging fruit (because low competetition). What exactly is GWWC’s approach of this sort? I’m still not clear what you will do with the staff time our money buys to churn out a hundred dollars per dollar.
There’s a couple of new bits.
First is the focus on effective giving. This makes the case for giving much stronger, to those who are convinced by the arguments. Your giving is really saving lives.
Related, the case is supported by analytical arguments, which really appeals to a certain type of person, who often isn’t engaged by existing charity.
Second is the size of the ask. Most charity fundraising focuses on small donations. GWWC focuses on a 10% lifetime pledge. This is much harder to get, but results in much more money. It seems like the extra difficultly doesn’t fully offset the extra money (at least when combined with the first point).
Now GWWC also has the advantages of a strong community, lots of experience, credibility, a large audience etc. which make it easier and easier to get more pledges on the margin.
Another way that Giving What We Can can beat normal fundraisers and faces little competition is this:
The vast majority of fundraising is done by organisations raising money for themselves. And it’s very rare someone will be willing to make a lifetime commitment to a specific organisation, because they want to stay flexible in where they donate.
As a result, for most fundraising organisations there’s no sense in them pushing for such a major long-term commitment. It won’t work. They could in theory ask for a lifetime commitment to any organisation, not just themselves, but in addition to being very weird, that doesn’t help them achieve their fundraising goals when people move on to other groups.
So one reason this approach is very neglected, is it’s only sensible for those who are fundraising for other groups, or a very general cause like ‘the common good’. And that’s a very niche and little explored approach.
A possible exception is churches/temples/mosques which people expect to be involved with for their entire life, but then they did have tithing and collected enormous sums that way.
One thing to bear in mind is that there will naturally be quite a bit of variance in fundraising ratios. There was a factor-of-20 difference between median returns from standard methods, and I’m sure quite a bit of variance for each method according to implementation. I think the GWWC team is quite talented and it would be hard for an arbitrary charity to duplicate it at the same salaries, which might make you think they’d be in the positive tail for the method chosen.
However, I think you’re right that “no conflict-of-interest” probably doesn’t carry you so far by itself.
I think the main new thing that GWWC has been doing is asking people to donate a large amount on an ongoing basis. Compared to normal ongoing giving which might be £5/month for five years, the GWWC pledge may be two or three orders of magnitude larger. The question is how much lower the rate of getting people is. My prior uncertainty on this would be extremely large—it’s obvious that it will be lower, but unclear whether just a bit or 6+ orders of magnitude.
I think the big ask means people thinking seriously about their lives, rather than making an in-the-moment decision (as I think most charitable donations are). I think it’s also much less plausible for individual charities to get this commitment from people than a general encouragement to give more—this is the proper force of my previous “no conflict-of-interest” point, but it’s broader than that because the GWWC pledge also doesn’t involve binding your future judgements about which charities actually are best.
Now, I’m not sure this is entirely new. People have tried to persuade others to be generous before. But it’s plausible that such efforts have in fact always been very effective. Because of a lack of data, and because no individual charity could scale this up to get large income for themselves, it’s not clear that the market would have been saturated even if it always were a great activity.
Note: Ben’s answer is better (more comprehensive and more to the point) than mine.
You say something like this every time Giving What We Can’s multiple gets discussed, but never point out specific problems with the estimates, or provide alternative estimates.
You can find the detailed calculations here.
I agree that if you’d asked me five years ago what one could expect in a fundraising ratio I would have been surprised by estimates like 100:1. Most charitable fundraising is in the ballpark of 10:1. Nevertheless, the folks at GWWC are very methodical about gathering huge amounts of data and processing it carefully and transparently. If you have any specific suggestions for the methodology I’d be very open to exploring them.
Thank you, my top two suggestions would be:
Break down which activities have led to which members in as much detail as possible.
Justify the “Counter-factual donation rate” more deeply. Use a graduate volunteer’s time to dig into it and present multiple explorations of it, some of which don’t rely on people’s subjective estimates of it when asked by GWWC at the time they’re pledging to it. Include some in-depth exploration of the counter-factual rate for a few members.
The GWWC fundraising prospectus sets out in quite extensive detail the observations and assumptions that underlie the figures, as well as providing the spreadsheets to let you explore how your own probability estimates would change them.
What further information do you think should be included?
I’ll give my own answer when I get time but the questions at http://effective-altruism.com/ea/ql/giving_what_we_can_needs_your_help_this_christmas/ look like a decent start.
There are indeed great questions there with extensive responses, many of which point to information that was already publicly available.
I think it’s awesome for people to ask questions: in the GWWC fundraising post there’s been a really productive discussion. But here, as on a previous occasion, you seem to be suggesting there is some deception going on. You’ve suggested in another post that you see these responses as ‘punching up’, but by keeping it vague it also looks a lot like mud-slinging (as opposed to an airing of your concerns, which I hope everybody would be keen to have happen).
Hopefully you’ll have time to elaborate on your concerns soon.
Hey Impala, You can find our entire methodology for those on our impact page, and we’ve been answering detailed questions and providing additional requested data on this forum post. We didn’t put all of it in the prospectus this time partly because we didn’t want to swamp people and partly because we think that this is not that useful for evaluating whether to give to us, compared to looking at our overall plans and team (see this helpful post from Ben Todd). It is more like a check to make sure we are continuing to plenty of money to effective charities compared to the money we are spending than the precise value we expect our leverage ratio to stably be going into the future. On the other hand, having a high leverage ratio so far does seem to be a good sign overall of how things are going. Finding it prima facie implausible seems like a good reason to think that it may decrease quite substantially in the future, but not a good reason to avoid updating on it at all. Could you elaborate on what you think we might be doing wrong, or what you think we could improve on? That way I might be able to provide more insight how why we act as we do, or improve our workings.
I wanted to add that Givewell, in their recent board meeting mentioned that they moved more than $20 Million to top charities excluding GoodVentures money in 2015. They said that Effective Altruism becoming more popular was the primary driver of increased web traffic this year based on analytics (note that this a non-verbatim summary). source: http://www.givewell.org/about/official-records/board-meeting-31
Thus, I think there’s a very good case to be made for EA outreach being very valuable.
The concept of EA Outreach is surely valuable.
There should however be significant question, that the EA Outreach team should answer, as to counterfactual growth, as well as whether the growth activities are putting EA in a better or worse position reputationally (and thus adding to or detracting from long-term growth potential).
We should distinguish between EA Outreach the organization and EA outreach the activity. I agree that the activity of EA outreach is very likely to be valuable, but I agree with Josh that EA Outreach the organization may not be effective despite this.
Unfortunately we’re not in a great position to know the counterfactuals on people added to the movement from our activity. Upper bound on new EAs added through EA Global is 110 people. I expect the actual number to be lower but I’m not sure by how much.
We also sold 17,000 books so far, but we don’t know the conversion rate from books to becoming involved in the EA community and we don’t know how many counterfactual sales we were responsible for.
In terms of the long-term potential of the movement, I feel quite confident defending that our 2016 plan will put the EA movement in a much better position in the future.