Hi Dale,
Thanks for your post. It’s great to see someone take such a deep interest, and thank you for the specificity of your suggestions. As it happens, we’re about to put up an impact evaluation. I’m sorry it’s been a little while since the previous one (done by Nick Beckstead). I was hoping to do it a bit earlier, but was tied up finishing my PhD. It will address various points like what we might expect the value of a new member to be, and how this changes with different discount rates. I’ll take your comments, like about wanting to be able to follow individual cohorts, into consideration. We’ve also recently taken on a Community Director (Alison Woodman) who focuses on members – both to improve membership retention and to increase our understanding of our membership for impact evaluation purposes. So hopefully in the future our impact evaluations will be more thorough and frequent.
Jacob’s already addressed the most pressing points. I just had a couple of other considerations:
As he said, the main thing in December was the Pledge event, which was amazing. Giving season and new year’s resolutions always cause a bit of a bump in membership too.
I’m don’t think that we should think of members who joined under the newer pledge as ‘lower quality’. The new pledge is more general, but it is so in the sense that it is cause neutral. People who plan to donate to animal charities because they think that they can help fellow creatures even more by doing so seem every bit as serious in helping others effectively as those donating to eradicate extreme poverty. We’ve even had a couple of people join because of the change in the pledge who donate to poverty eradication charities, but who thought that we might in future find an even more effective cause than that, and wanted to make sure if they took a lifelong pledge they would definitely be able to keep it. These people seem at least as likely to continue donating as others.
While I agree that it’s easier to join online than by sending off a letter, I’m less convinced that the new method of joining is going to be one which makes it less likely people feel engaged. We still send people a physical pledge form, to maintain the feeling of solidity. But doing it online means that an email immediately gets sent to the person, and we now make more effort to follow-up with and get to know members (which we have more capacity for because of the reduction in admin moving to the online system).
By the way, you seem very comfortable with numbers, knowledgable and engaged – how would you feel about working with our research team? We’d be very grateful for the help!
As it happens, we’re about to put up an impact evaluation.
Glad to hear it! I look forward to seeing the data.
I’m don’t think that we should think of members who joined under the newer pledge as ‘lower quality’. … People who plan to donate to animal charities because they think that they can help fellow creatures even more by doing so seem every bit as serious in helping others effectively as those donating to eradicate extreme poverty.
Well, that’s true unless you don’t think animals count as ‘others’. Presumably we should discount their value by whatever credence we place in animals counting as moral agents. A wide range of new focuses are now acceptable to GWWC members, and regardless of one’s specific personal beliefs, it seems likely that some of those members will focus on something you don’t personally think is very valuable. For example, it seems plausible that some people might (falsely) think that donating to a political party was the best way of helping people. Yet presumably no-one would think that increasing donations to both Democrat and Republican parties by $100 is equal to $200 going to AMF!
However, my complaint wasn’t really about the target of their donations—I assume that in practice most will mainly donate to AMF etc. anyway—but that they’ll see a higher dropout rate. I don’t think this concern can really be verified or refuted without seeing the data.
But doing it online means that an email immediately gets sent to the person,
This seems like a bit of a misnomer. People were always requesting a signup form online—you could combine paper forms in the post with an email saying ‘Welcome … the forms are on their way’.
Your point about the overhead costs is a good one though. My impression is this could be outsourced—at least in the US I’m pretty sure there are firms that will deal with your mailing for you, but that still costs money.
By the way, you seem very comfortable with numbers, knowledgable and engaged – how would you feel about working with our research team? We’d be very grateful for the help!
Very kind of you. Unfortunately I work a more-than-full-time job, so can’t really commit to anything. However, I get the impression that the EA community has a fair number of math-literate people who would be willing to work on relevant questions just out of interest if the data was available. In this case I’d probably analyse the cohort data if it was available—and the same probably goes for object-level issues as well. This would have a couple of advantages:
Save CEA resources
Get third parties more involved
Provide independent evaluation, which is more credible than internal evaluation (in the same way that companies get independent auditors, etc.)
Regardless of one’s specific personal beliefs, it seems likely that some of those members will focus on something you don’t personally think is very valuable
The relevance of this isn’t clear. Both before and after the change, every particular GWWC donor is giving to a charity that the donor likes but that some people don’t consider maximally valuable. The question is whether the change made things worse. On average, I expect it made things better, since I expect cause neutrality to be helpful.
Perhaps your point is that previously this impact was valuable to one constituency—people who prioritize development—whereas now the value is spread to a wider range of donors with a wider range of values. I can see why this might make fundraising harder, since it’s a classic example of a public goods problem, but it doesn’t seem to make the project less impactful overall.
but that they’ll see a higher dropout rate
I expect that GWWC’s dropout rate is increasing over time. I don’t see why this change in particular would be expected to lead to a higher dropout rate. At face value, one might expect a broader pledge to have a lower dropout rate.
On average, I expect it made things better, since I expect cause neutrality to be helpful.
I’m implicitly assuming that global poverty is an unusually uncontroversial cause.
I don’t see why this change in particular would be expected to lead to a higher dropout rate. At face value, one might expect a broader pledge to have a lower dropout rate.
I guess in the limit if the pledge became sufficiently expansive it would become impossible to fail to comply!
I think locally the effect is likely to be negative, though I don’t have much confidence either way till we see the data. Nor do I see much value in speculating on this, save to encourage the release of such data.
Thanks! We actually have had quite a few people with expertise go over the impact evaluation—as you say, it’s pretty fun. I was thinking of other things we could do with more researchers on. Sounds like you’re already plenty busy though!
I’m sorry it’s been a little while since the previous one (done by Nick Beckstead).
Here is that impact analysis. It’s well worth a read, particularly the section ‘Overview of GWWC’s performance so far’ as this speaks to many of Dale’s questions.
only 293 of 705 (41.6%) of the people who applied to join GWWC have returned their pledge forms.
The fraction of people who claimed to be fulfilling the pledge in 2011 (I take it GWWC does not have tax receipts etc.) was basically unvarying by year of pledge: 64% of 2009 pledges, 71% of 2010 pledges and 64% of 2011 pledges. This suggests that the ‘more keen people join first’ and ‘people drop out over time’ effects perfectly canceled out. (Though the sample size is small). However 65-70% claimed retention after 0-2 years suggests a relatively short half-life for members.
However 65-70% claimed retention after 0-2 years suggests a relatively short half-life for members.
It occurs to me that, if true, this would have important implications for GWWC’s marketing, namely that we should deemphasize students, as many of them would have dropped out before they start making bank, and increase the emphasis on mature targets.
Hi Dale, Thanks for your post. It’s great to see someone take such a deep interest, and thank you for the specificity of your suggestions. As it happens, we’re about to put up an impact evaluation. I’m sorry it’s been a little while since the previous one (done by Nick Beckstead). I was hoping to do it a bit earlier, but was tied up finishing my PhD. It will address various points like what we might expect the value of a new member to be, and how this changes with different discount rates. I’ll take your comments, like about wanting to be able to follow individual cohorts, into consideration. We’ve also recently taken on a Community Director (Alison Woodman) who focuses on members – both to improve membership retention and to increase our understanding of our membership for impact evaluation purposes. So hopefully in the future our impact evaluations will be more thorough and frequent.
Jacob’s already addressed the most pressing points. I just had a couple of other considerations: As he said, the main thing in December was the Pledge event, which was amazing. Giving season and new year’s resolutions always cause a bit of a bump in membership too.
I’m don’t think that we should think of members who joined under the newer pledge as ‘lower quality’. The new pledge is more general, but it is so in the sense that it is cause neutral. People who plan to donate to animal charities because they think that they can help fellow creatures even more by doing so seem every bit as serious in helping others effectively as those donating to eradicate extreme poverty. We’ve even had a couple of people join because of the change in the pledge who donate to poverty eradication charities, but who thought that we might in future find an even more effective cause than that, and wanted to make sure if they took a lifelong pledge they would definitely be able to keep it. These people seem at least as likely to continue donating as others.
While I agree that it’s easier to join online than by sending off a letter, I’m less convinced that the new method of joining is going to be one which makes it less likely people feel engaged. We still send people a physical pledge form, to maintain the feeling of solidity. But doing it online means that an email immediately gets sent to the person, and we now make more effort to follow-up with and get to know members (which we have more capacity for because of the reduction in admin moving to the online system).
By the way, you seem very comfortable with numbers, knowledgable and engaged – how would you feel about working with our research team? We’d be very grateful for the help!
Thanks for the response.
Glad to hear it! I look forward to seeing the data.
Well, that’s true unless you don’t think animals count as ‘others’. Presumably we should discount their value by whatever credence we place in animals counting as moral agents. A wide range of new focuses are now acceptable to GWWC members, and regardless of one’s specific personal beliefs, it seems likely that some of those members will focus on something you don’t personally think is very valuable. For example, it seems plausible that some people might (falsely) think that donating to a political party was the best way of helping people. Yet presumably no-one would think that increasing donations to both Democrat and Republican parties by $100 is equal to $200 going to AMF!
However, my complaint wasn’t really about the target of their donations—I assume that in practice most will mainly donate to AMF etc. anyway—but that they’ll see a higher dropout rate. I don’t think this concern can really be verified or refuted without seeing the data.
This seems like a bit of a misnomer. People were always requesting a signup form online—you could combine paper forms in the post with an email saying ‘Welcome … the forms are on their way’.
Your point about the overhead costs is a good one though. My impression is this could be outsourced—at least in the US I’m pretty sure there are firms that will deal with your mailing for you, but that still costs money.
Very kind of you. Unfortunately I work a more-than-full-time job, so can’t really commit to anything. However, I get the impression that the EA community has a fair number of math-literate people who would be willing to work on relevant questions just out of interest if the data was available. In this case I’d probably analyse the cohort data if it was available—and the same probably goes for object-level issues as well. This would have a couple of advantages:
Save CEA resources
Get third parties more involved
Provide independent evaluation, which is more credible than internal evaluation (in the same way that companies get independent auditors, etc.)
The relevance of this isn’t clear. Both before and after the change, every particular GWWC donor is giving to a charity that the donor likes but that some people don’t consider maximally valuable. The question is whether the change made things worse. On average, I expect it made things better, since I expect cause neutrality to be helpful.
Perhaps your point is that previously this impact was valuable to one constituency—people who prioritize development—whereas now the value is spread to a wider range of donors with a wider range of values. I can see why this might make fundraising harder, since it’s a classic example of a public goods problem, but it doesn’t seem to make the project less impactful overall.
I expect that GWWC’s dropout rate is increasing over time. I don’t see why this change in particular would be expected to lead to a higher dropout rate. At face value, one might expect a broader pledge to have a lower dropout rate.
I’m implicitly assuming that global poverty is an unusually uncontroversial cause.
I guess in the limit if the pledge became sufficiently expansive it would become impossible to fail to comply!
I think locally the effect is likely to be negative, though I don’t have much confidence either way till we see the data. Nor do I see much value in speculating on this, save to encourage the release of such data.
Thanks! We actually have had quite a few people with expertise go over the impact evaluation—as you say, it’s pretty fun. I was thinking of other things we could do with more researchers on. Sounds like you’re already plenty busy though!
Here is that impact analysis. It’s well worth a read, particularly the section ‘Overview of GWWC’s performance so far’ as this speaks to many of Dale’s questions.
Your link just points to this post?
Fixed.
Thanks very much. Some interesting points:
only 293 of 705 (41.6%) of the people who applied to join GWWC have returned their pledge forms.
The fraction of people who claimed to be fulfilling the pledge in 2011 (I take it GWWC does not have tax receipts etc.) was basically unvarying by year of pledge: 64% of 2009 pledges, 71% of 2010 pledges and 64% of 2011 pledges. This suggests that the ‘more keen people join first’ and ‘people drop out over time’ effects perfectly canceled out. (Though the sample size is small). However 65-70% claimed retention after 0-2 years suggests a relatively short half-life for members.
It occurs to me that, if true, this would have important implications for GWWC’s marketing, namely that we should deemphasize students, as many of them would have dropped out before they start making bank, and increase the emphasis on mature targets.