Let’s say only one other person in your network hears that you took the pledge and is inspired to do the same. That would be doubling your impact. If two people in your network were inspired to pledge based on your decision, that would be tripling your impact
Thanks Elizabeth, Owen, Brad and Kirsten for the constructive feedback! I agree that the post in its current form doesn’t live up to the epistemic standards we aspire to at GWWC, for many of the reasons you mention. My apologies for this.
The research team usually reviews any public impact claims we make, but this post slipped through the cracks of our review process in part because the main topic (i.e. “misconceptions about the pledge”) didn’t immediately seem research-related. We’ll tighten up the process to prevent this from happening in the future. I’ve now also reviewed and we’ll edit the post on the points you mentioned + a few others. We’ll post here once that’s done.
I didn’t find this paragraph to be off or particularly misleading fwiw.
It is roughly true (minus what they would have donated otherwise) when thinking in terms of counterfactual impact, and assuming you are an average pledger and would be inspiring other average pledgers (no expected difference in income or attrition, or effectiveness of charities donated to).
I think the caveats are sufficiently obvious that the reader could be expected to understand them on their own. For instance if you convince someone to donate $1000 it seems obvious that they should get most of the credit, but it still might be true that you were counterfactual in their decision.
Agree. Rejecting the argument because it double counts feels a bit like the “shouldn’t you be attributing some/most of the impact to the people that did the direct work” objection to estimates of how many lives your money can save.
I think everyone knows it’s a counterfactual claim contingent on you actually being a major influence on that person and not karmic accounting that assigns yourself all the credit
That said, it’s still a dubious approximation to say it ‘doubles’ your impact as it really depends on relative donation sizes of the people you persuade. That means it can be much more than double if you earn less than the people that listen to you, of course!
Hi Elizabeth—it would be great if you could explain why?
If you encourage a friend who wasn’t otherwise giving effectively or significantly to pledge—and they do—this seems like a potential doubling of donations going to high impact charities. Maybe more or less if their salary is higher or lower but as a rule of thumb I think it seems like generally a fine thing to say.
Maybe you were thinking that donations might be less counterfactual if these people are already in the effective altruism community(?), but many people GWWC reaches are outside the community who might be less likely to be giving effectively or significantly and we try to tailor our content for the broadest audience we reach, especially our blog content. I’ll get Alana to update that this is a link post of our blog post: https://www.givingwhatwecan.org/en/blog/5-things-you-ve-got-wrong-about-the-giving-what-we-can-pledge
Assigns 100% of their future impact to you, not counting their own contribution and the other sources that caused this change. It’s the same kind of simplification as “every blood donation saves 3 lives”, when what they mean is “your blood will probably go to three people, each of whom will receive donations from many people.”
Assumes perfect follow up. This isn’t realistic for a median pledger, but we might expect people who were tipped into pledging by a single act by a single person to have worse follow-up than people who find it on their own. You could argue it isn’t actually one action, there were lots of causes and that makes it stickier, but then you run into #1 even harder.
Reifies signing the pledge as the moment everything changes, while vibing that this is a small deal you can stop when you feel like it.
Assumes every pledger you recruit makes exactly the same amount. Part of me thinks this is a nit pick. You could assume people recruit people who on average earn similar salaries, or think it’s just not worth doing the math on likely income of secondary recruitment. Another part thinks it’s downstream of the same root cause as the other issues, and any real fix to those will fix this as well.
The word “effective” is doing a lot of work. What if they have different tastes than I do? What if they think PlayPumps are a great idea? .
Treating the counterfactual as 0.
As I write this out I’m realizing my objection isn’t just the bad math. It’s closer to treating pledge-takiers as the unit of measurement, with all pledges or at least all dollars donated being interchangeable. People who are recruited/inspired by a single person are likely to have different follow through and charitable targets than people inspired by many people over time, who are different than people driven to do this themselves. ?
I don’t think it does assume perfect follow-up, it just assumes roughly the same follow-up from them as you. I hear you that maybe people you tip into taking the pledge are systematically different in a way that makes you doubt that as well, but I’m not actually convinced this difference is that substantial.
Similarly, I don’t think different amounts of income feels like a big problem with this sentiment to me, as long as their income isn’t systematically less (or more!) than yours. It feels like an imprecision, but if it’s true on average it’s not one I particularly resent.
(I think the rest of your points seem fine so overall I still agree with your bottom line.)
My model is that at least one of the following must be true: you’re one factor among many that caused the change, the change is not actually that big, or attrition will be much higher than standard pledge takers.
Which is fine. Accepting the framing around influencing others[1]: you will be one of many factors, but your influence will extend past one person. But I think it’s good to acknowledge the complexity.
I separately question whether the pledge is the best way to achieve this goal. Why lock in a decision for your entire life instead of, say, taking a lesson in how to talk about your donations in ways that make people feel energized instead of judged?
I can’t speak for Elizabeth, but I also find that that paragraph feels off, for reasons something like:
Conflation of “counterfactual money to high-impact charities” with “your impact”
Maybe even if it’s counterfactually moved, you don’t get to count all the impact from it as your impact, since to avoid double-counting and impact ponzi schemes it’s maybe important to take a “share-of-the-pie” approach to thinking about your impact (here’s my take on that general question), and presumably they get a lot of the credit for their giving
Plus, maybe you do things which are importantly valuable that aren’t about your pledge! It’s at least a plausible reading (though it’s ambiguous) that “double your impact” would be taken as “double your lifetime impact”
As well as sharing credit for their donations with them, you maybe need to share credit for having nudged them to make the pledge with other folks (including but not limited to GWWC)
As you say, their donations may not be counterfactual even in the short-term
Even if a good fraction of them are maybe from outside the community, that’s still a fraction by which it reduces expected impact
Although on average I think it’s likely very good, I’m sure in some cases the EA push towards a few charities that have been verified as highly effective actually does harm by pulling people to give to those over some other charities which were in fact even more effective (but illegibly so)
Man, long-term counterfactuals are hard
Maybe GWWC/EA ends up growing a lot further, so that it reaches effective saturation among ~all relevant audiences
In that world, if someone was open to taking the GWWC pledge, they’d likely do it eventually, even if they are currently not at all connected to the community
Now, none of these points are blatant errors, or make me want to say “what were you thinking?!?”. But I feel taken together the picture is that in fact there’s a lot of complexity to the question of how impact should be counted in that case, and the text doesn’t help the reader to understand that there’s a lot of complexity or how to navigate thinking about it, but instead cheerfully presents the most favourable possible interpretation. It just has a bit of a vibe of slightly-underhand sales tactics, or something?
Thanks Owen—I think those things are all reasonable—we might look to update this paragraph on the blog and update our messaging around this! It’s always a challenge to write in a way that’s engaging and legible to those outside the community, and also reflects all of the nuance expected in the community. We of course, always try to do our best, but sometimes we might miss the mark and we’re always open to changing our minds!
Pledging may have some combination the effect of (a) actually increasing people’s lifetime donations to effective charities and (b) causing people to advertise giving they already were going to do. To the extent that a pledge is b rather than a, getting someone to pledge the same amount as you is not double your impact.
Many of the people who you cause to become pledgers might have become pledgers later, thus you probably just accelerated their pledge, greatly decreasing your actual impact vs if you cause someone to pledge (and this pledge causes them to donate more rather than encompasses donation that would otherwise happen).
There’s a possibility that you could anchor someone to donate less. Potentially someone could see your celebrated 10% pledge and view that as adequate, lowering their donations. Here, there is a risk of harm from the pledge.
All that said, I still think the pledge is an awesome way to promote and normalize effective giving.
Thanks Brad—I think all of those are reasonable considerations! As mentioned in my response to Owen—we’ll review this messaging based on this feedback! Thanks for sharing your reasoning!
I also really disliked this section. “Let’s say only one other person in your network hears that you took the pledge and is inspired to do the same.”
I don’t care if other people take the pledge! I only care if other people give, and give effectively.
If they’d be influenced to “take the pledge” because me taking the pledge, why wouldn’t they be influenced donate a proportion of their income effectively by seeing me donate?
You can make the argument for why the pledge is more effective than just donating, but you haven’t done it here.
This math seems off on several levels.
Thanks Elizabeth, Owen, Brad and Kirsten for the constructive feedback! I agree that the post in its current form doesn’t live up to the epistemic standards we aspire to at GWWC, for many of the reasons you mention. My apologies for this.
The research team usually reviews any public impact claims we make, but this post slipped through the cracks of our review process in part because the main topic (i.e. “misconceptions about the pledge”) didn’t immediately seem research-related. We’ll tighten up the process to prevent this from happening in the future. I’ve now also reviewed and we’ll edit the post on the points you mentioned + a few others. We’ll post here once that’s done.
Thanks again for holding us accountable!
I didn’t find this paragraph to be off or particularly misleading fwiw.
It is roughly true (minus what they would have donated otherwise) when thinking in terms of counterfactual impact, and assuming you are an average pledger and would be inspiring other average pledgers (no expected difference in income or attrition, or effectiveness of charities donated to).
I think the caveats are sufficiently obvious that the reader could be expected to understand them on their own. For instance if you convince someone to donate $1000 it seems obvious that they should get most of the credit, but it still might be true that you were counterfactual in their decision.
Agree. Rejecting the argument because it double counts feels a bit like the “shouldn’t you be attributing some/most of the impact to the people that did the direct work” objection to estimates of how many lives your money can save.
I think everyone knows it’s a counterfactual claim contingent on you actually being a major influence on that person and not karmic accounting that assigns yourself all the credit
That said, it’s still a dubious approximation to say it ‘doubles’ your impact as it really depends on relative donation sizes of the people you persuade. That means it can be much more than double if you earn less than the people that listen to you, of course!
Hi Elizabeth—it would be great if you could explain why?
If you encourage a friend who wasn’t otherwise giving effectively or significantly to pledge—and they do—this seems like a potential doubling of donations going to high impact charities. Maybe more or less if their salary is higher or lower but as a rule of thumb I think it seems like generally a fine thing to say.
Maybe you were thinking that donations might be less counterfactual if these people are already in the effective altruism community(?), but many people GWWC reaches are outside the community who might be less likely to be giving effectively or significantly and we try to tailor our content for the broadest audience we reach, especially our blog content. I’ll get Alana to update that this is a link post of our blog post: https://www.givingwhatwecan.org/en/blog/5-things-you-ve-got-wrong-about-the-giving-what-we-can-pledge
Assigns 100% of their future impact to you, not counting their own contribution and the other sources that caused this change. It’s the same kind of simplification as “every blood donation saves 3 lives”, when what they mean is “your blood will probably go to three people, each of whom will receive donations from many people.”
Assumes perfect follow up. This isn’t realistic for a median pledger, but we might expect people who were tipped into pledging by a single act by a single person to have worse follow-up than people who find it on their own. You could argue it isn’t actually one action, there were lots of causes and that makes it stickier, but then you run into #1 even harder.
Reifies signing the pledge as the moment everything changes, while vibing that this is a small deal you can stop when you feel like it.
Assumes every pledger you recruit makes exactly the same amount. Part of me thinks this is a nit pick. You could assume people recruit people who on average earn similar salaries, or think it’s just not worth doing the math on likely income of secondary recruitment. Another part thinks it’s downstream of the same root cause as the other issues, and any real fix to those will fix this as well.
The word “effective” is doing a lot of work. What if they have different tastes than I do? What if they think PlayPumps are a great idea? .
Treating the counterfactual as 0.
As I write this out I’m realizing my objection isn’t just the bad math. It’s closer to treating pledge-takiers as the unit of measurement, with all pledges or at least all dollars donated being interchangeable. People who are recruited/inspired by a single person are likely to have different follow through and charitable targets than people inspired by many people over time, who are different than people driven to do this themselves. ?
I don’t think it does assume perfect follow-up, it just assumes roughly the same follow-up from them as you. I hear you that maybe people you tip into taking the pledge are systematically different in a way that makes you doubt that as well, but I’m not actually convinced this difference is that substantial.
Similarly, I don’t think different amounts of income feels like a big problem with this sentiment to me, as long as their income isn’t systematically less (or more!) than yours. It feels like an imprecision, but if it’s true on average it’s not one I particularly resent.
(I think the rest of your points seem fine so overall I still agree with your bottom line.)
My model is that at least one of the following must be true: you’re one factor among many that caused the change, the change is not actually that big, or attrition will be much higher than standard pledge takers.
Which is fine. Accepting the framing around influencing others[1]: you will be one of many factors, but your influence will extend past one person. But I think it’s good to acknowledge the complexity.
I separately question whether the pledge is the best way to achieve this goal. Why lock in a decision for your entire life instead of, say, taking a lesson in how to talk about your donations in ways that make people feel energized instead of judged?
I can’t speak for Elizabeth, but I also find that that paragraph feels off, for reasons something like:
Conflation of “counterfactual money to high-impact charities” with “your impact”
Maybe even if it’s counterfactually moved, you don’t get to count all the impact from it as your impact, since to avoid double-counting and impact ponzi schemes it’s maybe important to take a “share-of-the-pie” approach to thinking about your impact (here’s my take on that general question), and presumably they get a lot of the credit for their giving
Plus, maybe you do things which are importantly valuable that aren’t about your pledge! It’s at least a plausible reading (though it’s ambiguous) that “double your impact” would be taken as “double your lifetime impact”
As well as sharing credit for their donations with them, you maybe need to share credit for having nudged them to make the pledge with other folks (including but not limited to GWWC)
As you say, their donations may not be counterfactual even in the short-term
Even if a good fraction of them are maybe from outside the community, that’s still a fraction by which it reduces expected impact
Although on average I think it’s likely very good, I’m sure in some cases the EA push towards a few charities that have been verified as highly effective actually does harm by pulling people to give to those over some other charities which were in fact even more effective (but illegibly so)
Man, long-term counterfactuals are hard
Maybe GWWC/EA ends up growing a lot further, so that it reaches effective saturation among ~all relevant audiences
In that world, if someone was open to taking the GWWC pledge, they’d likely do it eventually, even if they are currently not at all connected to the community
Now, none of these points are blatant errors, or make me want to say “what were you thinking?!?”. But I feel taken together the picture is that in fact there’s a lot of complexity to the question of how impact should be counted in that case, and the text doesn’t help the reader to understand that there’s a lot of complexity or how to navigate thinking about it, but instead cheerfully presents the most favourable possible interpretation. It just has a bit of a vibe of slightly-underhand sales tactics, or something?
Thanks Owen—I think those things are all reasonable—we might look to update this paragraph on the blog and update our messaging around this! It’s always a challenge to write in a way that’s engaging and legible to those outside the community, and also reflects all of the nuance expected in the community. We of course, always try to do our best, but sometimes we might miss the mark and we’re always open to changing our minds!
I could imagine a few things:
Pledging may have some combination the effect of (a) actually increasing people’s lifetime donations to effective charities and (b) causing people to advertise giving they already were going to do. To the extent that a pledge is b rather than a, getting someone to pledge the same amount as you is not double your impact.
Many of the people who you cause to become pledgers might have become pledgers later, thus you probably just accelerated their pledge, greatly decreasing your actual impact vs if you cause someone to pledge (and this pledge causes them to donate more rather than encompasses donation that would otherwise happen).
There’s a possibility that you could anchor someone to donate less. Potentially someone could see your celebrated 10% pledge and view that as adequate, lowering their donations. Here, there is a risk of harm from the pledge.
All that said, I still think the pledge is an awesome way to promote and normalize effective giving.
Thanks Brad—I think all of those are reasonable considerations! As mentioned in my response to Owen—we’ll review this messaging based on this feedback! Thanks for sharing your reasoning!
I also really disliked this section. “Let’s say only one other person in your network hears that you took the pledge and is inspired to do the same.”
I don’t care if other people take the pledge! I only care if other people give, and give effectively.
If they’d be influenced to “take the pledge” because me taking the pledge, why wouldn’t they be influenced donate a proportion of their income effectively by seeing me donate?
You can make the argument for why the pledge is more effective than just donating, but you haven’t done it here.