I would actually expect our marginal multiplier to be much closer to our average multiplier than the CEARCH method implies.
What is your best guess for the marginal multiplier of donating to GWWC now? I think it would be great if you estimated GWWCās marginal multiplier in your next impact evaluation. Do you plan to publish this in 2026, looking into your impact in 2025? I guess data until the end of 2025 would be enough for you to have a better idea about your marginal multiplier.
I donāt think I agree that the information on the website is misleading seeing as it just states the number of people who have taken the pledge.
Assuming the median person who checks your website, and reads the sentence I quoted above[1] believes something like 90 % of pledgers are fulfilling their pledges, but that in reality only 1ā3 are actually fulfilling it based on recorded and non-recorded donations, would you agree it would be better to update the website such that people do not think the size of the active community is much larger than it actually is? If you do agree, which fraction of pledgers are fulfilling their pledge based on past recorded and non-recorded donations? I understand there is uncertainty, but your estimate that only 59.4 % of 10 % Pledgers record a donation in the 1st year of their pledge makes me think the fraction of people fulfilling their pledges could me much lower than what people checking your website think. I think you should at least update the sentence I quoted above from your website to something like the following. ā9,840 people have pledged to donate at least 10% of their net incomeā. I suspect many will infer from the words ācommunityā and āpledgingā, which GWWC is currently using, that the vast majority of pledgers are fulfilling their pledge.
The regression you suggest is something we have considered, but donāt think it is an obvious improvement over our approach of taking the mean over the most recent pledge years. While there might be an effect of the year the pledge started on average first-year pledge donations, we do not think this trend is linear.
I still think assuming a (potentially small) linear effect of the year the pledge started on donations is better than supposing no effect at all. The effect may be non-linear, but I see this as a reason for running a non-linear regression (e.g. D(s, y) = a(s) + b(s)*y + c(s)*y^2), not for supposing an effect of 0.
For instance, the 2021 cohort had the second highest average first-year donations across all cohorts and the five cohorts with the lowest average first-year donations were 2010, 2017, 2018, 2016 and 2012.
Note the year the pledge started can affect donations in years of the pledge besides the 1st.
We currently arenāt considering retiring the š¹Trial Pledge. While in terms of direct donation value the š¹Trial Pledge contributes a relatively small fraction of our pledge impact, we believe the main value add of the š¹Trial Pledge comes from š¹Trial pledgers āupgradingā to šø10% Pledges.
Have you estimated the fraction of the impact you attributed to The 10 % Pledge which was caused by The Trial Pledge?
For example, roughly 10% of those who have taken a š¹Trial Pledge are now šø10% Pledges
Note many of these pledgers might have taken The 10 % Pledge anyway, just slightly later than they would have without The Trial Pledge.
Our community includes 9,840 lifetime members pledging ā„10% of their income, plus 1,117 trial pledgers, together making up our 10,957 strong giving community
Hi Vasco, thanks for your response! Sorry for my delay in getting back to you, I have just got back from leave. I have tried to leave responses to your main points below, but if I have missed anything please let me know.
Re: our marginal multiplier: This is not something we have explicitly tried to model. The most relevant information I can provide here is that our bar for undertaking new pledge acquisition activities is that they must at least exceed 5x in expectation, but this is still not the same as our marginal multiplier for a number of reasons. While we hope to publish an estimate of our average multiplier for 2025 in early 2026, I currently donāt expect to try to explicitly estimate our marginal multiplier. One key reason for this is that, unless we have identified a very scalable method for growing pledges, our marginal multiplier estimate would change quite quickly as we receive more funding and so may only be relevant for a brief period.
Re: how we report the number of pledgers on the website: Thanks for sharing your thoughts here! I continue to think that the statement on our website is accurate and that it isnāt misleading to use the terms ācommunityā and āpledgingā here. Simply, these are the numbers of people in our community who have taken pledges with GWWC. I donāt believe the text makes a claim about the number of pledgers who are reporting their donations (which is not a requirement of the pledge) or the number who are fulfilling their pledge (which we donāt have a reliable estimate of). It isnāt clear to me why we should think that the median person who reads the statement would assume that 90% of pledgers are donating.
Re: modelling pledge value for different cohorts: This kind of regression modelling will be something we continue to consider implementing in future evaluations, but currently it isnāt clear enough to me that these models will be better predictors of future cohort pledge donations than the āaverage of recent yearsā method we currently use. The trends to date have simply been too noisy for me to feel confident in any given mathematical model. I also think these models involve some tradeoffs in terms of time investment and legibility and that we also need to factor in these considerations when selecting our approach.
Re: Trial Pledges: We have not estimated the fraction of impact we attribute to the šø10% Pledge that was caused by the š¹Trial Pledge, but I would roughly guess for recent cohorts it is somewhere in the vicinity of 5ā20%. It is difficult to come up with a precise estimate because we donāt know how causally responsible the š¹Trial Pledge is for the šø10% Pledge in these cases (as you note).
While we hope to publish an estimate of our average multiplier for 2025 in early 2026
Great to know!
I currently donāt expect to try to explicitly estimate our marginal multiplier. One key reason for this is that, unless we have identified a very scalable method for growing pledges, our marginal multiplier estimate would change quite quickly as we receive more funding and so may only be relevant for a brief period.
I think estimating in early 2026 GWWCās marginal multiplier in 2025 would still be useful. A value higher than 1 would suggest GWWC should have received more funding this year.
The marginal multiplier should ideally not vary much across years. GWWC should move funds from the years with the lowest marginal multiplier to the years with the highest marginal multiplier until the marginal multiplier is the same across all years. If you estimated your marginal multiplier annually, you could realise you should have spent more/āless if you marginal multiplier decreased/āincreased.
Thanks for the feedback, Aidan! Strongly upvoted.
What is your best guess for the marginal multiplier of donating to GWWC now? I think it would be great if you estimated GWWCās marginal multiplier in your next impact evaluation. Do you plan to publish this in 2026, looking into your impact in 2025? I guess data until the end of 2025 would be enough for you to have a better idea about your marginal multiplier.
Assuming the median person who checks your website, and reads the sentence I quoted above[1] believes something like 90 % of pledgers are fulfilling their pledges, but that in reality only 1ā3 are actually fulfilling it based on recorded and non-recorded donations, would you agree it would be better to update the website such that people do not think the size of the active community is much larger than it actually is? If you do agree, which fraction of pledgers are fulfilling their pledge based on past recorded and non-recorded donations? I understand there is uncertainty, but your estimate that only 59.4 % of 10 % Pledgers record a donation in the 1st year of their pledge makes me think the fraction of people fulfilling their pledges could me much lower than what people checking your website think. I think you should at least update the sentence I quoted above from your website to something like the following. ā9,840 people have pledged to donate at least 10% of their net incomeā. I suspect many will infer from the words ācommunityā and āpledgingā, which GWWC is currently using, that the vast majority of pledgers are fulfilling their pledge.
I still think assuming a (potentially small) linear effect of the year the pledge started on donations is better than supposing no effect at all. The effect may be non-linear, but I see this as a reason for running a non-linear regression (e.g. D(s, y) = a(s) + b(s)*y + c(s)*y^2), not for supposing an effect of 0.
Note the year the pledge started can affect donations in years of the pledge besides the 1st.
Have you estimated the fraction of the impact you attributed to The 10 % Pledge which was caused by The Trial Pledge?
Note many of these pledgers might have taken The 10 % Pledge anyway, just slightly later than they would have without The Trial Pledge.
Hi Vasco, thanks for your response! Sorry for my delay in getting back to you, I have just got back from leave. I have tried to leave responses to your main points below, but if I have missed anything please let me know.
Re: our marginal multiplier: This is not something we have explicitly tried to model. The most relevant information I can provide here is that our bar for undertaking new pledge acquisition activities is that they must at least exceed 5x in expectation, but this is still not the same as our marginal multiplier for a number of reasons. While we hope to publish an estimate of our average multiplier for 2025 in early 2026, I currently donāt expect to try to explicitly estimate our marginal multiplier. One key reason for this is that, unless we have identified a very scalable method for growing pledges, our marginal multiplier estimate would change quite quickly as we receive more funding and so may only be relevant for a brief period.
Re: how we report the number of pledgers on the website: Thanks for sharing your thoughts here! I continue to think that the statement on our website is accurate and that it isnāt misleading to use the terms ācommunityā and āpledgingā here. Simply, these are the numbers of people in our community who have taken pledges with GWWC. I donāt believe the text makes a claim about the number of pledgers who are reporting their donations (which is not a requirement of the pledge) or the number who are fulfilling their pledge (which we donāt have a reliable estimate of). It isnāt clear to me why we should think that the median person who reads the statement would assume that 90% of pledgers are donating.
Re: modelling pledge value for different cohorts: This kind of regression modelling will be something we continue to consider implementing in future evaluations, but currently it isnāt clear enough to me that these models will be better predictors of future cohort pledge donations than the āaverage of recent yearsā method we currently use. The trends to date have simply been too noisy for me to feel confident in any given mathematical model. I also think these models involve some tradeoffs in terms of time investment and legibility and that we also need to factor in these considerations when selecting our approach.
Re: Trial Pledges: We have not estimated the fraction of impact we attribute to the šø10% Pledge that was caused by the š¹Trial Pledge, but I would roughly guess for recent cohorts it is somewhere in the vicinity of 5ā20%. It is difficult to come up with a precise estimate because we donāt know how causally responsible the š¹Trial Pledge is for the šø10% Pledge in these cases (as you note).
Thanks, Aidan!
Great to know!
I think estimating in early 2026 GWWCās marginal multiplier in 2025 would still be useful. A value higher than 1 would suggest GWWC should have received more funding this year.
The marginal multiplier should ideally not vary much across years. GWWC should move funds from the years with the lowest marginal multiplier to the years with the highest marginal multiplier until the marginal multiplier is the same across all years. If you estimated your marginal multiplier annually, you could realise you should have spent more/āless if you marginal multiplier decreased/āincreased.