Thank you! This was exactly the sort of thoughtful explanation I was hoping for.
For what it’s worth, in my experience at TLYCS it takes a lot more than just a website to move money. When I look at the things that seem to have driven our growth over the years, a lot of it is simply having the capacity to do basic things like communicate more with donors. And the relative steadiness in TLYCS’s multiplier (between 9x and 13x from 2016-2019) as expenses more than tripled suggests that there’s not a huge difference between the marginal multiplier and the average multiplier (and that if anything, the marginal multiplier might be higher).
I do think your “step function” argument is getting at something interesting (though I’d say you’re overestimating the availability and willingness of large donors to fund these transformative initiatives). There have definitely been discrete steps up in TLYCS’s history, most recently last year when we had a major launch of the updated book, overhauled our website, and more than doubled both money moved and expenses. The investments paid off: this year expenses will be down slightly and money moved will be up a lot, so the multiplier will break out of its recent range.
As you note, multiplier charities don’t get much scrutiny. Part of the motivation for this post is trying to figure out whether adding more scrutiny could be a good investment. After all, if additional vetting could make donors feel confident that multiplier organizations were offering legitimate 2x multiplier (let alone 10X or more), that would be a huge source of leverage for the EA community.
I’m glad to hear you found my reasoning useful, and I appreciate your explanation of where you think it may go astray. I’m a fairly marginal actor in the grand scheme of the EA community and don’t feel I am anywhere close to having a clear view on whether the returns to adding further vetting or oversight structures would outweigh the costs. Naïvely, it seems to me that some kinds of organizational transparency are pretty cheap. However, it occurs to me that even though I’ve spent a fair bit of time on the TLYCS website over the past several years and gave to your COVID-19 response fund back in the spring, I honestly have no recollection of the extent of your transparency in the status quo. In a similar vein, to put it more flippantly than you deserve, I don’t think most people I know in the community (myself included) really understand what you do. I was even unaware of how high your estimated multiplier is (if you had asked me to guess prior to your comment, there’s no way I would’ve gone higher than 4x), and now, I am quite curious about how you’re estimating that and what you think is driving such a high return. I expect this is probably my fault for not seriously investigating “multiplier charities” when deciding where to give and instead presuming that they likely aren’t a good fit for small donors like me for the reasons I explained. However, I also think I am exactly the persuadable small donor who you would want to be touching with whatever outreach or marketing you’re doing , so maybe there’s room for improvement on your part there, as well.
For what it’s worth, if you were going to invest in adding some kind of vetting or oversight structure, here are a few questions that—inspired by your comment—I would most want it to answer before making a determination about whether to give to TLYCS:
1. Why have TLYCS’s expenses tripled since 2016? Other than the website overhaul and the book launch, what have you been spending on? Are you aiming to engage in similar (financial) growth again in the near term? If not, would you be if you had more support from small donors?
2. What do you mean by “communicate with more donors?” What does that involve? How costly is it on a per-donor basis? How scalable is it?
3. When you spend more money (beyond your basic operating expenses: salaries, office space if you have it, etc.), and that spending seems to be associated with an increase in donor interest in your recommended charities, what do you think generally explains that relationship, and how do you determine that such an increase in donor interest was counterfactually caused by the increase in spending?
4. More generally, and this may be an extremely dumb question/something you have explained at length elsewhere, how do you arrive at your “money moved” estimates, and how do you ensure that they are counterfactually valid?
5. Do you personally believe that TLYCS will hit diminishing marginal returns on investments in growing its base of donors to its recommended charities sometime in the near or intermediate term?
You obviously do not have to answer these questions here or at all. I wrote them out only to provide a sense of what information I feel I am missing.
I don’t think most people I know in the community (myself included) really understand what you do.
I agree there’s a lack of understanding of our work, and hope this discussion helps clarify some things. And we haven’t done a great job of reaching out to the community to explain our work. One difficulty in operating a multiplier charity is that it can be tough to promote your own organization since your whole purpose is to promote other charities.
I was even unaware of how high your estimated multiplier is (if you had asked me to guess prior to your comment, there’s no way I would’ve gone higher than 4x).
FWIW, I think most (maybe all) multiplier organizations report multipliers well above 4x.
1. Why have TLYCS’s expenses tripled since 2016? Other than the website overhaul and the book launch, what have you been spending on? Are you aiming to engage in similar (financial) growth again in the near term? If not, would you be if you had more support from small donors?
Most of the increase was due to the book: expenses were around 300k in 2016 and 2017, ~450k in 2018, and a bit under $1m in 2019 as we ramped up for the book project. The increase in 2018 was due to adding a bit of headcount (by far our largest expense) and rationalizing some very low salaries that had been in place at the outset.
Going forward, we’d very much like to be able to grow our operational budget and would do so if we had more confidence in our ability to raise the necessary funds. Off the top of my head (definitely not an official organizational response) I’d say something like 30% annual growth would be manageable.
2. What do you mean by “communicate with more donors?” What does that involve? How costly is it on a per-donor basis? How scalable is it?
I meant this very broadly- it covers a lot of things, and the cost of those activities likely varies a lot. Over the past few years we’ve done things like: building out a CRM system to manage our donors and leads, personally emailing and/or calling more donors to thank them and build a relationship, have more one on one conversations with large donors/prospects, hold more donor/fundraising events, and add customization to our newsletter/email communications (so that, for example, donors and non-donors receive different newsletters.) The common thread is that this all involves work, and you need to pay someone to do that work.
I think there is enormous room to scale this stuff.
3. When you spend more money (beyond your basic operating expenses: salaries, office space if you have it, etc.), and that spending seems to be associated with an increase in donor interest in your recommended charities, what do you think generally explains that relationship, and how do you determine that such an increase in donor interest was counterfactually caused by the increase in spending?
Salaries account for the vast majority of our budget (meaning increased spending typically means increased headcount). We try to assess if our strategy and execution are working, and the details depend on the project. Sometimes we add expenses that don’t immediately impact donations, like hiring an accountant. We didn’t try to model out that ROI, we just knew we had grown beyond the point where it was feasible to operate with a volunteer accountant. When we overhauled the website, we were able to look at a lot of quantitative metrics (conversion rates, engagement rates, etc) and see that they improved a lot right after the change. When we launched TLYCS Australia, we didn’t have to watch the donations that came in for very long to know it was going to be a big success.
FYI our 2018 annual report has a good discussion of how we pivoted our strategy then assessed that change.
4. More generally, and this may be an extremely dumb question/something you have explained at length elsewhere, how do you arrive at your “money moved” estimates, and how do you ensure that they are counterfactually valid?
Thanks for asking- this is something most people probably don’t know.
It’s pretty simple: we count money that’s been donated to TLYCS to be regranted to our recommended charities + money donated to those charities (and reported back to us) where the donor indicates that we influenced the gift.
We’ve discussed counterfactuals in detail in the appendix to our 2017 annual report. There are definitely a lot of considerations, but I generally think counterfactual concerns are becoming less of an issue over time (TLYCS’s role in producing the new book really mitigates concerns that we’re measuring Peter Singer’s impact rather than the organization’s.)
One place we have made a counterfactual adjustment (which I think speaks to our attempts to be reasonable in our metrics) is with a specific family that has donated several million dollars over the past 5 years. We think it’s very likely those gifts would have been made without our involvement, so we’ve only counted 5% of their value in our numbers. FWIW, the charities involved told us they thought we should take full credit.
I don’t really know what TLYCS’s exact multiplier would be if you had perfect information and could account for all the counterfactuals. But I’m highly confident it’s well above the threshold of providing significant leverage. In 2020, even if our true impact is only 10% of our reported money moved figure (which I believe is conservative), we’d still provide >50% leverage. There’s a very large margin of safety (which you wouldn’t really have if you had a mental model that our multiplier was 4x or less per your comment above).
5. Do you personally believe that TLYCS will hit diminishing marginal returns on investments in growing its base of donors to its recommended charities sometime in the near or intermediate term?
Personally, I think TLYCS is just getting started. The new book is a powerful asset, and by getting free copies (and excerpts, video summaries, etc) in many people’s hands, I’m confident that our money moved will grow significantly over the long run. We know the first book influenced a ton of people (including Cari Tuna), now we have a book that will have a much wider reach and has an organization behind it.
I know our multiplier will go up in 2020, but after that I’m not really sure. We focus more on “Net Impact”, which is our Money Moved minus our expenses, rather than our multiplier which takes the ratio of those numbers.
I think Peter Hurford originally suggested the Net Impact metric back in the day, and it makes a lot of sense to us. We’d much rather spend $1 billion to move $5 billion than spend $1,000 to move $10,000. So potentially there could be diminishing marginal returns (i.e. a falling multiplier), but I don’t think that’s necessarily a problem if you’re trying to build an organization that does as much good as possible instead of one that’s as efficient as possible.
For what it’s worth – Giving What We Can also noticed a bump in pledges that came from The Life You Can Save book relaunch (and people specifying that is how they found out). There’s often spill over like this that isn’t directly tracked by the organisation doing the multiplying.
Thanks for this data point Luke! It’s a good reminder that counterfactuals work both ways for multiplier orgs. Sometimes we count money that would have been donated counterfactually (overestimating our impact), but sometimes there are donations we don’t count that wouldn’t have happened if we didn’t exist (underestimating our impact).
Also worth noting that sometimes the spillover effect is in an area that isn’t the multiplier orgs main focus. For instance, I’d also expect the book relaunch to help 80K which gets a nice discussion, but that’s not anything TLYCS will capture in its metrics.
Thank you! This was exactly the sort of thoughtful explanation I was hoping for.
For what it’s worth, in my experience at TLYCS it takes a lot more than just a website to move money. When I look at the things that seem to have driven our growth over the years, a lot of it is simply having the capacity to do basic things like communicate more with donors. And the relative steadiness in TLYCS’s multiplier (between 9x and 13x from 2016-2019) as expenses more than tripled suggests that there’s not a huge difference between the marginal multiplier and the average multiplier (and that if anything, the marginal multiplier might be higher).
I do think your “step function” argument is getting at something interesting (though I’d say you’re overestimating the availability and willingness of large donors to fund these transformative initiatives). There have definitely been discrete steps up in TLYCS’s history, most recently last year when we had a major launch of the updated book, overhauled our website, and more than doubled both money moved and expenses. The investments paid off: this year expenses will be down slightly and money moved will be up a lot, so the multiplier will break out of its recent range.
As you note, multiplier charities don’t get much scrutiny. Part of the motivation for this post is trying to figure out whether adding more scrutiny could be a good investment. After all, if additional vetting could make donors feel confident that multiplier organizations were offering legitimate 2x multiplier (let alone 10X or more), that would be a huge source of leverage for the EA community.
+1 to this. RC Forward wrote a bit about this in our year-in-review in 2019.
I’m glad to hear you found my reasoning useful, and I appreciate your explanation of where you think it may go astray. I’m a fairly marginal actor in the grand scheme of the EA community and don’t feel I am anywhere close to having a clear view on whether the returns to adding further vetting or oversight structures would outweigh the costs. Naïvely, it seems to me that some kinds of organizational transparency are pretty cheap. However, it occurs to me that even though I’ve spent a fair bit of time on the TLYCS website over the past several years and gave to your COVID-19 response fund back in the spring, I honestly have no recollection of the extent of your transparency in the status quo. In a similar vein, to put it more flippantly than you deserve, I don’t think most people I know in the community (myself included) really understand what you do. I was even unaware of how high your estimated multiplier is (if you had asked me to guess prior to your comment, there’s no way I would’ve gone higher than 4x), and now, I am quite curious about how you’re estimating that and what you think is driving such a high return. I expect this is probably my fault for not seriously investigating “multiplier charities” when deciding where to give and instead presuming that they likely aren’t a good fit for small donors like me for the reasons I explained. However, I also think I am exactly the persuadable small donor who you would want to be touching with whatever outreach or marketing you’re doing , so maybe there’s room for improvement on your part there, as well.
For what it’s worth, if you were going to invest in adding some kind of vetting or oversight structure, here are a few questions that—inspired by your comment—I would most want it to answer before making a determination about whether to give to TLYCS:
1. Why have TLYCS’s expenses tripled since 2016? Other than the website overhaul and the book launch, what have you been spending on? Are you aiming to engage in similar (financial) growth again in the near term? If not, would you be if you had more support from small donors?
2. What do you mean by “communicate with more donors?” What does that involve? How costly is it on a per-donor basis? How scalable is it?
3. When you spend more money (beyond your basic operating expenses: salaries, office space if you have it, etc.), and that spending seems to be associated with an increase in donor interest in your recommended charities, what do you think generally explains that relationship, and how do you determine that such an increase in donor interest was counterfactually caused by the increase in spending?
4. More generally, and this may be an extremely dumb question/something you have explained at length elsewhere, how do you arrive at your “money moved” estimates, and how do you ensure that they are counterfactually valid?
5. Do you personally believe that TLYCS will hit diminishing marginal returns on investments in growing its base of donors to its recommended charities sometime in the near or intermediate term?
You obviously do not have to answer these questions here or at all. I wrote them out only to provide a sense of what information I feel I am missing.
I agree there’s a lack of understanding of our work, and hope this discussion helps clarify some things. And we haven’t done a great job of reaching out to the community to explain our work. One difficulty in operating a multiplier charity is that it can be tough to promote your own organization since your whole purpose is to promote other charities.
FWIW, I think most (maybe all) multiplier organizations report multipliers well above 4x.
Most of the increase was due to the book: expenses were around 300k in 2016 and 2017, ~450k in 2018, and a bit under $1m in 2019 as we ramped up for the book project. The increase in 2018 was due to adding a bit of headcount (by far our largest expense) and rationalizing some very low salaries that had been in place at the outset.
Going forward, we’d very much like to be able to grow our operational budget and would do so if we had more confidence in our ability to raise the necessary funds. Off the top of my head (definitely not an official organizational response) I’d say something like 30% annual growth would be manageable.
I meant this very broadly- it covers a lot of things, and the cost of those activities likely varies a lot. Over the past few years we’ve done things like: building out a CRM system to manage our donors and leads, personally emailing and/or calling more donors to thank them and build a relationship, have more one on one conversations with large donors/prospects, hold more donor/fundraising events, and add customization to our newsletter/email communications (so that, for example, donors and non-donors receive different newsletters.) The common thread is that this all involves work, and you need to pay someone to do that work.
I think there is enormous room to scale this stuff.
Salaries account for the vast majority of our budget (meaning increased spending typically means increased headcount). We try to assess if our strategy and execution are working, and the details depend on the project. Sometimes we add expenses that don’t immediately impact donations, like hiring an accountant. We didn’t try to model out that ROI, we just knew we had grown beyond the point where it was feasible to operate with a volunteer accountant. When we overhauled the website, we were able to look at a lot of quantitative metrics (conversion rates, engagement rates, etc) and see that they improved a lot right after the change. When we launched TLYCS Australia, we didn’t have to watch the donations that came in for very long to know it was going to be a big success.
FYI our 2018 annual report has a good discussion of how we pivoted our strategy then assessed that change.
Thanks for asking- this is something most people probably don’t know.
It’s pretty simple: we count money that’s been donated to TLYCS to be regranted to our recommended charities + money donated to those charities (and reported back to us) where the donor indicates that we influenced the gift.
We’ve discussed counterfactuals in detail in the appendix to our 2017 annual report. There are definitely a lot of considerations, but I generally think counterfactual concerns are becoming less of an issue over time (TLYCS’s role in producing the new book really mitigates concerns that we’re measuring Peter Singer’s impact rather than the organization’s.)
One place we have made a counterfactual adjustment (which I think speaks to our attempts to be reasonable in our metrics) is with a specific family that has donated several million dollars over the past 5 years. We think it’s very likely those gifts would have been made without our involvement, so we’ve only counted 5% of their value in our numbers. FWIW, the charities involved told us they thought we should take full credit.
I don’t really know what TLYCS’s exact multiplier would be if you had perfect information and could account for all the counterfactuals. But I’m highly confident it’s well above the threshold of providing significant leverage. In 2020, even if our true impact is only 10% of our reported money moved figure (which I believe is conservative), we’d still provide >50% leverage. There’s a very large margin of safety (which you wouldn’t really have if you had a mental model that our multiplier was 4x or less per your comment above).
Personally, I think TLYCS is just getting started. The new book is a powerful asset, and by getting free copies (and excerpts, video summaries, etc) in many people’s hands, I’m confident that our money moved will grow significantly over the long run. We know the first book influenced a ton of people (including Cari Tuna), now we have a book that will have a much wider reach and has an organization behind it.
I know our multiplier will go up in 2020, but after that I’m not really sure. We focus more on “Net Impact”, which is our Money Moved minus our expenses, rather than our multiplier which takes the ratio of those numbers.
I think Peter Hurford originally suggested the Net Impact metric back in the day, and it makes a lot of sense to us. We’d much rather spend $1 billion to move $5 billion than spend $1,000 to move $10,000. So potentially there could be diminishing marginal returns (i.e. a falling multiplier), but I don’t think that’s necessarily a problem if you’re trying to build an organization that does as much good as possible instead of one that’s as efficient as possible.
For what it’s worth – Giving What We Can also noticed a bump in pledges that came from The Life You Can Save book relaunch (and people specifying that is how they found out). There’s often spill over like this that isn’t directly tracked by the organisation doing the multiplying.
Thanks for this data point Luke! It’s a good reminder that counterfactuals work both ways for multiplier orgs. Sometimes we count money that would have been donated counterfactually (overestimating our impact), but sometimes there are donations we don’t count that wouldn’t have happened if we didn’t exist (underestimating our impact).
Also worth noting that sometimes the spillover effect is in an area that isn’t the multiplier orgs main focus. For instance, I’d also expect the book relaunch to help 80K which gets a nice discussion, but that’s not anything TLYCS will capture in its metrics.
This is all fantastic information to have — thank you so much for explaining it! I’m really glad to have improved my understanding of this.
Glad it helped! Thanks for the great questions, I’m sure you’re not the only one who had them!