This article makes several statements that I interpret as somewhere between intentionally misleading and outright incorrect.
For instance, this reply says the 2.19% Fed Funds rate was the prevailing rate in 2019, and isn’t intended to be a forecast of the future. But since we know the Fed Funds rate is 0% today, why would that rate be used to argue for a forward-looking change in behavior?
Similarly, the 2019 return of a short-term bond fund gives literally no information about what returns a similar fund would offer today. Indeed the beauty of the bond market is that it does offer guaranteed future returns if you hold a bond to maturity. As of today, US government bonds (which carry no risk of default) offer a guaranteed annualized return of:
0.08% if held for 3 months,
0.10% if held for 12 months,
0.13% if held for 2 years,
0.94% if held for 10 years
So assuming an organization needed to make a grant within 12 moths, they could currently expect to make 0.10% on their cash balance. Needless to say this is 1/10th of 1%, or <1/20th of the figure cited in the post. And it seems a stretch to assume organizations hold cash for 12 months before granting it. So the likely benefit is perhaps at best 1/10th and potentially <1/100th of what’s cited here.
Granted, this is not necessarily obvious stuff. However it is and should be easily understood by anyone holding themselves out to be giving financial advice to other people or institutions.
This article makes several statements that I interpret as somewhere between intentionally misleading and outright incorrect.
Could you list out those statements? Are you talking about the article, or my comments? I would certainly not want to intentionally mislead, nor be outright incorrect.
For instance, this reply says the 2.19% Fed Funds rate was the prevailing rate in 2019, and isn’t intended to be a forecast of the future. But since we know the Fed Funds rate is 0% today, why would that rate be used to argue for a forward-looking change in behavior?
As I state in the introduction, “In this section, we will introduce our cash interest opportunity cost estimation methodology for U.S. 501(c)(3) charities.” The majority of this article is about open sourcing the estimation methodology we developed, which took considerable effort over multiple years. We analyzed financial data across over 300,000 nonprofits and ran the methodology past the IRS and an independent accountant. This methodology was not designed to forecast future opportunity costs, and doing so seemed especially unnecessary in the pre-COVID-19 environment of rising interest rates that we originally developed the methodology in.
As you can see by the dollar amount of the $4 million dollar forward-looking estimate for GiveWell (based on a 1% interest rate), the interest rate does not need to be 2% for this recommendation to make sense. Current bank yields are 0.5%, so if we assume that’s the case for the next five years, the five-year benefit is $2 million instead. Given that the implementation time is, say, a conservative 20 hours, the associated staff time cost might be 20 * $100 = $2,000. Should GiveWell spend $2,000 to make $2,000,000? That’s a 1000x ROI.
Part of the reason why the article was mostly about my estimation methodology is because the exact forward-looking numbers don’t change the conclusions of the article.
So assuming an organization needed to make a grant within 12 moths, they could currently expect to make 0.10% on their cash balance. Needless to say this is 1/10th of 1%, or <1/20th of the figure cited in the post. And it seems a stretch to assume organizations hold cash for 12 months before granting it. So the likely benefit is perhaps at best 1/10th and potentially <1/100th of what’s cited here.
You are citing numbers for an unappealing option to store cash in, and using that to argue that that’s the best that organizations can do. Furthermore, you’re doing this after I already mentioned bank interest rates of 0.5%+ in my reply to Rob that you are replying to. A quick Google search indicates that there are lower risk options (business savings accounts) that yield 0.5% and upwards. I have no idea why you are quoting a 0.10% figure.
You may not be understanding what the article means by “cash balance.” As I make very clear in the article, we are using the organization’s estimated average daily cash balance to calculate the historical and future opportunity costs. The average daily cash balance is the amount that an organization literally makes interest on; I can’t think of a better number to use to calculate the interest an organization previously made and could be making in the future. All organizations have a cash balance, which we can say is the amount of cash in their checking and/or savings accounts, that they regularly make grants out of. As you can see from the historical opportunity cost estimation, clearly organizations have cash reserves on hand, and clearly those cash reserves could be earning more interest than they are now, regardless of whether or how they make grants.
Could you list out those statements? Are you talking about the article, or my comments? I would certainly not want to intentionally mislead, nor be outright incorrect.
Sure:
Cash balance methodology
You say:
Dividing $26,260 by GiveWell’s end of year cash balance in 2019—the $41,101,155 we calculated in the previous section—produces an upper bound interest rate of 0.0006389, or 0.064%.
Multiplying GiveWell’s $41,101,155 in cash at the end of 2018 by a 2.16% rate of interest yields a total of $887,785 in potential interest earnings which is nearly 34 times higher than the actual investment income GiveWell earned.
Subtracting the $26,260 upper bound on how much GiveWell actually earned on its cash in 2019 results in an annualized opportunity cost of $861,525.
Incorrect and misleading statements in these four paragraphs:
First paragraph: Dividing interest earned over the year by end of year cash is not an upper bound estimate interest. It’s actually more like a lower bound estimate, because it implicitly assumes that the cash level at end of year was the same as the average cash level throughout the year.
Since GiveWell is a grant-making organization that’s a very dangerous assumption to make (and their 990 form shows them making ~$30m in grants over the course of the year).
You acknowledge this in the next section, but then propose an equally questionable workaround:
GiveWell earns interest on its average cash balance during the year, rather than its cash balance at one point in time like the end of the year. We can better approximate this figure by averaging GiveWell’s beginning of year and end of year zero-interest and non-zero-interest cash and then summing them. This produces an estimated average cash balance in 2019 of $33,441,682.
In your reply to me you suggest:
You may not be understanding what the article means by “cash balance.” As I make very clear in the article, we are using the organization’s estimated average daily cash balance to calculate the historical and future opportunity costs. The average daily cash balance is the amount that an organization literally makes interest on; I can’t think of a better number to use [emphasis added]to calculate the interest an organization previously made and could be making in the future.
Even this approach seems incorrect or misleading:
The page for their Maximum Impact Fund shows the grants they make (which would come out of the cash balance shown in the 990), and it seems to show a couple of months lag between donations and grants.
The same page shows that funds received in the final three months of 2019 were substantially greater than those received in the first nine months of 2019, which is another reason that the end-of-2019 cash balance paints a misleading picture.
Second paragraph: The year you’ve chosen for your analysis happens to be the year with the highest cash rates in the past 12 years. This isn’t acknowledged, and you don’t backtest your strategy in other years (e.g. all the years where rates were ~0%).
Third & fourth paragraphs: Same analytical error as first paragraph.
Potential returns of other investment options
You say:
The 2.16% U.S. federal funds rate in 2019 is one of the most conservative interest rates possible. Higher interest rates would result in significantly higher annual opportunity costs. Our top recommendation for an ultra-short-term bond ETF (JPST) in our recommendation article published in early 2019 earned 3.34% in 2019, according to Portfolio Visualizer. There were no drops in account value on a monthly basis (0% maximum drawdown). Multiplying GiveWell’s $30,033,677 reasonable savings balance by 3.34% generates an opportunity cost of over one million dollars per year, at $1,003,124.
Citing the one year, backward looking return of JPST is misleading, and citing the one year drawdown performance is naive:
There’s nothing about that asset that makes its one year backward-looking return able to be extrapolated. Indeed this is basically the equivalent of picking a different asset that you know now performed well over a one year period and using that to estimate opportunity cost. Why not Bitcoin in 2017 or Tesla stock in 2020?
And re citing a year of drawdown data—the fund had a drawdown in March 2020! If you look at a similar index with a longer timeseries you’ll see there have been drawdowns, even against the backdrop of a period of secularly falling interest rates that juiced bond returns.
What’s more, that particular fund tracks a bond benchmark but is actively managed, meaning its managers can take more risk than the benchmark implies (and charge a management fee for doing so).
Using an even higher interest rate of 5.51%, which is what a very conservative portfolio of 20% stocks and 80% bonds earned on average (CAGR) from January 2000 to December 2019 calculated using Portfolio Visualizer, the one-year opportunity cost on GiveWell’s 2019 reasonable savings balance would be $1.65 million dollars a year. The 5.51% figure is an approximation of GiveWell’s expected opportunity cost rather than what it would have experienced, which would have been a small loss of −1.15% in 2018 followed by a 13.01% gain in 2019. For those curious about how this portfolio would have fared during COVID-19, the maximum negative month-over-month change in account value was −3.20% since the beginning of 2020; the portfolio is now up 9.11% for the YTD period ending November 30, 2020.
This analysis suffers from the same problem. At least there’s an attempt to look at a window longer than one year, but again it’s a cherry-picked period of 20 years, and it’s backward looking returns data implied to be forward-looking. You also mention the drawdown in 2018. What was the maximum drawdown in 2008 to 2009?
Given the organization is granting funds a couple of months after receiving them, it has little ability to tolerate drawdowns, making comparisons to a higher risk fund incorrect to misleading.
Conclusion and staff time
You say:
Regardless of the estimation method, it’s very likely the counterfactual impact of changing GiveWell’s financial practices is in the millions of dollars. Not bad for a time investment that should take between 1–10 hours of staff time to set up, and 0–2 hours of staff time to administer on an annual basis!
As shown above, getting a counterfactual impact number in the millions of dollars is not “very likely… regardless of the estimation method”. That statement is somewhere between incorrect and misleading. For instance if the cash balance is 10% of what you cite, and the interest rate is more like 0.50%, it doesn’t hold up.
The staff time estimate is also somewhere between incorrect and misleading:
If it’s just opening bank accounts, then the impact is GiveWell on average earning roughly the prevailing bank interest rate (close to zero for much of the relevant period)
If instead you want them (without the benefit of hindsight that you employ) picking actively managed JP Morgan bond funds, or creating and balancing a bond + stock fund, then I’d hazard it’s more than two hours of work per year!
So either it’s not much effort, and not much return, or there can be higher returns, but clearly it’s more than 2 hours a year of effort.
Thanks for taking the time to list those out Steve. I downvoted your comments because while I’m very happy to engage in this type of discussion with commenters, and have with others that have commented on this article, I feel like you are jumping to conclusions that things are “incorrect” or “misleading.” In fact, many of your points are already mentioned in the article itself. Additionally, you are attacking my writing by making comments like something is “naive” which I feel is not conducive to a positive and intellectual discussion about this. You’ll find that I’ve thought out the reasoning for all of the points you mention, either in the article itself, or because it’s fairly easy to understand why.
Cash balance methodology
Dividing interest earned over the year by end of year cash is not an upper bound estimate interest
In the sentence directly above this quote, I explain what I mean by upper bound: “GiveWell’s total cash and investment income in 2019 was $26,260. GiveWell held investments in 2019, so using the full amount on Line 3 to estimate GiveWell’s interest on cash in 2019 likely overestimates GiveWell’s actual interest earned on cash in 2019.”
I am saying that the number overstates the interest earned on cash alone because of the confounding effect of investment income.
It’s actually more like a lower bound estimate, because it implicitly assumes that the cash level at end of year was the same as the average cash level throughout the year.
The reason why this methodology is being used is stated in this first line of the article “This methodology is easy for anyone to replicate and is based on publicly available nonprofit financial statements (IRS Form 990) that all U.S. charities are required to file yearly.”
The Form 990 does not include a better estimate of an organization’s average daily cash balance. This is why I am calling this an “estimation” methodology. In the “Estimation Methodology Caveats” section I cover this and other issues in depth (and I’ve already mentioned this to other commenters, who also brought up this valid point). A short quote: “The average of a nonprofit’s start of year and end of year cash could noticeably underestimate or overestimate a nonprofit’s actual average cash balance during the year because it only uses two days as data points...”
Since GiveWell is a grant-making organization that’s a very dangerous assumption to make (and their 990 form shows them making ~$30m in grants over the course of the year).
GiveWell is used as an example—this analysis is meant to highlight the methodology, including its shortcomings, not be a deep dive into GiveWell itself. I don’t have the data needed to estimate this better. Neither you nor I know how much money GiveWell actually holds during other times of the year. They receive funds year-round and need reserves on hand to pay operational expenses.
You acknowledge this in the next section, but then propose an equally questionable workaround
That is not correct, the workaround is, as I state in the article, meant to account for the confounding effects of money made from investments rather than cash holdings by only looking at the cash balance we know isn’t earning any interest. This “workaround” has nothing to do with the cash balance issue you mentioned.
The same page shows that funds received in the final three months of 2019 were substantially greater than those received in the first nine months of 2019, which is another reason that the end-of-2019 cash balance paints a misleading picture
Thanks for pointing this out. I didn’t look more closely into GiveWell’s specifics because GiveWell is used as an example of a methodology that generalizes to other charities as I stated in the article. Data like this is very difficult to correlate with an organization’s cash balance. We don’t know what percentage of GiveWell’s cash balance the Maximum Impact Fund is, what the Maximum Impact Fund balance is after a grant is made, whether the “amount” column reflects all donor inflows during a specified period or also granting from reserves or other revenue sources, etc.
The year you’ve chosen for your analysis happens to be the year with the highest cash rates in the past 12 years.
The most recent data is most relevant to GiveWell’s current and future opportunity costs.
This isn’t acknowledged, and you don’t backtest your strategy in other years (e.g. all the years where rates were ~0%).
That is a good point! Time did not permit, but as I stated in my reply to Rob, I do think that “to get savings accounts yields themselves, we might need to use a third-party service or design a benchmark ourselves. Given the current situation with the federal funds rate, it would make sense for us to find a better proxy for good savings accounts yields in 2020 so that we can better estimate the opportunity cost for 2020 once the 2020 Form 990s come out as well as use the rate as a point of reference for current and future expected earnings.”
So it is a good idea to use an alternative metric to the federal funds rate, particularly for lower interest rate years, so they more accurately reflect the opportunity cost. Off the top of my head I think that rates were around 1% (see Ally Bank’s 2015 and 2016 historical yields when the fed funds rate was 0.13% and 0.39% respectively), which is the exact same rate I used for the forward-looking estimate for GiveWell. Using savings yields from one bank or a blend of banks instead still wouldn’t change the article’s conclusions, given that those rates would be both higher than the federal funds rate and also significant enough to warrant consideration even in a zero interest rate environment.
Potential returns of other investment options
There’s nothing about that asset that makes its one year backward-looking return able to be extrapolated. Indeed this is basically the equivalent of picking a different asset that you know now performed well over a one year period and using that to estimate opportunity cost. Why not Bitcoin in 2017 or Tesla stock in 2020?
Exactly, why wouldn’t I pick Bitcoin or Tesla? Because both ultra-short-term bonds and a standard conservative risk portfolio are extremely common investment portfolios for more risk averse investors such as many nonprofits who require a steady rate of return with minimal drawdowns. It doesn’t matter what examples you use, low-risk options would have performed well during this time period and other time periods, just worse than riskier options.
Citing the one year, backward looking return of JPST is misleading
And another reason I used JPST as an example is because I recommend JPST in my early-2019 article before I could see its historical performance. So I’m not sure if “backward looking” is the right verbiage.
And re citing a year of drawdown data—the fund had a drawdown in March 2020! If you look at a similar index with a longer timeseries you’ll see there have been drawdowns, even against the backdrop of a period of secularly falling interest rates that juiced bond returns.
I already state in the article that I am quoting month-to-month drawdown data, not day-to-day data. This means the fund did not decline in value month by month. The highest historical month-to-month drawdown was −1.72% and it recovered in two months. Hardly a volatile investment.
This analysis suffers from the same problem. At least there’s an attempt to look at a window longer than one year, but again it’s a cherry-picked period of 20 years, and it’s backward looking returns data implied to be forward-looking. You also mention the drawdown in 2018. What was the maximum drawdown in 2008 to 2009?
Given the organization is granting funds a couple of months after receiving them, it has little ability to tolerate drawdowns, making comparisons to a higher risk fund incorrect to misleading.
I did not cherry pick the time period, stocks and bonds have generally performed well across time. Portfolio Visualizer has data starting from 1987, which indicates an annual return even higher than what I quoted, at 7.07%. Maximum drawdown is −8.49% from the 2008 crash. Exactly why this is considered a standard conservative portfolio by financial professionals.
Given the organization is granting funds a couple of months after receiving them, it has little ability to tolerate drawdowns, making comparisons to a higher risk fund incorrect to misleading.
As we can see, the drawdown is very low, and the expected value is positive and significant. For granted funds that GiveWell cannot afford to have a drawdown on (this does not represent all of their funds), then a “zero-risk” option like a savings account would make more sense. It all depends on the charity. Investment examples are for illustrative purposes, I have no idea what approaches GiveWell can or cannot take in actuality, beyond a savings account, which it should be able to do.
Conclusion and staff time
As shown above, getting a counterfactual impact number in the millions of dollars is not “very likely… regardless of the estimation method”. That statement is somewhere between incorrect and misleading. For instance if the cash balance is 10% of what you cite, and the interest rate is more like 0.50%, it doesn’t hold up.
That depends on what numbers we’re using. I think 10% is very low. As I’ve said, the Maximum Impact Fund cannot be used as a reliable estimate or predictor of GiveWell’s average daily cash balance. But if we assume just the fund itself represents all of GiveWell’s money, we can see that there’s a 3-month grant collection period (averages to 1.5 months of cash at 100%) plus a 2–3 month delay in granting (averages to 2.5 months of cash at 100%) which would suggest a minimum percentage of at least 33% (4/12 months) just based on the money collected during the last three months of the year.
The staff time estimate is also somewhere between incorrect and misleading:
If it’s just opening bank accounts, then the impact is GiveWell on average earning roughly the prevailing bank interest rate (close to zero for much of the relevant period)
If instead you want them (without the benefit of hindsight that you employ) picking actively managed JP Morgan bond funds, or creating and balancing a bond + stock fund, then I’d hazard it’s more than two hours of work per year!
So either it’s not much effort, and not much return, or there can be higher returns, but clearly it’s more than 2 hours a year of effort.
This is probably evident from the rest of my replies, but a 1% account savings rate or even 0.5% still produces significant financial returns. These numbers should be used instead of 0%.
And it would take a very small amount of time to keep funds in one fund for multiple years, whether that’s JPST or a multi-asset fund that has, say, a 20% stock allocation and 80% bond allocation. Vanguard, a well-known and trusted provider, has a great ultra-short-term bond fund as a JPST alternative. A nonprofit might not maximize returns by shifting between funds on the reg (which is, as you say, time consuming, although Antigravity Investments is more than happy to help with that), but evidently they will do much better than nothing. Many nonprofits self-manage their investment approach.
I agree with Rob’s points.
This article makes several statements that I interpret as somewhere between intentionally misleading and outright incorrect.
For instance, this reply says the 2.19% Fed Funds rate was the prevailing rate in 2019, and isn’t intended to be a forecast of the future. But since we know the Fed Funds rate is 0% today, why would that rate be used to argue for a forward-looking change in behavior?
Similarly, the 2019 return of a short-term bond fund gives literally no information about what returns a similar fund would offer today. Indeed the beauty of the bond market is that it does offer guaranteed future returns if you hold a bond to maturity. As of today, US government bonds (which carry no risk of default) offer a guaranteed annualized return of:
0.08% if held for 3 months,
0.10% if held for 12 months,
0.13% if held for 2 years,
0.94% if held for 10 years
So assuming an organization needed to make a grant within 12 moths, they could currently expect to make 0.10% on their cash balance. Needless to say this is 1/10th of 1%, or <1/20th of the figure cited in the post. And it seems a stretch to assume organizations hold cash for 12 months before granting it. So the likely benefit is perhaps at best 1/10th and potentially <1/100th of what’s cited here.
Granted, this is not necessarily obvious stuff. However it is and should be easily understood by anyone holding themselves out to be giving financial advice to other people or institutions.
Could you list out those statements? Are you talking about the article, or my comments? I would certainly not want to intentionally mislead, nor be outright incorrect.
As I state in the introduction, “In this section, we will introduce our cash interest opportunity cost estimation methodology for U.S. 501(c)(3) charities.” The majority of this article is about open sourcing the estimation methodology we developed, which took considerable effort over multiple years. We analyzed financial data across over 300,000 nonprofits and ran the methodology past the IRS and an independent accountant. This methodology was not designed to forecast future opportunity costs, and doing so seemed especially unnecessary in the pre-COVID-19 environment of rising interest rates that we originally developed the methodology in.
As you can see by the dollar amount of the $4 million dollar forward-looking estimate for GiveWell (based on a 1% interest rate), the interest rate does not need to be 2% for this recommendation to make sense. Current bank yields are 0.5%, so if we assume that’s the case for the next five years, the five-year benefit is $2 million instead. Given that the implementation time is, say, a conservative 20 hours, the associated staff time cost might be 20 * $100 = $2,000. Should GiveWell spend $2,000 to make $2,000,000? That’s a 1000x ROI.
Part of the reason why the article was mostly about my estimation methodology is because the exact forward-looking numbers don’t change the conclusions of the article.
You are citing numbers for an unappealing option to store cash in, and using that to argue that that’s the best that organizations can do. Furthermore, you’re doing this after I already mentioned bank interest rates of 0.5%+ in my reply to Rob that you are replying to. A quick Google search indicates that there are lower risk options (business savings accounts) that yield 0.5% and upwards. I have no idea why you are quoting a 0.10% figure.
You may not be understanding what the article means by “cash balance.” As I make very clear in the article, we are using the organization’s estimated average daily cash balance to calculate the historical and future opportunity costs. The average daily cash balance is the amount that an organization literally makes interest on; I can’t think of a better number to use to calculate the interest an organization previously made and could be making in the future. All organizations have a cash balance, which we can say is the amount of cash in their checking and/or savings accounts, that they regularly make grants out of. As you can see from the historical opportunity cost estimation, clearly organizations have cash reserves on hand, and clearly those cash reserves could be earning more interest than they are now, regardless of whether or how they make grants.
Sure:
Cash balance methodology
You say:
Incorrect and misleading statements in these four paragraphs:
First paragraph: Dividing interest earned over the year by end of year cash is not an upper bound estimate interest. It’s actually more like a lower bound estimate, because it implicitly assumes that the cash level at end of year was the same as the average cash level throughout the year.
Since GiveWell is a grant-making organization that’s a very dangerous assumption to make (and their 990 form shows them making ~$30m in grants over the course of the year).
You acknowledge this in the next section, but then propose an equally questionable workaround:
In your reply to me you suggest:
Even this approach seems incorrect or misleading:
The page for their Maximum Impact Fund shows the grants they make (which would come out of the cash balance shown in the 990), and it seems to show a couple of months lag between donations and grants.
The same page shows that funds received in the final three months of 2019 were substantially greater than those received in the first nine months of 2019, which is another reason that the end-of-2019 cash balance paints a misleading picture.
Second paragraph: The year you’ve chosen for your analysis happens to be the year with the highest cash rates in the past 12 years. This isn’t acknowledged, and you don’t backtest your strategy in other years (e.g. all the years where rates were ~0%).
Third & fourth paragraphs: Same analytical error as first paragraph.
Potential returns of other investment options
You say:
Citing the one year, backward looking return of JPST is misleading, and citing the one year drawdown performance is naive:
There’s nothing about that asset that makes its one year backward-looking return able to be extrapolated. Indeed this is basically the equivalent of picking a different asset that you know now performed well over a one year period and using that to estimate opportunity cost. Why not Bitcoin in 2017 or Tesla stock in 2020?
And re citing a year of drawdown data—the fund had a drawdown in March 2020! If you look at a similar index with a longer timeseries you’ll see there have been drawdowns, even against the backdrop of a period of secularly falling interest rates that juiced bond returns.
What’s more, that particular fund tracks a bond benchmark but is actively managed, meaning its managers can take more risk than the benchmark implies (and charge a management fee for doing so).
This analysis suffers from the same problem. At least there’s an attempt to look at a window longer than one year, but again it’s a cherry-picked period of 20 years, and it’s backward looking returns data implied to be forward-looking. You also mention the drawdown in 2018. What was the maximum drawdown in 2008 to 2009?
Given the organization is granting funds a couple of months after receiving them, it has little ability to tolerate drawdowns, making comparisons to a higher risk fund incorrect to misleading.
Conclusion and staff time
You say:
As shown above, getting a counterfactual impact number in the millions of dollars is not “very likely… regardless of the estimation method”. That statement is somewhere between incorrect and misleading. For instance if the cash balance is 10% of what you cite, and the interest rate is more like 0.50%, it doesn’t hold up.
The staff time estimate is also somewhere between incorrect and misleading:
If it’s just opening bank accounts, then the impact is GiveWell on average earning roughly the prevailing bank interest rate (close to zero for much of the relevant period)
If instead you want them (without the benefit of hindsight that you employ) picking actively managed JP Morgan bond funds, or creating and balancing a bond + stock fund, then I’d hazard it’s more than two hours of work per year!
So either it’s not much effort, and not much return, or there can be higher returns, but clearly it’s more than 2 hours a year of effort.
Thanks for taking the time to list those out Steve. I downvoted your comments because while I’m very happy to engage in this type of discussion with commenters, and have with others that have commented on this article, I feel like you are jumping to conclusions that things are “incorrect” or “misleading.” In fact, many of your points are already mentioned in the article itself. Additionally, you are attacking my writing by making comments like something is “naive” which I feel is not conducive to a positive and intellectual discussion about this. You’ll find that I’ve thought out the reasoning for all of the points you mention, either in the article itself, or because it’s fairly easy to understand why.
Cash balance methodology
In the sentence directly above this quote, I explain what I mean by upper bound: “GiveWell’s total cash and investment income in 2019 was $26,260. GiveWell held investments in 2019, so using the full amount on Line 3 to estimate GiveWell’s interest on cash in 2019 likely overestimates GiveWell’s actual interest earned on cash in 2019.”
I am saying that the number overstates the interest earned on cash alone because of the confounding effect of investment income.
The reason why this methodology is being used is stated in this first line of the article “This methodology is easy for anyone to replicate and is based on publicly available nonprofit financial statements (IRS Form 990) that all U.S. charities are required to file yearly.”
The Form 990 does not include a better estimate of an organization’s average daily cash balance. This is why I am calling this an “estimation” methodology. In the “Estimation Methodology Caveats” section I cover this and other issues in depth (and I’ve already mentioned this to other commenters, who also brought up this valid point). A short quote: “The average of a nonprofit’s start of year and end of year cash could noticeably underestimate or overestimate a nonprofit’s actual average cash balance during the year because it only uses two days as data points...”
GiveWell is used as an example—this analysis is meant to highlight the methodology, including its shortcomings, not be a deep dive into GiveWell itself. I don’t have the data needed to estimate this better. Neither you nor I know how much money GiveWell actually holds during other times of the year. They receive funds year-round and need reserves on hand to pay operational expenses.
That is not correct, the workaround is, as I state in the article, meant to account for the confounding effects of money made from investments rather than cash holdings by only looking at the cash balance we know isn’t earning any interest. This “workaround” has nothing to do with the cash balance issue you mentioned.
Thanks for pointing this out. I didn’t look more closely into GiveWell’s specifics because GiveWell is used as an example of a methodology that generalizes to other charities as I stated in the article. Data like this is very difficult to correlate with an organization’s cash balance. We don’t know what percentage of GiveWell’s cash balance the Maximum Impact Fund is, what the Maximum Impact Fund balance is after a grant is made, whether the “amount” column reflects all donor inflows during a specified period or also granting from reserves or other revenue sources, etc.
The most recent data is most relevant to GiveWell’s current and future opportunity costs.
That is a good point! Time did not permit, but as I stated in my reply to Rob, I do think that “to get savings accounts yields themselves, we might need to use a third-party service or design a benchmark ourselves. Given the current situation with the federal funds rate, it would make sense for us to find a better proxy for good savings accounts yields in 2020 so that we can better estimate the opportunity cost for 2020 once the 2020 Form 990s come out as well as use the rate as a point of reference for current and future expected earnings.”
So it is a good idea to use an alternative metric to the federal funds rate, particularly for lower interest rate years, so they more accurately reflect the opportunity cost. Off the top of my head I think that rates were around 1% (see Ally Bank’s 2015 and 2016 historical yields when the fed funds rate was 0.13% and 0.39% respectively), which is the exact same rate I used for the forward-looking estimate for GiveWell. Using savings yields from one bank or a blend of banks instead still wouldn’t change the article’s conclusions, given that those rates would be both higher than the federal funds rate and also significant enough to warrant consideration even in a zero interest rate environment.
Potential returns of other investment options
Exactly, why wouldn’t I pick Bitcoin or Tesla? Because both ultra-short-term bonds and a standard conservative risk portfolio are extremely common investment portfolios for more risk averse investors such as many nonprofits who require a steady rate of return with minimal drawdowns. It doesn’t matter what examples you use, low-risk options would have performed well during this time period and other time periods, just worse than riskier options.
And another reason I used JPST as an example is because I recommend JPST in my early-2019 article before I could see its historical performance. So I’m not sure if “backward looking” is the right verbiage.
I already state in the article that I am quoting month-to-month drawdown data, not day-to-day data. This means the fund did not decline in value month by month. The highest historical month-to-month drawdown was −1.72% and it recovered in two months. Hardly a volatile investment.
I did not cherry pick the time period, stocks and bonds have generally performed well across time. Portfolio Visualizer has data starting from 1987, which indicates an annual return even higher than what I quoted, at 7.07%. Maximum drawdown is −8.49% from the 2008 crash. Exactly why this is considered a standard conservative portfolio by financial professionals.
As we can see, the drawdown is very low, and the expected value is positive and significant. For granted funds that GiveWell cannot afford to have a drawdown on (this does not represent all of their funds), then a “zero-risk” option like a savings account would make more sense. It all depends on the charity. Investment examples are for illustrative purposes, I have no idea what approaches GiveWell can or cannot take in actuality, beyond a savings account, which it should be able to do.
Conclusion and staff time
That depends on what numbers we’re using. I think 10% is very low. As I’ve said, the Maximum Impact Fund cannot be used as a reliable estimate or predictor of GiveWell’s average daily cash balance. But if we assume just the fund itself represents all of GiveWell’s money, we can see that there’s a 3-month grant collection period (averages to 1.5 months of cash at 100%) plus a 2–3 month delay in granting (averages to 2.5 months of cash at 100%) which would suggest a minimum percentage of at least 33% (4/12 months) just based on the money collected during the last three months of the year.
This is probably evident from the rest of my replies, but a 1% account savings rate or even 0.5% still produces significant financial returns. These numbers should be used instead of 0%.
And it would take a very small amount of time to keep funds in one fund for multiple years, whether that’s JPST or a multi-asset fund that has, say, a 20% stock allocation and 80% bond allocation. Vanguard, a well-known and trusted provider, has a great ultra-short-term bond fund as a JPST alternative. A nonprofit might not maximize returns by shifting between funds on the reg (which is, as you say, time consuming, although Antigravity Investments is more than happy to help with that), but evidently they will do much better than nothing. Many nonprofits self-manage their investment approach.