Can we apply start-up investing principles to non-profits?

How do you find the best non-prof­its to donate to? This is an im­por­tant ques­tion that is crit­i­cal to effec­tive al­tru­ism.

One sug­ges­tion comes from Holden Karnofsky at the Open Philan­thropy Pro­ject, who de­scribes a strat­egy called “hits-based giv­ing”. In this frame­work, you make a num­ber of in­vest­ments, some of which are very counter-in­tu­itive and against ex­pert con­sen­sus, with the un­der­stand­ing that many will not amount to much but those that work will gen­er­ate ex­cess re­turns to make the over­all port­fo­lio have a high al­tru­is­tic re­turn on philan­thropic in­vest­ment.

This strat­egy origi­nates from YCom­bi­na­tor. In the es­say “Black Swan Farm­ing”, Paul Gra­ham ar­gues that fund­ing for-profit star­tups is the art of hunt­ing for the one deal that will make it big. You have a lot of “misses” when you in­vest, but the one time you make a “hit”, it will hit big and re­pay all your losses and then some. In or­der to guess right, you have to make many gam­bles. YCom­bi­na­tor has been work­ing on this prob­lem since 2005, and has since in­vested over $170M into over 1400 differ­ent start-ups. The com­bined val­u­a­tion of their cur­rent start-up batch is stated to now be over $80B.

“Black swan farm­ing” seems to work well for YCom­bi­na­tor. But does it ap­ply well when donat­ing to non-prof­its? Does hits-based giv­ing work? Since writ­ing that post on April 2016, OpenPhil has already al­lo­cated over $197M ac­cord­ing to this philos­o­phy. YCom­bi­na­tor is also ap­ply­ing hits-based giv­ing to their own batch of non-prof­its, to which they have donated $3M.

The “Start Up” Approach

Separately, Ben Todd out­lines that many donors con­cerned with effec­tive­ness judge or­ga­ni­za­tions based on their short-term marginal im­pact. For ex­am­ple, as Todd men­tions, GiveWell had re­turns lower than its costs for the first four years, but then quickly ex­ploded in its fundrais­ing ra­tio, dou­bling money moved from 2012 to 2013, again from 2013 to 2014, and mov­ing more money in 2015 than twice as much raised in 2013 and 2014 com­bined. An im­pact as­sess­ment fo­cused solely on short-run fundrais­ing ra­tios in 2011 would have missed GiveWell as an in­cred­ibly valuable in­vest­ment.

In con­trast, Todd ar­gues for eval­u­at­ing early “start-up” non-prof­its with stan­dard start-up met­rics, such as mak­ing sure they have a high-qual­ity product, a large ad­dress­able mar­ket, and the abil­ity to “sell” to this mar­ket at scale. Similarly, the or­ga­ni­za­tion should have a good growth rate and the team should ideally demon­strate com­pe­tence and have a track record. For ex­am­ple, GiveWell had a su­pe­rior re­search product with the abil­ity to scale to mil­lions of small donors plus dozens of in­ter­ested large-scale foun­da­tions. While the team did not have much of a prior track record, they showed their com­pe­tence through their early re­search and early trac­tion with donors.

Lastly, Todd im­plies that up­front, early in­vest­ments in rigor­ous cost-effec­tive­ness analy­ses are pre­ma­ture, as they draw at­ten­tion away from grow­ing the core product in qual­ity and scale, and they likely fo­cus too much on the short-run im­pact, ig­nor­ing long-run op­por­tu­ni­ties.

Ven­ture Cap­i­tal vs. Hedge Funds

While the ter­minol­ogy ap­plied is very loose and hard to gen­er­al­ize, the ar­gu­ments by Karnos­fky and Todd seem to com­pare non-profit dona­tions to “start up” in­vest­ments via ven­ture cap­i­tal—do­ing what Gra­ham and Thiel sug­gest and mak­ing hun­dreds of guesses to find the few di­a­monds in the rough that provide out­sized re­turns.

How­ever, this is not the only form of for-profit in­vest­ing. One might also con­sider the ap­proach of hedge funds, which ap­pear to rel­a­tively em­ploy less of a “hits-based” ap­proach and more of an up­front in­vest­ment in an­a­lyt­ics. While VCs do some due dili­gence, hedge funds of­ten em­ploy very com­plex risk mod­el­ing when mak­ing in­vest­ments.

This means if we could suc­cess­fully gen­er­al­ize and com­pare ven­ture cap­i­tal ver­sus hedge funds and see if one strat­egy gen­er­ates su­pe­rior re­turns com­pared to the other we could have some pre­limi­nary ev­i­dence for whether it is bet­ter to be “hits-based” or “ev­i­dence-based”.

Frus­trat­ingly, it is very difficult to com­pare the av­er­age re­turns of ven­ture cap­i­tal and hedge funds be­cause the in­tra-group vari­a­tion be­tween in­di­vi­d­ual firms is mas­sive and dwarfs com­par­i­sons be­tween the two groups. Also, the pri­vate na­ture of firms and se­lec­tion bias in re­port­ing makes find­ing ac­cu­rate, sys­tem­atic sum­mary statis­tics quite hard.

A liter­a­ture re­view of rele­vant re­search on VC firms finds that the av­er­age re­turns of VC are roughly equiv­a­lent to that of the stock mar­ket though with sig­nifi­cant vari­a­tion and method­olog­i­cal un­cer­tainty (Rin, Hel­l­mann, & Puri, 2011, p78-80). Fur­ther­more, choices of sam­pling pe­ri­ods and method­ol­ogy can dra­mat­i­cally change whether ven­ture cap­i­tal is de­ter­mined to be more or less prof­itable than pri­vate equity on av­er­age (Rin, Hel­l­mann, & Puri, 2011, p90).

Over­all, high vari­a­tion be­tween in­di­vi­d­ual firms in the same gen­eral cat­e­gory, the loose­ness of cat­e­gory defi­ni­tions, the highly pri­va­tized na­ture of in­di­vi­d­ual firm strate­gies, and the high un­cer­tainty in re­sults means that we un­for­tu­nately can­not draw firm con­clu­sions from this line of in­ves­ti­ga­tion. This might mean that ei­ther ap­proach is fine, but varies a lot more based on man­age­ment and cir­cum­stances than ap­proach to in­vest­ing, but it is very hard to gen­er­al­ize this to non-prof­its.

How Similar Are For-Profit and Non-Profit In­vest­ments?

How­ever, even com­par­ing non-profit dona­tions to hedge funds im­plies that mak­ing a dona­tion is like for-profit in­vest­ing. This is a view that many im­pact-in­ter­ested donors ap­pear to hold—the prevalence of the phrase “im­pact in­vest­ing” as a term for effi­cient giv­ing drives this anal­ogy home. How­ever, the stan­dard ad­vice for in­di­vi­d­ual for-profit in­vestors is to avoid try­ing to “beat the mar­ket” by search­ing for in­vest­ment op­por­tu­ni­ties on one’s own and in­stead to in­vest in in­dex funds. Does this mean that non-profit in­vestors should be ad­vised to donate to al­tru­is­tic “in­dex funds” as well?

How similar is for-profit and non-profit in­vest­ing? It ap­pears to me like there are nu­mer­ous key differ­ences:

  • Non-profit in­vest­ing af­fords you the op­por­tu­nity to be far more risk-neu­tral than you can in for-profit in­vest­ing, which changes your op­tions. In­dex funds are typ­i­cally cho­sen less be­cause the di­ver­sifi­ca­tion in­creases av­er­age re­turns, but rather be­cause the di­ver­sifi­ca­tion de­creases the var­i­ance of the in­vest­ment, ex­pos­ing you to less risk. A risk-neu­tral for-profit in­vestor might be pur­su­ing var­i­ance in­creas­ing strate­gies in­stead, like lev­er­age. How­ever, al­tru­is­tic in­vest­ments are not used with the in­ten­tion of sav­ing for one’s own fu­ture, which al­lows the al­tru­ist to be more risk-neu­tral to chase higher ex­pected re­turns.

  • The prices of non-profit in­vest­ments don’t in­stantly change when they’re iden­ti­fied as more valuable, al­low­ing good deals to be available much longer. If anal­y­sis shows that a par­tic­u­lar stock is very hot, say offer­ing the op­por­tu­nity to in­vest $5 to get $50, ac­cord­ing to the effi­cient mar­ket hy­poth­e­sis, that anal­y­sis will nearly in­stantly be priced into the stock and the stock will quickly be­come worth ~$50. How­ever, if a dona­tion op­por­tu­nity al­lows you to donate $3500 to save a life, ar­guably worth about $9.1M, the dona­tion op­por­tu­nity does not sud­denly get bid up to $9.1M per life saved. In­stead, the dona­tion op­por­tu­nity is used up un­til diminish­ing marginal re­turns mean it no longer ex­ists, which hap­pens sig­nifi­cantly more slowly than a hot stock changes price. There­fore we should ex­pect good giv­ing to be hard, but not nearly as hard as find­ing a hot stock.

  • For-profit in­vest­ing typ­i­cally does not have mas­sive nega­tive re­turns, but non-profit in­vest­ing can. Un­less you’re in­vest­ing with lev­er­age, break­ing the law, are a mas­sive “too big to fail” fi­nan­cial in­sti­tu­tion, or oth­er­wise are do­ing some­thing weird, the for-profit in­vest­ments you make will typ­i­cally not lose any more money than you put in. If you in­vest $1M, the worst that can hap­pen is that you lose $1M. How­ever, with non-profit in­vest­ing, when you donate $1M, you run the risk of the non-profit be­ing some­how net nega­tive in con­fus­ing ways. Be­ing able to guard against these risks is im­por­tant in non-profit in­vest­ing, but not in for-profit in­vest­ing.

  • Non-profit in­vest­ing lets you ar­bi­trage based on your val­ues, whereas for-profit in­vest­ing does not. Many foun­da­tions spend­ing mil­lions of dol­lars spend it based on val­ues that are differ­ent than EA prin­ci­ples of be­ing neu­tral to­ward the lo­ca­tion or species of those that you help, or be­ing neu­tral to­ward tak­ing very far-fu­ture bets. The more your val­ues de­part from the global main­stream, the eas­ier it should be to find good giv­ing op­por­tu­ni­ties (e.g., donat­ing or start­ing some­thing your­self) that max­i­mize those val­ues, be­cause the good ones won’t have been taken yet (un­less your val­ues are so weird that no one is will­ing to help you make progress on them).

  • More peo­ple are try­ing a lot harder to “beat you” in for-profit in­vest­ing than non-profit in­vest­ing. In the for-profit world, your quest to find a hot stock with amaz­ing ROI is go­ing up against hun­dreds of thou­sands of in­cred­ibly well-funded an­a­lysts col­lec­tively work­ing billions of hours a year to out­com­pete you. On the other hand, in the non-profit world, while some sharp non-profit in­vestors are buy­ing up the best op­por­tu­ni­ties already, it seems like most peo­ple don’t care, and the to­tal com­mu­nity of peo­ple chas­ing op­ti­mal dona­tions is a few thou­sand peo­ple per­haps col­lec­tively spend­ing at most 2M hours a year. This makes it many times eas­ier to out­com­pete the mar­ket in non-profit in­vest­ing.

  • Non-profit in­vest­ing isn’t even a com­pe­ti­tion and an­a­lysts will share their best op­por­tu­ni­ties with you for free. Un­like effec­tive for-profit in­vest­ing, effec­tive al­tru­ism isn’t a com­pe­ti­tion and since great dona­tion op­por­tu­ni­ties take years to go away, they can be shared with you for free. If ev­ery sin­gle for-profit hedge fund gave you in­stant, free ac­cess to their ad­vice in an eas­ily sum­ma­rized form, I imag­ine for-profit in­vest­ing would slant away from in­dex funds too.

  • The re­turns for for-profit funds are rel­a­tively clear, but non-profit re­turns re­quire a lot of work to un­der­stand. While there might be is­sues of ap­ply­ing the cor­rect method­ol­ogy, you can gen­er­ally look at how much cash you get back for how much cash you put in. With non-profit in­vest­ing, there is no clear mea­sure of your re­turn on in­vest­ment. In­stead, you have to use com­plex anal­y­sis to as­sess your re­turn and some in­vest­ments will never be able to show a con­clu­sive re­turn even if they do have one.

  • Non-profit in­vestors do not have a clear “in­dex fund”. An in­dex fund tries to di­ver­sify as much as pos­si­ble by in­vest­ing in a wide va­ri­ety of stocks from a wide va­ri­ety of mar­kets. The S&P 500 is ba­si­cally like in­vest­ing a tiny bit in ev­ery large com­pany in the US. On the other hand, in­vest­ing in GiveWell’s top char­i­ties or in Effec­tive Altru­ism Funds is a lot more like in­vest­ing in an ac­tively man­aged fund that has no ex­pense ra­tio—a fund that looks at many pos­si­ble stocks but se­lects only the few that they think will beat the av­er­age. A true al­tru­is­tic “in­dex fund” would in­stead in­vest a small amount in ev­ery sin­gle char­ity and di­ver­sify as much as pos­si­ble. The fact that this sounds like such a bad idea (and con­versely that in­vest­ing ev­ery­thing in only one stock sounds like a bad idea for per­sonal fi­nance) shows the differ­ence be­tween for-profit and non-profit in­vest­ing.

There­fore, while pur­su­ing higher re­turns at the chance of higher risk can be a good strat­egy for both start-up in­vest­ing and op­ti­mal donat­ing, there are also im­por­tant differ­ences be­tween these two ac­tivi­ties. For-profit in­vestors have to ex­ploit their in­sider knowl­edge and con­nec­tions via start-up in­vest­ing to beat the mar­ket, but in the non-profit world, the differ­ences in pric­ing, the pool­ing of wis­dom, and the rel­a­tive lack of com­pe­ti­tion means that high re­turns might be found through an ev­i­dence-based ap­proach. More­over, the difficulty of un­der­stand­ing non-profit re­turns would sug­gest that non-profit in­vestors would have to col­lect a lot of ev­i­dence just to un­der­stand how well their port­fo­lios are do­ing.

The In­cu­ba­tion Approach

The difficulty of un­der­stand­ing non-profit re­turns, the abil­ity to widely dis­sem­i­nate im­pact anal­y­sis, plus the lack of quickly diminish­ing re­turns, places a high pre­mium on the value of col­lect­ing in­for­ma­tion and com­mu­ni­cat­ing it with the rest of the im­pact-in­ter­ested com­mu­nity. Ar­guably, the effec­tive al­tru­ism com­mu­nity cur­rently un­der-in­vests in ex­plo­ra­tion, and this anal­y­sis pro­vides some ad­di­tional the­o­retic rea­sons why ex­plo­ra­tion could be so highly valuable.

The mere pres­ence of large “hits” com­bine with the pos­si­bil­ity of miss­ing them is not, in it­self, per­sua­sive—that’s just FOMO. For ex­am­ple, the pos­si­bil­ity of there be­ing “hit” lot­tery tick­ets does not sug­gest it is a good idea to do “hits-based” lot­tery ticket buy­ing. In­deed, any in­vest­ing strat­egy has both a false pos­i­tive and a false nega­tive rate, and care needs to be paid to both. If this comes at the cost of oc­ca­sion­ally miss­ing out on a big “hit”, that doesn’t mean your strat­egy is wrong. In­stead, we would want to as­cer­tain whether we have prop­erly bal­anced our false pos­i­tive and false nega­tive rates to pro­duce the high­est ex­pected re­turns. While I know Karnofsky and Todd have thought about this a lot and are not solely driven by FOMO, I do not think there has been enough pub­lished anal­y­sis of this type.

On the other hand, I agree with Todd that many EAs overem­pha­size the false nega­tive rate with too much de­sire for rigor. I agree it’s im­por­tant to have prin­ci­ples that would al­low for tak­ing risk on in­no­va­tive ideas, and would have al­lowed you to fund or­ga­ni­za­tions like The Against Malaria Foun­da­tion in the be­gin­ning, be­fore they be­gan to show signs of im­pact.

Another non-profit in­vest­ing frame­work I think is worth con­sid­er­ing is rep­re­sented by GiveWell’s in­cu­ba­tion grants, the Global In­no­va­tion Fund, Char­ity Science’s search for co-founders for fu­ture ex­cep­tional global poverty char­i­ties, and maybe EA Grants (though I’m not sure yet). While I don’t un­der­stand the ex­act pro­cess for vet­ting and ap­prov­ing these grants from these orgs, these grants seem like a great way to buy in­for­ma­tion.

My ideal­ized ver­sion of the vet­ting pro­cess might go some­thing like this:

  1. Get a large pool of can­di­date pro­jects and pro­ject teams, whether by so­lic­it­ing ap­pli­ca­tions and/​or wait­ing for peo­ple to ap­ply.

  1. Fol­low­ing Todd’s ap­proach, eval­u­ate each pro­ject based on the qual­ity of the team mem­bers, the qual­ity of the core ser­vice or product the non-profit aims to provide, the up­side op­por­tu­nity of what the non-profit could po­ten­tially achieve, and the like­li­hood of achiev­ing this scale. This could be done with a few in­ter­views and some guess­work. It’s im­por­tant to ac­knowl­edge there are many ways in which this eval­u­a­tion may not ac­cu­rately pre­dict im­pact.

  1. Iden­tify the top pro­jects you are will­ing to fund that meet the cut based on the crite­ria in step 2. Challenge these pro­jects to come up with a plan to “prove” their model within a short but rea­son­able timeframe (e.g., 1-3 years). Offer them fund­ing to cover all of their costs while they prove them­selves, re-eval­u­at­ing af­ter each year. If they don’t ap­pear to make the bar, re­quire them to try some­thing else.

  1. When an or­ga­ni­za­tion does demon­strate their cost-effec­tive­ness to an ad­e­quate de­gree, give them all the fund­ing they need to scale up (e.g., by be­ing a GiveWell top char­ity and re­ceiv­ing mil­lions in fund­ing).

  1. Whether the or­ga­ni­za­tion suc­ceeds or fails, write up and pub­li­cly pub­lish co­pi­ous notes and ret­ro­spec­tives, both qual­i­ta­tive and quan­ti­ta­tive, on why the or­ga­ni­za­tion suc­ceeded or failed.

My un­der­stand­ing is that this ap­proach has two key differ­ences from the ap­proach em­ployed by Karnofsky and Todd. First, step three re­quires each team to be pro­duc­ing in­for­ma­tion in a very tan­gible way—they ei­ther suc­ceed and demon­strate a suc­cess­ful pro­gram for scale-up or they fail and we learn from their failure. Se­cond, step two could in­clude ad­di­tional se­lec­tion crite­ria, such as be­ing on a list of pri­or­ity pro­grams or gen­er­at­ing in­for­ma­tion rele­vant to iden­ti­fy­ing pri­or­ity pro­grams.

This con­trasts with Todd’s view that thor­ough eval­u­a­tive work is not worth do­ing within the first few years of a new non-profit, since it takes a lot of work to know whether what you’re do­ing is work­ing. This also elab­o­rates on Karnofsky’s view that while you may not “re­quire a strong ev­i­dence base be­fore fund­ing some­thing”, you still should aim to­ward build­ing that ev­i­dence base.

I can­not claim that this pro­cess would prop­erly bal­ance false pos­i­tives and false nega­tives, but it does look good that this pro­cess avoids the dual traps of con­tin­u­ing a pro­gram that ap­pears to work but doesn’t ac­tu­ally work (c.f., PlayPumps and many many many oth­ers) while also not fal­ling into the short-sight­ed­ness that Todd warns us about by be­ing able to fund an early GiveWell, AMF, Char­ity Science Outreach, Giv­ing What We Can, or Google.

Can We Fund the Fu­ture?

A larger con­cern would be whether the pro­cess could risk fal­ling into the nar­row-mind­ed­ness that Karnofsky and Todd warn us about—would we be able to rec­og­nize and fund work with long pay­offs, like the Green Revolu­tion, Peter Singer’s early an­i­mal rights work, LGBTQ rights work in the ’60s, or civil rights work in the ’30s? Cer­tainly you can’t do a ran­dom­ized con­trol­led trial to see whether MIRI is ac­tu­ally re­duc­ing ex­is­ten­tial risk, but would that mean they could never get a grant un­der this in­cu­ba­tion ap­proach?

Ac­count­ing for long pay­offs is pos­si­ble, but would re­quire a lot more do­main ex­per­tise, in­clud­ing a few break­throughs, in how we mea­sure and eval­u­ate char­i­ties. This may in­volve in­vest­ing in more fun­da­men­tal re­search to un­der­stand, e.g. protest­ing, poli­ti­cal in­fluenc­ing, or sci­ence R&D, be­fore mak­ing con­crete-level grants in very long-run ar­eas.

Per­haps in­di­vi­d­ual donors with sharp do­main knowl­edge in a par­tic­u­lar field may feel com­fortable that they can iden­tify hits with­out wait­ing for more fun­da­men­tal re­search. I see that as the best ar­gu­ment for “hits-based giv­ing”. Whether or not mak­ing these type of long-term bets with high do­main knowl­edge would outdo mid-range bets or short-term marginal im­prove­ments is, nat­u­rally, un­clear.

Either way, I’d en­courage trans­par­ent grant­mak­ing with a pro­cess that gen­er­ates as much use­ful in­for­ma­tion for other fu­ture grant­mak­ers. This in­cu­ba­tion pro­cess seems quite promis­ing to me, and I’d love to see it scaled up to ex­pand to other cause ar­eas be­yond global poverty, with the large-scale fund­ing and trans­parency needed to find demon­stra­bly good op­por­tu­ni­ties across many cause ar­eas.