Charity Science 2.5 Year Internal Review and Plans Going Forward

Summary

Char­ity Science has been run­ning for 2.5 years and con­ducted six ex­per­i­ments (with three failures, two suc­cesses and one pend­ing). It has raised an es­ti­mated $300,000 CAD spend­ing about $80,000 CAD in op­er­at­ing costs for GiveWell recom­mended char­i­ties. Our best guess for a ra­tio is 1:4, al­though we in­clude many differ­ent ways of es­ti­mat­ing im­pact in our re­port. We feel as though our biggest mis­takes in­clude (run­ning too cheaply, not hiring soon enough and rely­ing too heav­ily on skil­led EAs). Our plans mov­ing for­ward are for half of our team to con­tinue con­duct­ing ex­per­i­ments and scale up suc­cesses in Char­ity Science and for half of our team to move onto the new pro­ject of Char­ity En­trepreneur­ship.

After 2.5 years of work­ing on Char­ity Science, we felt like it was time to write an in­ter­nal re­view. This post aims to be a broad overview of the work Char­ity Science has done his­tor­i­cally, what we’ve ac­com­plished, to give in­sight as to what hasn’t worked, and to provide some of the ra­tio­nale and con­text for our fu­ture plans.

What this blog post in­cludes:

Core con­cept of Char­ity Science

His­tory and sum­mary of experiments

1) Grants

2) Networking

3) P2P events

4) Skep­tics niche marketing

5) Le­gacy fundraising

6) On­line ads

Results

Ex­pected value

Oper­a­tional benefits

Soft Benefits

Key mistakes

Idea Mistakes

Idea is­sues we are very un­cer­tain on

Ex­e­cu­tion mistakes

Ex­e­cu­tion is­sues we are very un­cer­tain on

Our plan mov­ing forward

New high level project

New experiments

Long term goals

Core con­cept of Char­ity Science

The rea­son we started Char­ity Science was be­cause we thought that there was a strong case that we could in­crease our im­pact if we fundraised more money for the most effec­tive char­i­ties than we spent. There was con­sid­er­able in­ter­est in an or­ga­ni­za­tion try­ing this in the effec­tive al­tru­ism com­mu­nity and it also seemed like it was pos­si­ble it would gen­er­ate con­sid­er­able learn­ing value for the com­mu­nity. You can see more in­for­ma­tion on our core con­cept and key val­ues here.

His­tory and sum­mary of ex­per­i­ments

His­tor­i­cally we have run six fundrais­ing ex­per­i­ments, three of these failed to raise enough money to jus­tify their time and were thus con­sid­ered failures. Two of these were suc­cess­ful enough to con­tinue putting time in and scal­ing up, and one ex­per­i­ment is still pend­ing. Short sum­maries with key in­for­ma­tion on each of these ex­per­i­ments is be­low. Our main suc­cess met­ric is coun­ter­fac­tual money moved, which is ba­si­cally money that would not have been donated to effec­tive char­i­ties with­out Char­ity Sciences in­fluence.

1) Grants

De­scrip­tion: writ­ing grant ap­pli­ca­tions to large pri­vate foun­da­tions and gov­ern­men­tal bod­ies re­quest­ing fund­ing for GiveWell-recom­mended char­i­ties.

Coun­ter­fac­tual money moved: we re­ceived one grant for $10,000 USD and pos­si­bly in­fluenced one grant for $10,000+. We also ac­quired Google Ad­words grants ac­quired for sev­eral or­ga­ni­za­tions. We didn’t count Google Ad­words grants or the in­fluenced grant as money moved.

Time spent: ~12 full-time staff months.

Fur­ther read­ing: our more de­tailed re­view which links to all we have writ­ten about this ex­per­i­ment. We con­sid­ered this ex­per­i­ment a failure.

2) Networking

De­scrip­tion: at­tend­ing mul­ti­ple con­fer­ences, events and so­cial gath­er­ings of peo­ple po­ten­tially in­ter­ested in effec­tive char­i­ties.

Coun­ter­fac­tual money moved: ~$1,000 - ~$10,000 de­pend­ing largely on whether effects of net­work­ing on peer-to-peer (P2P) fundrais­ing are counted here or in the P2P ex­per­i­ment.

Time spent: ~7 full-time staff months

Fur­ther read­ing: our more de­tailed re­view. We con­sid­ered this ex­per­i­ment a failure.

3) P2P events

De­scrip­tion: en­courag­ing and sup­port­ing in­ter­ested in­di­vi­d­u­als to run a per­sonal fundrais­ing event rais­ing money from their friends, fam­ily and col­leagues.

Coun­ter­fac­tual money moved: About $200,000 spread over 4 cam­paigns of Christ­mas 2014 (~$88,000), birth­days (~$25,000), (~$25,000) Liv­ing on Less and the (~$12,000) Char­ity Science walk. Christ­mas 2015 (~$50,000)

Time spent: ~12 full-time staff months

Fur­ther read­ing: our more de­tailed re­view. We con­sid­ered this ex­per­i­ment a suc­cess.

4) Skep­tics niche marketing

De­scrip­tion: Tar­get­ing a spe­cific niche with a cus­tom made web­site and tar­geted out­reach. Skep­tics were cho­sen due to their in­ter­est in effec­tive char­i­ties and the num­ber of skep­tics who have be­come Effec­tive Altru­ists.

Coun­ter­fac­tual money moved: ~$0

Time spent: ~6 full-time staff months

Fu­ture read­ing: our plan here, in­ter­nal write up here. We con­sid­ered this ex­per­i­ment a failure.

5) Le­gacy fundraising

De­scrip­tion: pro­mot­ing and pro­vid­ing es­tate plan­ning ser­vices for those in­ter­ested in be­queath­ing to effec­tive char­i­ties.

Coun­ter­fac­tual money moved: Our money moved is an ex­pected value as it’s based on planned dona­tions as op­posed to ex­e­cuted es­tates.Thus, even though the origi­nal sum is hun­dreds of thou­sands of dol­lars, it has been dis­counted based on time dis­counts, pes­simistic at­tri­tion rates, pro­bate rates and coun­ter­fac­tu­als. Non-time dis­counted (but ad­justed for at­tri­tion rates, pro­bate rates and coun­ter­fac­tu­als) our es­ti­mates are 7k-101k. Add an ag­gres­sive dis­count rate of 50% re­duc­tion in value per ev­ery 10 years past be­fore the money is donated. This changes our es­ti­mates for money moved to 3k − 35k. Over­all, we are very pleased with these num­bers as our ser­vice was only up for an ex­tremely short time and was not ag­gres­sively mar­keted yet.

Time spent: ~5 full time staff months

Fur­ther read­ing: We have not writ­ten ex­ten­sively about this ex­per­i­ment yet and this ex­per­i­ment is still run­ning. We con­sid­ered this ex­per­i­ment a suc­cess.

6) On­line ads

De­scrip­tion: Us­ing free Google Ad­words to pro­mote our fundrais­ing and out­reach ac­tivi­ties.

Coun­ter­fac­tual money moved: ~$500

Time spent: ~1 full time staff month (al­though we plan to spend more in the fu­ture)

Fu­ture read­ing: Our plan is here. We have not writ­ten ex­ten­sively about this ex­per­i­ment be­cause it is still in progress.

We con­sider our P2P and legacy ex­per­i­ments suc­cess­ful (green) and our grants, net­work­ing and skep­tics niche mar­ket­ing ex­per­i­ments un­suc­cess­ful (red). On­line ads is still cur­rently be­ing ex­per­i­mented with (yel­low). We also spent a large amount of time on op­er­a­tions and re­search ac­tivi­ties (blue). It is worth not­ing we are not sure which cam­paign to at­tribute a con­sid­er­able amount of money that has been moved through our web­site. Some of these cam­paigns may be more suc­cess­ful than our es­ti­mates would sug­gest.

Results

Ex­pected value


We gen­er­ally view his­tor­i­cal ra­tios as the most re­li­able met­ric of suc­cess. Th­ese num­bers are less sub­jec­tive than the other num­bers that can be used to es­ti­mate suc­cess. We es­ti­mate that af­ter about 2.5 years of Char­ity Science run­ning we will have raised ~$300k.

It was re­cently sug­gested that we also should es­ti­mate the long term fundrais­ing effects of our ac­tivi­ties. One way to look at im­pact is to ask how much im­pact Char­ity Science would have had up to this point if we im­me­di­ately with­drew all founder sup­port and staff. For ex­am­ple, P2P events or our legacy ser­vice will likely con­tinue for many years with some pos­i­tive money moved even if we were to close Char­ity Science to­mor­row. We would guess that our yearly re­turns if Char­ity Science was run by vol­un­teers would be close to an av­er­age of $100k raised per year for 3 years and 50k for an­other 3 years than 10k for an ad­di­tional 3 years be­yond that. In these es­ti­mates we have at­tempted to ac­count for value drift, donor fa­tigue and de­creased coun­ter­fac­tual im­pact. This to­tals to around $480k of ex­pected value over the next ten years. To come to this num­ber, we used much more pes­simistic es­ti­mates than we have seen other or­ga­ni­za­tions use, but feel as though these num­bers are re­al­is­tic. We are un­sure of how to weight this num­ber al­though are con­fi­dent we would weight it lower than our his­tor­i­cal money-moved num­bers. If this num­ber was taken at face value, it could be our ex­pected value is $780,000 (tak­ing the lower bound from our his­tor­i­cal num­bers and our pes­simistic as­sump­tions on fu­ture effects).

We have spent just over $80k in the last 2.5 years of di­rect staff and pro­gram costs. Our full bud­get de­tails can be seen here.

A sec­ond fac­tor, our com­mu­nity has ex­pressed an in­ter­est in staff coun­ter­fac­tu­als. Although the spe­cific de­tails of staff’s backup plans are con­fi­den­tial, the co-founder es­ti­mates of the to­tal coun­ter­fac­tual im­pact of Char­ity Science em­ploy­ees/​vol­un­teers are close to ~$500k at the up­per bound for all cur­rent com­bined staff/​vol­un­teer coun­ter­fac­tu­als over the last 2.5 years. This num­ber is ex­tremely soft and based on limited ev­i­dence. Us­ing this es­ti­mate one could up the coun­ter­fac­tu­als in­cluded costs of Char­ity Science to $580k


We have only listed the pes­simistic as­sump­tions, bounds and es­ti­mates through­out this re­port as we find the most op­ti­mistic num­bers listed tend to be the ones peo­ple re­fer to in con­ver­sa­tions and when con­sid­er­ing im­pact, re­gard­less of the qual­ifi­ca­tions added be­fore­hand. We also con­sis­tently find our way of mea­sur­ing and look­ing at im­pact to be con­sis­tently more skep­ti­cal and pes­simistic than other ways of count­ing and want to be cau­tious of the urge to post the most flat­ter­ing num­bers pos­si­ble.

Spe­cific ra­tios vary hugely de­pend­ing on what fac­tors to in­clude, how to weight each fac­tor, and how skep­ti­cal one tends to be of long term im­pact. Us­ing the num­bers listed above the range can be be­tween 1:11 Char­ity Science life­time re­turns to 1:0.5 Char­ity Science life­time re­turns. For fun we also tried to use iden­ti­cal dis­count num­bers similar to some other calcu­la­tion meth­ods we have seen and we got a ra­tio of 1:66-1:128. Our ra­tios can also change con­sid­er­ably if iso­lated by ex­per­i­ment (rang­ing from 1:16 to 1:0). Tak­ing all fac­tors into ac­count yields a ra­tio of 1:1.3 coun­ter­fac­tu­ally ad­justed dol­lars per dol­lar spent. Tak­ing only harder, more ev­i­dence-based fac­tors into ac­count yields a ra­tio of about 1:4. We feel as though these es­ti­mates are re­al­is­tic ex­pec­ta­tions of what one would ex­pect in the fundrais­ing com­mu­nity. We would be con­cerned that our mea­sure­ments might be off if our ra­tios came back as sev­eral times higher than what other or­ga­ni­za­tions re­port (page 7). This fol­lows from the com­mon sense rule that ex­traor­di­nary claims re­quire ex­traor­di­nary ev­i­dence. Fi­nally, these ra­tios do not take into ac­count op­er­a­tional benefits or soft benefits, which are de­scribed more in de­tail be­low.

Oper­a­tional benefits

We have found there have been some or­ga­ni­za­tional benefits that we feel are sig­nifi­cant enough to be worth men­tion­ing in this re­port. The first, and most clearly benefi­cial one, is the tax benefits we have been able to offer. Cana­dian donors can now sup­port all of Givewell’s recom­mended char­i­ties while re­ceiv­ing tax benefits with­out be­ing charged the Tides fees or other similar ser­vices charge. This has been pos­si­ble by hav­ing a reg­istered Cana­dian foun­da­tion and the ex­tremely helpful col­lab­o­ra­tion of EA groups like EACH, GWWC and the Com­mon­wealth mar­ket. We do not have an es­ti­mate of how much ad­di­tional money was donated to effec­tive char­i­ties be­cause of this benefit, or if there were other benefits to donors, like get­ting to se­lect from more char­i­ties. We would ex­pect this is seen as a large benefit by some.

Regis­ter­ing as a foun­da­tion also has other benefits like al­low­ing other pro­jects to grow out from un­der the same tax sta­tus and or­ga­ni­za­tional brand. We feel as though this will aid our up­com­ing pro­ject and pos­si­bly other pro­jects in the fu­ture.

Soft Benefits

The fi­nal benefits are cat­e­go­rized as “soft benefits”. Although we are un­sure how to weigh these, many of our donors and ad­vi­sors think these are some of the largest benefits of Char­ity Science.

The first of these is per­sonal learn­ing value. Skills like fundrais­ing, mar­ket­ing, com­mu­ni­ca­tion and out­reach are all very use­ful and some­what less com­mon in the more ev­i­dence-based com­mu­ni­ties of char­ity. We feel our per­sonal skills have mas­sively im­proved, and our con­fi­dence and abil­ity to raise funds for char­i­ties is vastly stronger than when we started. We do feel as though these per­sonal benefits have started to ta­per off as we have spent more time in the field. Another set of skills we have de­vel­oped are gen­eral en­trepreneur­ship and or­ga­ni­za­tional skills. We think these skills have im­proved and will con­tinue to im­prove for sev­eral years. They should prove valuable in al­most any field, but par­tic­u­larly in the fields that the co-founders plan on go­ing into.

The sec­ond benefit is com­mu­nal learn­ing value. We have made great efforts to be ex­tremely trans­par­ent in our or­ga­ni­za­tional ac­tivi­ties. A big part of why is be­cause we want to al­low oth­ers to learn from our ex­pe­riences, mis­takes and strate­gies so that the larger effec­tive char­ity com­mu­nity can benefit. We have seen var­i­ous soft­wares, habits and ap­proaches spread through­out this com­mu­nity, al­though it is very difficult to draw clear causal lines back to Char­ity Science.

The fi­nal and per­haps largest benefit (in my view) is the team we have built. Over time Char­ity Science has grown as an or­ga­ni­za­tion and now has enough ca­pac­ity to have staff run­ning two large scale pro­jects (more on our new pro­ject at the end of this re­port). We think the benefits of hav­ing this large and skil­led team will have sig­nifi­cant and far-reach­ing pos­i­tive fu­ture effects.

It’s difficult to weigh all these benefits among more clearly quan­tifi­able ones, but over­all the co-founders feel that found­ing Char­ity Science was one of the high­est im­pact things they could do with their time.

Key mistakes

Every or­ga­ni­za­tion makes mis­takes and in our ex­pe­rience the best or­ga­ni­za­tions ad­mit these mis­takes pub­li­cly. This is not in­tended to be an ex­haus­tive re­view of ev­ery mis­take we’ve made, but rather what our in­ter­nal staff sees as the most sig­nifi­cant ones we have made his­tor­i­cally and the largest con­sid­er­a­tions we are un­sure we are mak­ing the right call on. Our self-crit­i­cism broadly falls into the two cat­e­gories of “con­cep­tual” and “ex­e­cu­tion”. Th­ese crit­i­cisms were ini­tially made to as­sist an ex­ter­nal re­viewer who was go­ing to re­view Char­ity Science but then be­came too busy.

Con­cep­tual Mistakes

The first con­cep­tual mis­take was the as­sump­tion that.fundrais­ing on be­half of di­rect-level char­i­ties is prefer­able to char­i­ties sim­ply fundrais­ing for them­selves. We have found that fundrais­ing is more difficult if you are not the di­rect char­ity. There are a num­ber of pos­si­ble rea­sons for this, in­clud­ing:

  1. The fundrais­ing meth­ods available to ex­ter­nal groups like us are much more limited than for the effec­tive char­ity it­self. For ex­am­ple many grants we ap­plied for needed spe­cific and de­tailed in­for­ma­tion about the founder that was hard to give as an ex­ter­nal or­ga­ni­za­tion.

  2. Peo­ple get at­tached to the peo­ple they are in­ter­act­ing with per­son­ally so if they in­ter­act with Char­ity Science staff, they tended to like and want to fund Char­ity Science in­stead of the char­i­ties we recom­mend.

  3. There are added lo­gis­ti­cal com­plex­ities of be­ing a sep­a­rate or­ga­ni­za­tion such as hav­ing to do sep­a­rate bud­get­ing, dona­tion re­ceipt­ing and re-grant­ing. Ad­di­tion­ally our model was con­fus­ing to many peo­ple which made fundrais­ing from them harder.


The sec­ond mis­take was the be­lief that sci­ence could be eas­ily and effec­tively ap­plied to fundrais­ing. We con­sis­tently found the re­search weaker and less clear cut than we were ex­pect­ing. Even when talk­ing to ex­perts or paid con­sul­tants, the amount of em­piri­cal data or clear ex­pert opinion was much weaker than ex­pected. We found we gained much more knowl­edge from ex­e­cut­ing small-scale ex­per­i­ments than we did from on­line re­search or talk­ing to ex­perts about differ­ent strate­gies.

Con­cep­tual is­sues we are very un­cer­tain on

One is­sue is the pos­si­bil­ity that bet­ter re­search is more likely to bring in more fund­ing than ex­plic­itly fo­cus­ing on fundrais­ing or out­reach. This is some­what sup­ported by the much larger amount of fund­ing that an or­ga­ni­za­tion like GW has moved (com­pared to Char­ity Science). We are cur­rently very un­cer­tain about whether this is true or not and can see a strong case for both view­points.


Another is­sue is that fundrais­ing may be bet­ter done in a much broader way. For in­stance, mov­ing peo­ple from lo­cal char­i­ties to global char­i­ties in­stead of from one spe­cific char­ity to an­other. This might be more effec­tive or more per­sua­sive to peo­ple. We feel as though this comes down to spec­u­la­tive guesses re­gard­ing both the mar­ketabil­ity of broad ver­sus spe­cific ap­proaches and com­par­i­sons be­tween effec­tive char­i­ties and global or lo­cal ones. We do not have con­fi­dent es­ti­mates on ei­ther of these is­sues.

Fundrais­ing may be far less effec­tive than stan­dard money-moved ra­tios might make it seem be­cause the differ­ence be­tween the char­i­ties that peo­ple switch be­tween may not be that large (e.g. Ox­fam to AMF). This is an­other is­sue we are un­cer­tain about, al­though more cer­tain than the above two. This might mean earn­ing to give might be bet­ter even if it’s a smaller amount of money moved per per­son, it is money that oth­er­wise would not go to any char­ity.

Fi­nally, pos­si­bly our largest con­cern is how our time would be best spent coun­ter­fac­tu­ally. We are ex­tremely un­cer­tain this is the case (al­though we were more cer­tain when we founded Char­ity Science). It seems plau­si­ble that it would be bet­ter for highly ded­i­cated EA’s to work on found­ing and im­prov­ing char­i­ties with the in­ten­tion of hav­ing gov­ern­ments or very large fun­ders cover the costs. This is par­tic­u­larly the case with large effec­tive foun­da­tions such as Good Ven­tures and the Bill and Melinda Gates Foun­da­tion, who seem like they would would be keen to fund ex­tremely effec­tive char­i­ties. That be­ing said we do think mov­ing more money to great char­i­ties (via earn­ing to give or fundrais­ing) is a great fit for most EAs, and it’s rare to have the right ded­i­ca­tion level and skill set for the former op­tion to be a good fit.

Ex­e­cu­tion mistakes

Our first mis­take was run­ning our or­gani­sa­tion too eco­nom­i­cally. We spent too much time try­ing to save money in ways that, in hind­sight, were not an effec­tive use of our time. We think this was some­what a re­ac­tion to other or­ga­ni­za­tions we had seen mak­ing the op­po­site mis­take. This might also have been tied to our fo­cus on ra­tios in­stead of net money moved.

Our sec­ond large mis­take was not hiring soon enough. Be­cause we wanted to run at a low bud­get we were averse to hiring staff. How­ever, we have found hiring staff to be hugely benefi­cial and over­all has made our or­ga­ni­za­tion much more effec­tive. We likely could have run all of our cur­rent ex­per­i­ments over half the time if we had hired ag­gres­sively from day one.

Our third mis­take is rely­ing on the gen­er­ally high skill of the co-founders and other highly in­volved EAs. We think that Char­ity Science likely could have been run by a sin­gle highly in­volved EA and a team of peo­ple new to EA con­cepts or em­ploy­ees from more stan­dard non­profit cir­cles. This choice was likely af­fected by our wor­ries about cost/​bud­get and our un­der­weight­ing of staff coun­ter­fac­tu­als.

Our fourth mis­take was an un­der­uti­liza­tion of more tra­di­tional busi­ness prac­tices (e.g. clear hi­er­ar­chies, more spe­cial­iza­tion, clearer del­e­ga­tion sys­tems). We feel this is some­what in­evitable given the youth of our char­ity, but we still feel we could have ex­e­cuted bet­ter by tak­ing a more tra­di­tional ap­proach be­fore ex­per­i­ment­ing in these ar­eas.

Ex­e­cu­tion is­sues we are very un­cer­tain on

We have some con­cerns about count­ing the coun­ter­fac­tu­als of dona­tions from EAs. We have tried ex­tremely hard to be self-crit­i­cal and in some peo­ple’s view go too far in un­der­count­ing our im­pact. We have thought a lot about this pro­cess, but are still not fully con­fi­dent whether we over­count or un­der­count our own im­pact. (An ex­am­ple of how we do this)

Prior to Char­ity Science, our team had a low level of pre­vi­ous fundrais­ing and out­reach ex­pe­rience. We are un­sure about the differ­ence this would have made, as our ad­vi­sors and con­sul­tants did not have nearly as much in­sight as we were ex­pect­ing. Even the con­sul­tants with im­pres­sive track records (e.g. they say they have raised mil­lions a year) seemed much less im­pres­sive when we dug deeper–of­ten they only raised a few hun­dred thou­sand marginal coun­ter­fac­tual funds and the rest is just a build up of the donor base over the decades of ex­is­tence of the char­ity.

We are un­sure if we fo­cused too much on quick out­put or mea­surable ac­tivi­ties. Cur­rently our ex­per­i­ments have suc­ceeded at quite a high rate (33%) but we have not had any ex­cep­tion­ally large wins (e.g. a 5 mil­lion dol­lar grant). It’s pos­si­ble that tak­ing a riskier ap­proach to fundrais­ing would yield more re­turns in the long run.

Our plan mov­ing forward

New high level project

The biggest change to Char­ity Science is that 50% of our cur­rent team (in­clud­ing the ma­jor­ity of co-founder time) is mov­ing to the new pro­ject of Char­ity En­trepreneur­ship. We feel that this is an ex­tremely high im­pact pro­ject and al­lows us to move closer to the ideal model of Char­ity Science (with less re­li­ance on founders and highly ex­pe­rienced and in­volved EAs). This will largely lower the coun­ter­fac­tual cost of Char­ity Science in the fu­ture. This ex­per­i­men­tal pro­ject may also an­swer some of the un­cer­tain­ties we have around fundrais­ing as an ex­ter­nal or­ga­ni­za­tion ver­sus be­ing the di­rect char­ity. Our full plans for Char­ity En­trepreneur­ship can be found here and some more ra­tio­nale to why this might be a high im­pact pro­ject can be found on the EA fo­rum, GiveWell and 80k.

New experiments

Over the next 6 months we have a new batch of ex­per­i­ments that we are con­sid­er­ing with a stronger fo­cus on break­ing out of the EA move­ment and reach­ing new com­mu­ni­ties. We ex­pect to run 2-4 ex­per­i­ments over the year as well as main­tain­ing past suc­cesses. Th­ese new ex­per­i­ments are face-to-face fundrais­ing, fur­ther on­line ads test­ing, donor stew­ard­ship and helping vol­un­teers run small scale ex­per­i­ments. You can see all of the ar­eas we are con­sid­er­ing here.

Long term goals

Our long term goals are to con­tinue to ex­per­i­ment for ap­prox­i­mately the next year and to run suc­cess­ful ex­per­i­ments in­definitely. We ex­pect to hit diminish­ing re­turns on fundrais­ing ex­per­i­ments some­where in 2016 or 2017 and will change our ma­jor fo­cus to scal­ing up suc­cess or scal­ing down the re­quired staff time. We ex­pect to lower Char­ity Science’s coun­ter­fac­tual cost sub­stan­tially in 2016 and to raise its net money moved slowly over time. Our plan is for Char­ity Science to take min­i­mal co-founder time while con­tin­u­ing to gen­er­ate pas­sive im­pact.

We see four pos­si­ble very long term plans for Char­ity Science a growth plan, a sta­ble plan, vol­un­teer based plan and an in­te­gra­tion (into a larger di­rect char­ity) based plan. Th­ese each have pros and cons and we feel like we will be bet­ter able to eval­u­ate these op­tions mid­dle or end of 2016.