On increasing and decreasing (marginal) returns:
I see that you said “claiming that expected returns are normally diminishing is compatible with expecting that true returns increase over some intervals. I think that true returns often do increase over some intervals, but that returns generally decrease in expectation.”
I wasn’t sure why this would be true in a model that describes the organization’s behavior, so I spent some time thinking it through. Here is a way to reconcile increasing returns and decreasing expected returns, with a graph. Note that when talking about “funding” here (and the x-axis of the graph) I mean “funding the organization will receive over the next planning period, i.e. calendar year”, and assume there’s no uncertainty over funding received, same as in Max’s model.
I think it’s reasonable to assume that “increasing returns” in organization’s impact often come from cases of “lumpy investments”, i.e. things with high impact and high fixed costs. In this case nothing would happen until a certain level of funding is reached, and at that point there is a discrete jump in impact. For the sake of the argument let’s assume that everything the organization does has this nature (we’ll relax this later). So you’d expect the true returns function to be a step function (see the black curve on graph).
How does the organization makes decision? First, let’s assume that these “lumpy investments” (call them “projects”) aren’t actually 0 or 1; rather, the closer the level of funding is to the “required” level, the more likely the project will happen (e.g. maybe AMF is trying to place an order for bed nets and the minimum requirement is 1000 nets, but it’s possible that they can convince the supplier to make an order of 900 nets with probability less than 1). For simplicity let’s assume the probability grows linearly (we’ll relax this later). Then the expected returns is actually the red piecewise linear function in the graph. Note that overall the marginal returns are still weakly diminishing (but they are constant within each project) because given the red expected returns function the organization would choose to first do the project with the highest marginal return (i.e. slope), then the second highest, etc.
Note: We assume the probability grows linearly. If we relax this assumption, things get more complicated. I illustrate the case where probabilities grow in a convex way within each project with the ugly green curves (note that this also covers the case with no uncertainty in the project happening or not, but rather the project has a “continuous” nature and increasing marginal returns). It’s true that you cannot call the whole thing concave (and I don’t know if mathematicians have a word to describe something like this). But from the perspective of a donor who, IN ADDITION to the model here that assumes certainty in funding levels, has uncertainty over how much funding the organization has, the “expected-expected” returns function they face (with expectation over funding level and impact) would probably be closer to the earlier piecewise linear thing, or concave. If the probabilities grow in some weird ways that are not completely convex (note that this also covers the case with no uncertainty in the project happening or not, but rather the project has a “continuous” nature and weirdly shaped, non-convex marginal returns), things may get more complicated (e.g. switching projects half way may happen if the organization always spends the next dollar on the next thing with highest marginal return) -- maybe we should abandon such possibilities since they are unintuitive.
Note: If the organization does some projects that look more like linear in the relationship between impact and funding, 1) we can still use the red piecewise linear graph, and organizations will still start with projects with the highest slopes; 2) at a fine level things are still discrete so we’ll be back to (mini) step functions.
Note: We also assumed the only uncertainty here is whether a project would happen at a funding level less than “required”. There could also be uncertainty over impact, conditional the project happening—this is not in our model, but my guess is it shouldn’t change the main results much (of course it might depend on the shape of the new layer of uncertainty, and I haven’t thought about it carefully).
All of the above is essentially based on the old idea that organizations do highest returns things first. The main addition is to look at a model where there are discrete projects (with elements of increasing returns) and still arrive at the same general conclusion.
I don’t know how many people find this useful, but I was very confused by this issue (and said some incoherent things in my earlier comments, which I’ve delete to avoid confusing people), and found that I had to think through what the organization actually does in the case of lumpy investments.
Other important issues that are related but out of the scope of this discussion include how organizations and donors act under uncertainty over donation to be received by the organization.
Tom and Peter:
For an early stage charity like ACE it seems that capacity building is indeed a very important consideration (related to Ben Todd’s point about the growth approach). E.g. it would allow them to move much more money later, and at the moment moving not that much money is a reason why they don’t look so good in our model. Unfortunately we aren’t able to incorporate this in our quantitative model (IMO another reason to look beyond quantitative models for decision making at this point, but people may have ways of incorporating it quantitatively—it won’t be hard to make a theoretical model of R&D, but fitting it empirically will be the big challenge).
On (i), Open Phil’s Lewis Bollard’s recommendation and ACE’s own plan make it look like capacity building is something they try to do.
On (ii) and (iii), these have been true for GiveWell historically. E.g. on (iii), last year they added quite a few top charities. But I don’t know ACE enough to say if they will grow in the way GiveWell did.
Max, thanks for the post!
For someone like GiveWell that spends a lot of time investigating charities, they may have enough information about the charity’s budget to tell when there is (something similar to) a discrete jump in the derivative of the returns function. E.g. the way they talk about “capacity-relevant funding” and “execution funding” in the post you linked to (“incentive funding” is for a completely different purpose that has no direct relationship with returns).
Also, to fix ideas it helps to think what we represent by the funding axis on the impact against funding graph, i.e. returns function. Is the function specifying the relationship between total impact, and total funding the charity expects to receive for a time period (i.e. next year), or we are looking within a time period and plotting what the charity does as (unexpected) new money comes in? In the latter case, diminishing returns seems most likely. In the former case, increasing returns is possible (but diminishing returns is as well).
Ben Todd has written about increasing returns in small organizations here. I wrote here that “Whether returns are increasing or decreasing in additional funding depends on how the funding is received. Expecting a large chunk of funding (either in the form of receiving such amounts at once, or even expecting a total large amount received in small chunks if there is no lumpy investment or borrowing constraint) could enable an organization to do more risk taking, while getting unanticipated small amounts of funding at a time—even if the total adds up to more—will probably just lead the organization to use the marginal dollar to “fund the activity with the lowest (estimated) cost-effectiveness”. … The scenario Ben Todd has in mind probably applies more when a large funder is considering how much to give to an organization. This may be another argument to enter donor lottery or donate through the EA fund: giving a large and certain amount of donations to a small organization enables them to plan ahead for more risky but growth enhancing strategies, hence could be more valuable than uncoordinated small amounts even if the latter add up to the same total (because the latter may be less certain). … This mechanism is articulated in “5.2 The funding uncertainty problem” on this page about the EA fund.” (There are probably some analogous economic model of firm investment under liquidity constraint and uncertainty, but I don’t have one on the top of my head.)
In practice it may not be a big deal: even if the charity receives random small amounts of money during the year, it is probably at least as good as receiving the total amount all at once at the beginning of next year when they do the next round of planning. But for small organisations where earlier growth is much better, it could be much more preferable to have small amounts of donations be coordinated and committed at the same time to help with more ambitious planning and growth. (Of course we are assuming the charity is borrowing constrained; otherwise if earlier growth is much better they’d borrow to achieve it and repay with later donation. Also, if the market is efficient and earlier growth is really much better, then some donors should capture the opportunity … but of course market may not be!)
My personal take on the issue is that, the better we understand how the updating works (including how to select the prior), the more seriously we should take the results. Currently we don’t seem to have a good understanding (e.g. see Dickens’ discussion: the way of selecting the median based on Give Directly seems reasonable, but there doesn’t seem to be a principled way of selecting the variance, and this seems to be the best effort at it so far), so these updating exercises can be used as heuristics but the results are not to be taken too seriously, and certainly not literally (together with the reason that input values are so speculative in some cases).
This is just my personal view and certainly many people disagree. E.g. my team decided to use the results of Bayesian updating to decide on the grant recipient.
My experience with the project lead me to be not very positive that it’s worth investing too much in improving this quantitative approach for the sake of decision making, if one could instead spend time on gathering qualitative information (or even quantitative information that don’t fit neatly in the framework of cost-effectiveness calculations or updating) that could be much more informative for decision making. This is along the lines of this post and seems to also fit the current approach of the Open Philanthropy Project (of utilizing qualitative evidence rather than relying on quantitative estimates). Of course this is all based on the current state of such quantitative modeling, e.g. how little we understand how updating works as well as how to select speculative inputs for the quantitative models (and my judgment about how hard it would be to try to improve on these fronts). There could be a drastically better version of such quantitative prioritization that I haven’t been able to imagine.
It could be very valuable to construct a quantitative model (or parts of one), think about the inputs and their values, etc., for reasons explained here. E.g. The MIRI model (in particular some inputs by Paul Christiano; see here) has really helped me realize the importance of AI safety. So does the “astronomical waste” argument, which gives one a sense of the scale even if one doesn’t take the numbers literally. Still, when I make a decision of whether to donate to MIRI I wouldn’t rely on a quantitative model (at least one like what I built) and would instead put a lot of weight on qualitative evidence that is likely impossible (for us yet) to model quantitatively.
Peter, indeed your point #2 about uncertainty is what I discuss in the last point of “2) Outcome measures”, under “Model limitations”. I argued in a handwaving way that because 80K still causes some lower risk and lower return global health type interventions—which our aggregation model seems to favor, probably due to the Bayesian prior—it will probably still beat MIRI that focuses exclusively on high risk, high return things that the model seems to penalize. But yes we should have modeled it in this way.
Robin, for what you quoted about increasing returns I was thinking only in the case of labor. Overall you are right that, if the organization has been maximizing cost-effectiveness, then they probably would have used the money they had before reaching fundraising targets in a way that makes it more cost-effective than money coming in later (assuming they are more certain about the amount of money up to fundraising target, and less certain about money coming in after that).
Something that will complicate the effects is that money given to people may increase not only consumption today but also consumption tomorrow through investment. This could be investments in physical capital (e.g. iron roof, livestocks) or human capital (e.g. health and education). Most of the time when people are given money, some will be consumed and some saved/invested (and consumption itself could have investment effects too, if better nutrition improves ability to work/learn), e.g. see Give Directly recipients.
This is relevant if we think that, for instance, poor people in Kenyan villages have more profitable investment opportunities than poor people in the US, for the cash they receive—which is probably the case, e.g. there are many more opportunities to start small businesses in Kenyan villages (or higher returns to improving nutrition because they start at such a low level, though I remember “Poor Economics” says there’s not much of evidence for a nutrition-based poverty trap, so probably not). In that case the benefits of giving cash to poor Kenyans (relative to giving to poor Americans) is further amplified. In fact in GiveWell’s cost-effectiveness calculation for Give Directly, future increase in consumption is responsible for a substantial fraction of the effect (even with discounting) if we assume some persistence in investment returns (even if it’s not compounded).
Ben, to recap a bit what people have said: working as a software engineer at an EA organization
may not be the most technically challenging/engaging job
may not be great for future career development
may not pay much
This probably applies more to EA organizations like CEA and 80,000 Hours. Give Directly may be different since you probably work with Mpesa, similar to Wave; and maybe New Incentives too since they do conditional cash transfers.
And Wave is basically like a regular tech company in the above aspects (and probably better because it’s a startup hence work could be more challenging and interesting than say at Google). They pay less than Google. So it’s a good example that when you have to make sacrifices in 1 dimension, out of the 3 I mentioned above, you can still find really good EA-type software engineers to join.
But for EA organizations like CEA and 80,000 Hours, you need to make sacrifices in all 3 dimensions (and pay is probably less than Wave) -- no wonder it’s harder.
That being said, there’s no reason it can’t work. Just think about people doing similar jobs to the IT positions you want to hire for at CEA/80k—there are plenty of people doing them at other companies or organizations. In this sense, the EA organizations may pay less than the alternative but offers the opportunity to work for an EA organization, so the EA-types among these people would be attracted to such jobs, like EA-type Google engineers would be attracted to Wave.
If the job isn’t the most technically challenging/engaging, you probably shouldn’t be looking for people who value these too much, since they already need to take a pay cut which is a sacrifice even for EA-types, and asking people to make sacrifices on more than one dimension makes it harder to attract them. Look for people who are EA-types working in similar jobs in non-EA places, or people who would be at least indifferent between working these jobs at a non-EA org with lower pay and working these jobs at another place with higher pay.
(But maybe the jobs ARE technically challenging/engaging and great for future career prospect.. I don’t know, you should probably ask actual engineers for their views on these!)
Something relevant: how the Trump presidency can affect global health http://www.economist.com/news/united-states/21715736-global-gag-rule-likely-hit-fight-against-hiv-aids-policy-intended-cut
Wave is really good! (I use it) Another thing one can do is to work for some mobile money company in a developing country to design products that benefit the poor (e.g. saving, credit, that I mention in the other post), like the American guys I met in Myanmar’s Wave Money (but they are still early stage and has many challenges before having an impact). (Not suggesting you should do it though—involves moving to a developing country etc., and could be much less likely to succeed due to regulations etc.). BTW this is the mobile credit scoring company I had in mind: http://tala.co/.
I just got back from Myanmar and I talked with some people running Wave Money (one of the mobile money companies in Myanmar, and the only licensed one so far; not related to the Wave that Jeff mentioned which sends money to Africa).
Getting people to adopt could be a big challenge, depending on the country. In Kenya, the anecdotal story of why mobile money took off so quickly is 1) the need to send remittances, 2) preexisting methods for this being not very good for various reasons (insecurity is one); some also argue that Safaricom’s unusually high market share in the country played a role in speeding up adoption through bundling of services + network effects (telecommunication markets in other countries seem more fragmented). Mobile money has not taken off in some countries, e.g. Nigeria (this article argues it’s due to regulation https://iea.org.uk/blog/why-mobile-money-transformed-kenya-failed-to-take-in-nigeria). In Myanmar it remains an open question: the traditional hundi system for remittances works well for most purposes (being cheap, reliable, and not too slow), which may hinder adoption of mobile money.
Other potential functions of mobile money: other than through remittances (which is what Suri’s paper is estimating), it can also help poor populations by
Saving: providing a safe place to save. This may be important as many poor people seem “savings constrained” (e.g. see https://www.econ.uzh.ch/dam/jcr:5f6e818a-ad07-466a-8962-fdc77bb1dfc2/casaburi_macchiavello_dairy_20160731.pdf, http://www.simonrquinn.com/research/TwoSidesOfTheSameRupee.pdf) and bank are either scarce or expensive in many rural places—although it might be hard to convince people to adopt mobile money just for saving purposes.
Credit: e.g. the Mpesa-based mobile loan Mshwari. There are some startups (including one in Kenya, whose name I forgot) that creates credit scores for people based on their mobile money transaction history, mobile phone records etc. (Mshwari does a version of this but doesn’t seem very sophisticated; the startups probably use more “big data”) which may help the poor access credit in a way that’s much cheaper and sustainable than the traditional microfinance model. (In Myanmar I know one startup trying to do this and giving out small amounts of loan for shorter periods, basically competing with money lenders in slums—seems hard but could be really good if they succeed; they are quite early stage now.)
Sending government benefits, e.g. India is considering introducing universal basic income, and already have biometric identification for most citizens, but one of the remaining barriers is the scarcity of bank branches in rural places. If each village has a mobile money agent things would be much easier (and this has implications not only for poverty reduction but maybe also curbing corruption and improving local governance etc.).
Hi carneades, in reply I just want to make 2 general points here:
Many things need to be done in the developing world, e.g.the ones you mentioned: protecting people against malaria, creating jobs, improving the quality of governments… The most effective intervention for one purpose could be not very useful for another, but that’s still okay because it would be better than trying to do something that serves multiple purposes but is ineffective in all of them. (e.g. for protecting people against malaria, the most effective intervention could be distributing bed nets for free; for job creating, it could be improving the infrastructure and legal system; for improving the quality of governments it could be educating voters—though for this to affect government provision of bed nets down the road would take many years and if we solely rely on this to protect people against malaria we would miss the opportunity to save many lives now compared to using the most effective intervention on this)
By saying the studies are good I mean the researchers take into account that respondents may not report the truth and seem to manage to find out the truth despite that. This is not just because I trust these famous economists to do a good job or I know some of them personally and know that they care about doing a good job, but I also find evidence from the papers. For instance in the Cohen and Dupas paper on free distribution of bed nets, it says (on p14, under “III.C Data”) “During the home visits, respondents were asked to show the net, whether they had started using it, and who was sleeping under it. Surveyors checked to see whether the net had been taken out of the packaging, whether it was hanging, and the condition of the net.” In the Haushofer and Shapiro paper on Give Directly, on p28 they talk about potential desirability bias on alcohol and tobacco and how they address it, and on p32 they mention assets including metal roofs and livestocks which surveyors could easily check (I don’t think they mentioned surveyors checking this but the study is done by IPA in the same region I work in Kenya so I imagine it should have the high standard of work done by this organization and surveyors should check things when they can). In general economists don’t like to rely on survey data precisely because people may lie, and in developing countries when this is often inevitable we try hard to get around the problem and mention how we address them in papers (otherwise you would get a lot of questioning from presentation audience, referees, etc. so there’s plenty of incentive in academia to do that; e.g. I can imagine these two studies having survived such scrutiny).
Also regarding your point of how things on the ground could appear to be completely different from the truth if you don’t know the community well enough, in my experience typically local surveyors understand the context well enough to be able to explain to us foreigners what is happening. I agree that this could be a problem for some donors/organizations/studies that aren’t very careful, so I can only say that this won’t be a big problem if things are done well (which should be the case for charities recommended by GiveWell but I don’t know about others).
(I spent 5 months in a town in western Kenya, not super integrated into the local community or fluent in the local language so take this with a grain of salt if you want, but I do know very well how this kind of studies and surveys work since I did one myself.)
Because it is helpful to think about exactly what intervention is needed to help mobile money expand (which may differ by country), I’m throwing here a few potential barriers (mostly based on my own experience in Kenya and Myanmar):
Regulatory barriers (India allowed it only recently because of this; in Myanmar it’s still ongoing)
Network effects: in Kenya I heard that an important reason it took off was that Safaricom had a very high market share (maybe near 70%?); in Nigeria I heard that the fragmentation of the telecommunication market is one reason it didn’t take off. I’m not sure if more countries are similar to Kenya or Nigeria, also these are all anecdotal. One interesting thing though is that lack of competition in Kenya may have contributed to the high charges (though there is more competition now including from mobile carriers and banks).
Lack of trust: people may not trust mobile carriers or mobile money agents. Probably less of a problem in a close knit community where agents are shopkeepers. Also, lack of trust in banks is a common problem in developing countries but I have no idea about trust on mobile carriers/agents.
Existing alternatives already good enough: this has been mentioned to me in Myanmar, that the traditional “hundi” system of money transfer works well and is cheap which may dampen adoption of mobile money. If that’s true then mobile money wouldn’t contribute much anyway, but I’m skeptical since mobile money is really much more convenient. (Also it can be used as a savings tool like a checking account, and the poor often face savings constraint too, but I’m not sure how effective that will be; interventions that tackle “self-control” have worked well on this so such elements might need to be bundled in order for mobile money to help with saving)
Hi carneades, thank you for your post! It is great to see a post by an international development professional on effective altruism. As someone who did field work in Africa during PhD, I am sympathetic to what you conclude from your own observation. However, it is important to see what rigorous studies conclude and based on my reading of the literature I have some disagreements.
On job creation, taking into account the environment in most poor countries in terms of infrastructure, legal environment and productivity of the labor force, it would be much more costly to produce bed nets there than importing from somewhere that can make it cheaper. So given the choices of (A) importing cheaper bed nets that can save many more children in the poor country, and (B) producing bed nets locally at much higher cost (and by the way one would need to sell it at much higher price, or put in large subsidy for its production, neither of which makes much sense) while creating not that many jobs, (A) seems much better. (And I said “creating not that many jobs” because you are talking about simply setting up a bed net factory to meet local demand; for significant job creation the country would need China-type export manufacturing but that would require transforming the whole economy in terms of the points mentioned above — infrastructure, legal environment and productivity of the labor force — rather than setting up and probably subsidizing a few unproductive bed net factories which seems like bad industrial policy.)
On need vs. demand for insecticide-treated bed nets, see this article linked to by Fluttershy, especially the 2nd point under “Points of possible disagreement”: “irrationality about one’s health is common in the developed world. In the developing world, there are substantial additional obstacles to properly valuing medical interventions such as lack of the education and access necessary to even review the evidence. The effects of something like bed nets (estimated at one child death averted for every ~200 children protected) aren’t necessarily easy for recipients to notice or quantify.” There is a chapter in the book “Poor Economics” that argues that poor people fail to implement health practices with high returns like treating their drinking water or getting vaccinated, not because they are less rational than people in rich countries. People in rich countries may do no better under the same circumstances, but governments in rich countries provide the infrastructure, nudges or mandatory requirements to make these practices much less costly or even compulsory. Also, there is not only evidence that free distribution of bed nets does not lead to decreased usage, but also that it increases demand and usage in the first year (initial demand is very sensitive to price) which causes people to learn about its benefits and demand more in the future.
On Give Directly, see this study on how people use the money they get. It’s ex ante unclear how people would use the money, but the study shows that credit (and savings) constraint is a really big problem in these people’s lives and people end up using their money to improve food security, invest in durable goods or businesses etc. There was no significant increase on alcohol or tobacco consumption (the study tried to rule out desirability bias including using list randomization questionnaire) or decrease in labor supply.
Of course the studies cited here aren’t perfect but they seem pretty well done to me (and many experts in the field), so I would trust them more than anecdotal evidence which could vary a lot from place to place.