Concerns about AMF from GiveWell reading—Part 4
I’ve now read everything on the GiveWell website about the Against Malaria Foundation, a top rated charity since 2011. This has helped me increase my understanding of the work they do and the challenges involved. This is the fourth in a series of posts summarising my outstanding questions from this reading.
It may be that I’ll find the answers to some of these questions by looking elsewhere, for example reading the AMF website or getting in touch with them directly. That means this is not the final word on my view of the Against Malaria Foundation. However, I’m capturing my progress at this stage so that I have a clear basis to build on for further work.
Concern #4: Distributing bednets may have some downsides, and overall we risk doing more harm than good.
Many discussions of aid cost-effectiveness focus exclusively on the cost per life saved. There is no allowance for the other benefits of distributing bednets, such as preventing non-fatal cases of malaria or other mosquito-transmitted diseases, and no allowance for the other costs due to distributing bednets, such as damage to the local economy and community independence. All of these impacts, good and bad, occur in the real world when we donate to AMF, so they should be considered even if they cannot be quantified.
We could make a reasonable estimate of the extra benefits, as non-fatal malaria cases and other diseases can be measured in the same way as fatal malaria cases and with a similar level of accuracy. GiveWell do this in their cost-effectiveness calculations, and then allow for the extra income people will earn by avoiding illness in childhood.
Quantifying the downsides is harder. We could estimate the number of people who would have been making anti-malaria nets locally if we didn’t provide such nets for free. In many countries nets were made and sold locally before international action changed expectations to be that such things should be provided for free. Those who worked making and selling bednets lost their jobs and it is unlikely anybody would start up a new business knowing the next free distribution is only a year or two away. On top of this are second-order impacts on connected industries, where reduced investment and a less-skilled population means clothing & textiles manufacturers are also affected by donations of bednets.
This matters because the countries receiving bednets are some of the poorest in the world, so anything that reduces income is a significant matter. Developing the economy of these countries will allow the people that live there to escape poverty and all of its harms, not just malaria. Economic development is not just about factories, it cuts across the whole of society. Political power, public institutions and the daily interactions between individuals and groups all combine to generate wealth. By trying to help in one specific area we are definitely disrupting this complex web. Whether that disruption is overall good or bad is a matter of debate.
Including these concerns in our assessment of donating bednets makes the range of outcomes even wider. Perhaps net donations are an even better deal than GiveWell allows for, where a few thousand dollars saves a life AND prevents 100 non-fatal malaria cases without any noticeable effect on incomes. Or perhaps net donations do more harm than good, and holding back economic development more than cancels out the benefit of fewer cases of malaria. Even if we did have the data and knew exactly what the impacts were, these things are hard to compare. How many jobs lost are a good trade-off for malaria lives saved? The answer probably varies from person to person, both in the country receiving the nets and among donors. But ignoring the question doesn’t make it go away.
What do people receiving donated bed nets want? A large number of these people have tough lives, with less wealth and more illness than we are used to. If they want to prioritise increasing their wealth, which in turn will naturally reduce illness, we would need to be very confident in our assessment of long lasting insecticidal nets to override that choice.
As a thought experiment, imagine if we gave recipients a choice between receiving a bed net or $5 (the typical cost of the bed net). How many do we think would choose the $5? Certainly a significant proportion would take the money, since bednet coverage was not universal before global aid programs kicked in. That may be because they do not understand the risks of malaria as well as we do, though again we’d need to be careful in making that claim considering they have day-to-day experience of life with malaria. It may be because there’s a coordination problem, where they believe bednets only work if everybody else buys them too, or because the people who make buying decisions are not the ones who would benefit from bednets. Or it may just be that there are people in sub-Saharan countries who have a full understanding of their own needs and can think of much better things to do with $5 than buy a bednet to reduce the risk of malaria.
There are some people who think free bed nets are a bad deal. One international development worker writes:
Here in West Africa, Malaria is seen very much like the flu…Imagine that you need jobs in your small town, but a foreign NGO comes in and claims you need more flu shots to eradicate the flu. In fact they put your local doctors out of business by bringing in their own trying to eradicate it. Your community did not want flu shots, you wanted jobs, but this foreign NGO came in and ignored your right to choose your own intervention and did more harm than good. That is exactly what AMF does in villages here every day.
One way to reduce the uncertainty here would be to buy bednets locally. The Against Malaria Foundation don’t do this as they see their duty as buying bednets at the cheapest price they can to save as many lives as possible. If they didn’t, the cost-per-life saved for this intervention would increase, perhaps beyond the threshold considered acceptable by GiveWell. And trying to solve many problems at the same time, rather than a targetted intervention with measurable outcomes, is a typical failing of aid programmes generally.
As a donor, I think there’s a huge difference between the following three scenarios
“You’re probably doing some good, and at worst you’re doing no harm”
“You’re probably doing a lot of good but at the cost of doing some harm”
“You’re probably doing some good, but there’s a chance you’re making a bad situation worse”.
This point about unknown impacts is not restricted to malaria and bednets. There is a wider conversation about cluelessness and the interactions between aid and economic development. The majority of people quietly think we can’t really help those living in the world’s poorest countries. Those who think we can risk focussing on short term benefits which are clear and measurable and overlooking long term costs which are fuzzy and hard to measure. There is no clear feedback mechanism for donors who live far away and don’t see the consequences of their interventions, meaning there will always be the risk of doing harm when we try to help those most in need.
Measuring the causal impact of malaria net distributions is hard. Your first post pointing that out was informative and valuable.
In contrast, it’s easier and there’s concrete methods to measure industry and ideas like substitution/elasticity, that exactly allow us to look at displacement and market externalities.
I think the statements like “[economies are a] complex web. Whether that disruption is overall good or bad is a matter of debate” don’t reflect the availability of those methods or thought.
I think global health is different from the other major EA cause areas. I think there’s decades of work and thought from very talented people, and a graveyard of thoughts and projects of almost the same size.
From this, I expect there’s knowledge or sophistication about the displacement issue you raise, as well as sophistication on a lot of other issues in general.
Because of this knowledge, I think it would have good to hear more substantive content about how nets displace and damage indigenous industry, like even basic stats or a set of anecdotes.
I think bringing up issues in a general or theoretical sense is not informative, and pointing them at AMF or any other charity isn’t compelling, in the context above.
I agree that in theory you can measure such economic impacts. In practice I don’t believe anybody is.
If a body of practical knowledge on this point exists, then it would be straightforward to quantify the economic downsides of bednet distibutions and include it in the GiveWell calculation. I am confident GiveWell are intellectually honest enough to do such a thing. I believe the reason they haven’t done this is that the information isn’t out there.
When the information isn’t out there, all you can do is make general/theoretical points and share anecdotes like the one I linked to above. There must be a risk of doing harm when we ignore such general points and anecdotes with the only justification being “the evidence must be out there”, even though nobody has pointed to where the evidence actually is.
I’m skeptical that having day-to-day experience makes one sophisticated, when it comes to fairly low-probability event like malaria deaths. The middle ages black plague-era peoples got it wrong a lot, it seems from my limited historical knowledge. Sometimes the ‘birds eye’ view is better. And I believe there is a great deal of scientific knowledge about malaria transmission and risks. (Although still perhaps not enough resources devoted to this.)
I agree that humans are generally bad at risk management for low-likelihood-high-impact events. I think this is not limited to black plague era peoples or those living with malaria today. I believe it’s a feature of human brains which scientific knowledge helps mitigate.
Despite this, we generally let people make their own decisions about things which affect their health. People smoke, drink, over-eat, fail to exercise, etc but societies rarely force interventions on their own residents to prevent this. If a benevolent foreign government gave every citizen in my country a free exercise bike worth $300 I’d think they missed an opportunity to do even more good by just giving everybody $300.
Moving from individuals to groups, it isn’t clear to me that the governments of bednet-receiving countries would make these choices either. The main recipient of AMF bednets this year is the DRC, whose govenment spends 12% of GDP each year, about a quarter of which is on healthcare. A bednet per person costs about 1% of GDP (since nets costs around $5.50 each and GDP is around $550 per person). If a govenment with access to scientific knowledge hasn’t spent its first 12% of GDP on this project, how confident are we that’s what it would spend the next 1% on?
Good points and apologies for picking on maybe the less strong part of your argument rather than steelmanning or whatever it’s called but:
But we do frequently tax alcohol and cigarettes and propagandise and subsidise healthy behaviours and exercise
Actual question: how confident are you that the DRC government tends to make good spending choices?
I like “steelmanning”, so thanks for sharing that.
Sin taxes & behavioural nudges seem to support my point rather than work against it. The US banned alcohol and discovered many people kept on drinking anyway, so now limits itself to talking a good game and collecting the extra tax income. Most health professionals are very clear that alcohol is bad, and many claim if it were invented today governments would ban it like so many other drugs. Yet no government I know says “We’ve looked at the scientific evidence, this is a clear example of people making bad choices, so we’re going to force them to make a good choice”.
I know almost nothing about the DRC government. My best guess is that it makes OK spending choices. More than a quarter of government spending is on health, about half as much again on education. (3.3% and 1.5% of GDP, out of a total spend of 11.9%.) That seems reasonable. High level statistics on health & education have been improving over the last 20 years despite there being a civil war for a lot of that time. Turning it around, if we think the DRC government, health administrators, doctors and nurses make bad spending decisions then that seems like a much better opportunity to improve things.
Ok prohibition didn’t work but I don’t think we know of alcohol and tobacco taxes etc are having good or ill effects.
I agree the overall shares look ok on DRC spending but that doesn’t tell the whole story obviously. According to a quick Wikipedia dig and my memory of reading DRC is known for tremendous corruption.
Another data point against “locals know there own interests best”?:
On this point:
What intervention/opportunity would you propose to address this?
I don’t have a specific intervention/opportunity in mind for the scenario where health spending is broken.
I’m reminded of a survey of several poor countries which revealed many were not following best practice for treating complications in pregnancy and childbirth despite the treatments being cheap and well-known. Digging into it showed there wasn’t a single reason for this, so no single intervention would change things everywhere.
If the underlying reality is locals make bad choices, as normal individuals and health practitioners and policiticians, I don’t think distributing nets for free is going to make much difference. The moral argument for intervening seems to be a lot weaker too. In that set-up we have replaced Singer’s drowning child with an adult who refuses swimming lessons and ignores all advice to stay out of the water.
I’ve responded to some of Carneades’ points on how AMF might affect local economies on a separate thread. I’m not an expert on global health or economics, but it was hard for me to put together a compelling story where the local economic considerations led to nets being substantially less beneficial, much less net-negative (no pun intended).
If any of what I’ve written sounds off to you, please let me know! I was hoping for some kind of critical/challenging response, but no one really noticed the comment.
An excerpt:
Thanks Aaron for this comment, and the longer one you made elsewhere.
On the excerpt, I think there’s anecdotal evidence for bullet points 1-3 in a few places. GiveWell mention this in an old blogpost about people not buying subsidised nets:
If this happens in a minority of locations, we may just accept that’s the cost of doing business. In order to do a large amount of good overall we accept we’re going to do a small amount of harm. If it happens in a large number of locations, the harm is increased.
Agree with the logic of your last bullet above, though I don’t know the difference in difference in impact between 20% coverage and 80% coverage. It isn’t necessarily linear—could be better or worse than that—and from memory the RCTs here all had universal coverage so no insight from there.
In terms of other stories where distributions may be net negative, I outlined a few in the comments recently. For citizens, workers, business owners and governments there may be wider impacts beyond the market-demand-for-nets idea you’ve described above. How important you think those ideas are probably depends on how you think the economy really works and what might promote or prevent future growth.
A belated thanks for this reply! I’ve reached the end of my knowledge/spare time for research at this point, but I’ll keep an eye out for any future posts of yours on these topics.
I think it would not have been difficult for you to do a back of the envelope calculation for how many net makers would be out of business for each amount of nets distributed (a net maker can make X nets, coverage was Y% before AMF arrived). The lack of even a bare bones quantitative case reinforces my prior that this is very unlikely to be a significant issue.
Agree a simple calculation as outlined wouldn’t be hard. That would effectively increase the cost-per-life-saved by 20%, say, which is noteworthy but not fundamentally changing things.
The real risk is the longer-term, hard-to-measure impacts which may hold back economic progress generally. These are by definition hard to fit in to a cost-per-life saved calculation but that doesn’t mean the impacts don’t exist. Knowing these risks exist and intervening anyway is a choice some donors will be comfortable with but others will not.
This doesn’t seem likely to me. Sketching the simple logic …
AMF gives out X free or nearly-free nets
The people making and selling nets lose net income X(p-c), where p is the price and c is the materials cost (p-c is their markup). But they can use their recovered time T to produce and earn at least something else, worth w*T (w is their productivity or ‘wage’ in the next-best job).[1]
The people who previously bought the nets get back X*p they can use to spend on something else, stimulating other ‘multipliers’.
The economy as a whole is augmented by Xc (as AMF pays the cost of nets) + wT (the recovered value of time of the net-makers)
I also don’t think that being ‘frozen out of the job market’ is as big a thing in low-income countries, as they tend to have much less regulated and less rigid labor markets… more casual and informal labor
Agree with your outline of first-order impacts. My concern is with wider consequences not included in that view. The world is a complicated place and unintended side effects can be significant and negative. This is even possible under the assumption that such distributions are viewed as a good thing by locals. For example:
For citizens, the lesson may be “Random acts from people far away determine my circumstances, so there’s little I can do to improve my lot”. That would hold back all economic development in a way that outstripped the benefit of anti-malarial bednets.
For workers, the lesson may be “You can lose your job and your whole industry overnight, so don’t specialise”. That’s a bad outcome because specialisation is a huge driver of economic progress.
For business owners, the lesson may be “The risks of starting a new venture are higher than before”. Destroying incentives to start useful businesses is certainly a backwards step.
For governments, the lesson may be “It’s better to be appealing to foreign donors than to sort out our own problems”. Another backward step, because no nation in history became rich waiting for others to help them out.
I don’t think these the the most likely outcomes, just that they are possible. If somebody thought these things were not a risk, they would need to have an extremely high level of confidence in their model of international development and be able to explain why.
Those are some reasonable negatives. I’m working to brainstorm some positive knock on effects to counter this. :)
Thank you for raising some interesting concerns JP.
I just wanted to note that the value of a market for bednets may be small relative to the value of philanthropic funding for several reasons:
Having gone down the philanthropy path, ceasing to provide bednets philanthropically now would be unlikely to lead to a flourishing bednet market. See more on this here under “People may not purchase ITNs because they are unavailable in local markets or because they expect to be given them for free”
There are many reasons people may buy fewer bednets in a market than is socially optimal: lack of available funds, present bias, positive externalities (not internalising the societal benefit of reducing malaria transmission).
Business owners can sell other, less crucial goods and services. But in poverty stricken locations, they cannot provide and distribute thousands of life-saving/improving bednets to the poor.
Warm regards,
Lucas
Thanks Lucas. Agree that these may be reasons for 100% coverage to be a reasonable philanthropic target which would be unachievable through commercial means.
Your first point includes the idea that some people may not purchase ITNs because they expect to be given them for free. This reinforces the idea in the essay that there is an extra cost to such distributions as nobody can make a living from selling nets in areas where people think the price should be zero.