“Fishing,” said the old man “is at least as complicated as any other industry”.
I was sitting in a meeting of representatives of the other end of the fishing industry: fleets of North Sea trawlers turning over >£1million each per year, fishing in probably the world’s most studied at-risk fishing ecosystem. They were fuming because in the view of the scientists studying North Sea fish, cod stocks had reached dangerously low levels and their quotas needed reducing, but in the view of the fishermen actually catching the fish, cod stocks off the east coast of England were at such high levels they hit their month’s cod quota in a day whilst actively trying to avoid catching cod. (I have no reason to believe that either view was uninformed or deceptive). “What they’re probably not factoring in,” he closed on, “is that cod populations in different regions are cyclical”
The point of that waffly anecdote is that factoring in the effects of mosquito nets on local fish ecosystems would actually be really hard, because an RCT in one area over one year really isn’t going to tell you much about the ecosystems in other areas, or in other years. Even more so in isolated African watercourses. (I think we can largely rule out the hypothesis that amateurs with free 2x2m malaria nets instead of proper nets and boats are depleting the seas more than foreign factory ships with nets hundreds of metres long...)
Like you, I don’t find the idea that ~80% of distributed nets are appropriately used (at least for the first year) to have settled the debate, but quite a few things have to be the case for AMF’s distribution programs to be the major factor in ecosystem harm caused by overfishing:
the level of fishing with relatively small mosquito nets is sufficient to destroy fish stocks in the immediate area
other areas are unable to replenish the fish stocks depleted in the local area
in the absence of a continuing supply of free mosquito nets, people wouldn’t nevertheless use mosquito nets—which are also available for sale at a relatively low cost—for fishing
the most likely alternatives to mosquito nets for local fishermen—potentially including other fabrics which also don’t allow smaller fish to pass through—don’t have the same impact on the ecosystem
for the program to be net negative for humans, you’ve also got to assume
avoidable future falls in fishing stocks actually kill people. Or at least that the net economic loss from them is so great it outweighs the lives saved and malaria cases averted (and short term positive impact on availability of nutritious fish!). For the overall programme to be net negative we’re looking at ~24k deaths per year...
Plus of course many of the nets are distributed in regions where fishing wasn’t viable in the first place...
There is a tension between different EA ideas here in my view. Early on, I recall, the emphasis was on how you need charity evaluators like GiveWell, and RTCs by randomista development economists, because you can’t predict what interventions will work well, or even do more good than harm, on the basis of common sense intuition. (I remember Will giving examples like “actually, having prisoners talk to children about how horrible being in prison is seems to make the children more likely to grow up to commit crimes.) But it turns out that when assessing interventions, there are always points where there just isn’t high quality data on whether some particular factor importantly reduces (or increases) the intervention’s effectiveness. So at that point, we have to rely on commonsense, mildly disciplined by back of the envelope calculations, possibly supplemented by poor quality data if we’re lucky. And then it feels unfair when this is criticized by outsiders (like the recent polemical anti-EA piece in Wired) because well, what else can you possibly do if high-quality studies aren’t available and it’s not feasible to do them yourself? But I guess from the outsider’s perspective, it’s easy to see why this looks like hypocrisy: they criticized other people for relying on their general hunches about how things work, but now the EAs are doing it themselves! I’m not really sure what the general solution (if any) to this is. But it does feel to my like there are a vast number of choice points in GiveWell’s analyses where they are mostly guessing, and if those guesses are all biased in some direction rather than uncorrelated, assessments of interventions will be way off.
“Fishing,” said the old man “is at least as complicated as any other industry”.
I was sitting in a meeting of representatives of the other end of the fishing industry: fleets of North Sea trawlers turning over >£1million each per year, fishing in probably the world’s most studied at-risk fishing ecosystem. They were fuming because in the view of the scientists studying North Sea fish, cod stocks had reached dangerously low levels and their quotas needed reducing, but in the view of the fishermen actually catching the fish, cod stocks off the east coast of England were at such high levels they hit their month’s cod quota in a day whilst actively trying to avoid catching cod. (I have no reason to believe that either view was uninformed or deceptive). “What they’re probably not factoring in,” he closed on, “is that cod populations in different regions are cyclical”
The point of that waffly anecdote is that factoring in the effects of mosquito nets on local fish ecosystems would actually be really hard, because an RCT in one area over one year really isn’t going to tell you much about the ecosystems in other areas, or in other years. Even more so in isolated African watercourses. (I think we can largely rule out the hypothesis that amateurs with free 2x2m malaria nets instead of proper nets and boats are depleting the seas more than foreign factory ships with nets hundreds of metres long...)
Like you, I don’t find the idea that ~80% of distributed nets are appropriately used (at least for the first year) to have settled the debate, but quite a few things have to be the case for AMF’s distribution programs to be the major factor in ecosystem harm caused by overfishing:
the level of fishing with relatively small mosquito nets is sufficient to destroy fish stocks in the immediate area
other areas are unable to replenish the fish stocks depleted in the local area
in the absence of a continuing supply of free mosquito nets, people wouldn’t nevertheless use mosquito nets—which are also available for sale at a relatively low cost—for fishing
the most likely alternatives to mosquito nets for local fishermen—potentially including other fabrics which also don’t allow smaller fish to pass through—don’t have the same impact on the ecosystem
for the program to be net negative for humans, you’ve also got to assume
avoidable future falls in fishing stocks actually kill people. Or at least that the net economic loss from them is so great it outweighs the lives saved and malaria cases averted (and short term positive impact on availability of nutritious fish!). For the overall programme to be net negative we’re looking at ~24k deaths per year...
Plus of course many of the nets are distributed in regions where fishing wasn’t viable in the first place...
There is a tension between different EA ideas here in my view. Early on, I recall, the emphasis was on how you need charity evaluators like GiveWell, and RTCs by randomista development economists, because you can’t predict what interventions will work well, or even do more good than harm, on the basis of common sense intuition. (I remember Will giving examples like “actually, having prisoners talk to children about how horrible being in prison is seems to make the children more likely to grow up to commit crimes.) But it turns out that when assessing interventions, there are always points where there just isn’t high quality data on whether some particular factor importantly reduces (or increases) the intervention’s effectiveness. So at that point, we have to rely on commonsense, mildly disciplined by back of the envelope calculations, possibly supplemented by poor quality data if we’re lucky. And then it feels unfair when this is criticized by outsiders (like the recent polemical anti-EA piece in Wired) because well, what else can you possibly do if high-quality studies aren’t available and it’s not feasible to do them yourself? But I guess from the outsider’s perspective, it’s easy to see why this looks like hypocrisy: they criticized other people for relying on their general hunches about how things work, but now the EAs are doing it themselves! I’m not really sure what the general solution (if any) to this is. But it does feel to my like there are a vast number of choice points in GiveWell’s analyses where they are mostly guessing, and if those guesses are all biased in some direction rather than uncorrelated, assessments of interventions will be way off.
Thanks that is helpful. It’s frustrating how hard it is to be sure about this.