Hey, thanks for the thoughtful response. Here are my thoughts:
A may seem better from an international perspective, but if you ask the people here if they would rather have a factory and more jobs, something they could be proud of, a reason for young people to not leave the country in droves, and the feeling that they are able to solve their own problems, or more bed nets, they would pick A every time. I have asked many communities across the country in Community Analysis surveys what they need, they often say electricity, clean water, better roads, and consistently the most popular answer is more jobs. But I have never seen a community ask for bed nets, or a cure for malaria. To them, it is like the flu. The point is that, just because it seems better to you, it does not mean that it is what people want, and who are you to tell people in countries that you have never visited what they need? I agree that it would be more work to subsidize a factory, but the benefits to the morale of the people, (something which is hard to quantify beyond the percent of people that die trying to illegally immigrate to Europe) would drastically outweigh the costs, and you would be listening to what the people actually want instead of deciding for them.
But why do governments in richer countries implement these kinds of programs? Surely it is because the people in that country are more educated, and vote in leaders, or vote for initiatives which inevitably improve their health. Programs like AMF which cut the voter out of the loop do significant damage to democracies. When politicians are beholden to foreign companies and governments for aid instead of their people for votes, they will serve the interests of the foreign organizations, not the people. If the appeal is made to the government, not the people, then he people fail to see the importance of the intervention, and it is not sustainable. We should be educating people so that they ask their government to implement these changes, not cutting the people out of the equation for the sake of saving costs, and thereby cutting young democracies off at the knees. I would be surprised if bed net distributions would actively decrease use of nets, but I have seen many communities actively attempt to deceive researchers about their use of bed nets because they know what “the right answer” is and they know what they need to say to get the foreigners to give them more things. So while I doubt that use decreased, I’m skeptical that it increased at the rates touted by these organizations. My question is, why could we not instead do campaigns solely focused on increasing demand, so that people would actually ask their governments for these interventions. This might lead to people holding their officials (who are actually, at least in principle answerable to the people) responsible for health interventions, as opposed to foreign organizations who are answerable to their donors.
I do not live somewhere where Give Directly works, but there are similar programs where I live. Though the study was interesting, my central concern is that, at least here, lying to strangers is not merely culturally acceptable, it is expected, especially when those strangers are foreigners. Therefore any answers given by families on the baseline or endline surveys are certainly in question (especially since people here simply tell you what they think you want to hear). To give you a sense, we have local staff here who conduct surveys in villages, and it is difficult for them to even accurately find out the primary language of the village, without spending several weeks there since everyone just speaks to you in the language you start speaking to them in until they trust you more. Simply, if the people there are anything like the people here, I would be surprised if they took sufficient precautions to rule out desirability bias. The point about labor supply is interesting, but the study does note that it was unable to study true long term effects of these interventions. Give Directly is promoting providing villages money for ten years, from my understanding, the study did not analyze the effects of such actions over long periods of time anything more than four months “this variation in the present study is not sufficient to obtain reliable estimates for the evolution of the treatment effect over time” those are the interventions that I am concerned with, and those are the ones which could do more harm.
In your concluding sentence you make three points that I would like to address. First you claim that the studies are well done, second you claim that they should be trusted over anecdotal evidence, and third you claim that this may vary from place to place.
4) To the first point, simply because a study is well done, it does not make it true. You can do everything in your power to stop someone from lying to you, or telling you just what they think you want them to hear, but, at least where I am unless you live for weeks with a community, they will lie to you. I have seen so many researchers get survey answers from families that I am aware are false, simply because that is the culture here. Just because something is well done, it does not mean that it is true.
5) Yes, the information that I present is anecdotal, but in a culture where it is expected that you lie to strangers, that is all that we have. The problem is that so few donors actually come and experience the reality of what the organizations they support are doing, and even when they do, it is often for such a short time, or they are sufficiently insulted from the community that they fail to experience the catastrophic failures of the programs they are supporting (especially since they are often there seeing only what the program wants them to see). Due to the culture of lying to strangers here, I would trust an anecdote form someone that knows a community well over a well thought out study any day. Maybe that is just because I have seen so many studies here get things so wrong. How long were you here? Where? What kind of research did you do?
6) I would like to end in agreement, so to your last point, I think you are correct. My experience is limited to where I live, and West Africa is certainly different from East Africa in many ways. Problems that we face here, may not be significant concerns over there. I share these concerns and experiences because it seems to me that the majority of the people in this forum have spent little time in the actual communities that receive donations from these organizations. Those communities are unable to speak for themselves (usually no internet), so I am simply trying to give them a voice and explain why they would disagree so drastically with many of the proposed interventions. Thanks for sharing these articles and views. I have committed my life to doing the most good in the world, and from my experience organizations promoted by Give Well are not the best way. All the best!
Hi carneades, in reply I just want to make 2 general points here:
Many things need to be done in the developing world, e.g.the ones you mentioned: protecting people against malaria, creating jobs, improving the quality of governments… The most effective intervention for one purpose could be not very useful for another, but that’s still okay because it would be better than trying to do something that serves multiple purposes but is ineffective in all of them. (e.g. for protecting people against malaria, the most effective intervention could be distributing bed nets for free; for job creating, it could be improving the infrastructure and legal system; for improving the quality of governments it could be educating voters—though for this to affect government provision of bed nets down the road would take many years and if we solely rely on this to protect people against malaria we would miss the opportunity to save many lives now compared to using the most effective intervention on this)
By saying the studies are good I mean the researchers take into account that respondents may not report the truth and seem to manage to find out the truth despite that. This is not just because I trust these famous economists to do a good job or I know some of them personally and know that they care about doing a good job, but I also find evidence from the papers. For instance in the Cohen and Dupas paper on free distribution of bed nets, it says (on p14, under “III.C Data”) “During the home visits, respondents were asked to show the net, whether they had started using it, and who was sleeping under it. Surveyors checked to see whether the net had been taken out of the packaging, whether it was hanging, and the condition of the net.” In the Haushofer and Shapiro paper on Give Directly, on p28 they talk about potential desirability bias on alcohol and tobacco and how they address it, and on p32 they mention assets including metal roofs and livestocks which surveyors could easily check (I don’t think they mentioned surveyors checking this but the study is done by IPA in the same region I work in Kenya so I imagine it should have the high standard of work done by this organization and surveyors should check things when they can). In general economists don’t like to rely on survey data precisely because people may lie, and in developing countries when this is often inevitable we try hard to get around the problem and mention how we address them in papers (otherwise you would get a lot of questioning from presentation audience, referees, etc. so there’s plenty of incentive in academia to do that; e.g. I can imagine these two studies having survived such scrutiny).
Also regarding your point of how things on the ground could appear to be completely different from the truth if you don’t know the community well enough, in my experience typically local surveyors understand the context well enough to be able to explain to us foreigners what is happening. I agree that this could be a problem for some donors/organizations/studies that aren’t very careful, so I can only say that this won’t be a big problem if things are done well (which should be the case for charities recommended by GiveWell but I don’t know about others).
(I spent 5 months in a town in western Kenya, not super integrated into the local community or fluent in the local language so take this with a grain of salt if you want, but I do know very well how this kind of studies and surveys work since I did one myself.)
Certainly there are many different kinds of interventions that we can implement. Some interventions are mutually exclusive, others are not. Toms shoes comes and donates shoes to a community, where another organization is attempting to improve small businesses, particularly tailors, some of the few professionals in very small communities. The donation puts many local tailors out of business. Therefore, we either need to develop interventions which are not mutually exclusive or choose between interventions. If we want to do the first, organizations like AMF should help local businesses instead of destroying them. If we want to do the second, we should choose between interventions by asking the people what they need, not deciding what we think is best without ever talking to the people on the ground. If they don’t want us to save their lives at the expense of job opportunities, we should not. People should have the right to choose their own interventions, especially when those interventions can do their communities harm. If you want to go in without the community’s consent, the least that you can do first is do no harm.
As for the studies, at the end of the day I’m a philosophical skeptic all the way down (which means that I have so serious concerns about the relation between truth and the scientific method, for legitimate philosophical reasons, such as the problem of induction , and the problem of underdetermination, check out the Stanford Encyclopedia of Philosophy for more on that) but we can put that aside for now. As for the trust, I trust the people that live here about what they need, and what they are doing more than I trust foreigners who don’t spend much time with the people, famous economists or not. Your quote about nets fails to address a number of concerns, such as communities that put up nets whenever they see the inspectors coming (you would be surprise how many people do this), and the fact that most people here hang out outside until well after peak mosquito biting time, so even if they sleep under a mosquito net, and are fully truthful in an interview, since they don’t actually understand or appreciate the importance of bed nets, and someone values something much less when they get it for free. Checking for a metal roof seems like a clear method, but there are so many more which could be lied about. Livestock wanders freely here, and is rarely kept in any kind of pen, and only the people that live in a village really know who owns what. I’m not saying that all of their metrics are necessarily off, I’m saying that I’m not convinced that they are correct. As for the academic criticism, I’m concerned that the people that provide the scrutiny are, unfortunately, not the people with an in-depth experiential understanding of the cultural practices of these communities. I am also concerned that there is a gap between something being able to survive academic criticism, and providing a fully accurate picture of the state of affairs here.
As for the final point, why do you think that the local surveyors are able to solicit truthful responses any more than foreigners? If those local surveyors did not grow up in the same tiny area of the village, the families will lie through their teeth. I have lived here for years, and I am close to fluent in a local language and I have seen families lie to every census worker that comes through, local or foreign. I have also seen translators blatantly lie to researchers about what respondents say. Truth is not valued here. Unless you do speak the language and have lived with a family for quite some time, there’s no way for you to know if they actually sleep under a bed net, if they actually exclusively breast feed, or if they really own that goat. As noted before, I cannot say that this happens everywhere as Africa is a big place, and cultures differ drastically. Perhaps in East Africa there are different cultural practices which make it easier to conduct research, but here even the Government cannot correctly take a census.
Thanks for taking the time to engage in in this discussion. I hope that you reconsider donating to charities which ignore what communities actually want in favor of providing cheap, but ultimately harmful interventions.
Hey, thanks for the thoughtful response. Here are my thoughts:
A may seem better from an international perspective, but if you ask the people here if they would rather have a factory and more jobs, something they could be proud of, a reason for young people to not leave the country in droves, and the feeling that they are able to solve their own problems, or more bed nets, they would pick A every time. I have asked many communities across the country in Community Analysis surveys what they need, they often say electricity, clean water, better roads, and consistently the most popular answer is more jobs. But I have never seen a community ask for bed nets, or a cure for malaria. To them, it is like the flu. The point is that, just because it seems better to you, it does not mean that it is what people want, and who are you to tell people in countries that you have never visited what they need? I agree that it would be more work to subsidize a factory, but the benefits to the morale of the people, (something which is hard to quantify beyond the percent of people that die trying to illegally immigrate to Europe) would drastically outweigh the costs, and you would be listening to what the people actually want instead of deciding for them.
But why do governments in richer countries implement these kinds of programs? Surely it is because the people in that country are more educated, and vote in leaders, or vote for initiatives which inevitably improve their health. Programs like AMF which cut the voter out of the loop do significant damage to democracies. When politicians are beholden to foreign companies and governments for aid instead of their people for votes, they will serve the interests of the foreign organizations, not the people. If the appeal is made to the government, not the people, then he people fail to see the importance of the intervention, and it is not sustainable. We should be educating people so that they ask their government to implement these changes, not cutting the people out of the equation for the sake of saving costs, and thereby cutting young democracies off at the knees. I would be surprised if bed net distributions would actively decrease use of nets, but I have seen many communities actively attempt to deceive researchers about their use of bed nets because they know what “the right answer” is and they know what they need to say to get the foreigners to give them more things. So while I doubt that use decreased, I’m skeptical that it increased at the rates touted by these organizations. My question is, why could we not instead do campaigns solely focused on increasing demand, so that people would actually ask their governments for these interventions. This might lead to people holding their officials (who are actually, at least in principle answerable to the people) responsible for health interventions, as opposed to foreign organizations who are answerable to their donors.
I do not live somewhere where Give Directly works, but there are similar programs where I live. Though the study was interesting, my central concern is that, at least here, lying to strangers is not merely culturally acceptable, it is expected, especially when those strangers are foreigners. Therefore any answers given by families on the baseline or endline surveys are certainly in question (especially since people here simply tell you what they think you want to hear). To give you a sense, we have local staff here who conduct surveys in villages, and it is difficult for them to even accurately find out the primary language of the village, without spending several weeks there since everyone just speaks to you in the language you start speaking to them in until they trust you more. Simply, if the people there are anything like the people here, I would be surprised if they took sufficient precautions to rule out desirability bias. The point about labor supply is interesting, but the study does note that it was unable to study true long term effects of these interventions. Give Directly is promoting providing villages money for ten years, from my understanding, the study did not analyze the effects of such actions over long periods of time anything more than four months “this variation in the present study is not sufficient to obtain reliable estimates for the evolution of the treatment effect over time” those are the interventions that I am concerned with, and those are the ones which could do more harm.
In your concluding sentence you make three points that I would like to address. First you claim that the studies are well done, second you claim that they should be trusted over anecdotal evidence, and third you claim that this may vary from place to place.
4) To the first point, simply because a study is well done, it does not make it true. You can do everything in your power to stop someone from lying to you, or telling you just what they think you want them to hear, but, at least where I am unless you live for weeks with a community, they will lie to you. I have seen so many researchers get survey answers from families that I am aware are false, simply because that is the culture here. Just because something is well done, it does not mean that it is true.
5) Yes, the information that I present is anecdotal, but in a culture where it is expected that you lie to strangers, that is all that we have. The problem is that so few donors actually come and experience the reality of what the organizations they support are doing, and even when they do, it is often for such a short time, or they are sufficiently insulted from the community that they fail to experience the catastrophic failures of the programs they are supporting (especially since they are often there seeing only what the program wants them to see). Due to the culture of lying to strangers here, I would trust an anecdote form someone that knows a community well over a well thought out study any day. Maybe that is just because I have seen so many studies here get things so wrong. How long were you here? Where? What kind of research did you do?
6) I would like to end in agreement, so to your last point, I think you are correct. My experience is limited to where I live, and West Africa is certainly different from East Africa in many ways. Problems that we face here, may not be significant concerns over there. I share these concerns and experiences because it seems to me that the majority of the people in this forum have spent little time in the actual communities that receive donations from these organizations. Those communities are unable to speak for themselves (usually no internet), so I am simply trying to give them a voice and explain why they would disagree so drastically with many of the proposed interventions. Thanks for sharing these articles and views. I have committed my life to doing the most good in the world, and from my experience organizations promoted by Give Well are not the best way. All the best!
Hi carneades, in reply I just want to make 2 general points here:
Many things need to be done in the developing world, e.g.the ones you mentioned: protecting people against malaria, creating jobs, improving the quality of governments… The most effective intervention for one purpose could be not very useful for another, but that’s still okay because it would be better than trying to do something that serves multiple purposes but is ineffective in all of them. (e.g. for protecting people against malaria, the most effective intervention could be distributing bed nets for free; for job creating, it could be improving the infrastructure and legal system; for improving the quality of governments it could be educating voters—though for this to affect government provision of bed nets down the road would take many years and if we solely rely on this to protect people against malaria we would miss the opportunity to save many lives now compared to using the most effective intervention on this)
By saying the studies are good I mean the researchers take into account that respondents may not report the truth and seem to manage to find out the truth despite that. This is not just because I trust these famous economists to do a good job or I know some of them personally and know that they care about doing a good job, but I also find evidence from the papers. For instance in the Cohen and Dupas paper on free distribution of bed nets, it says (on p14, under “III.C Data”) “During the home visits, respondents were asked to show the net, whether they had started using it, and who was sleeping under it. Surveyors checked to see whether the net had been taken out of the packaging, whether it was hanging, and the condition of the net.” In the Haushofer and Shapiro paper on Give Directly, on p28 they talk about potential desirability bias on alcohol and tobacco and how they address it, and on p32 they mention assets including metal roofs and livestocks which surveyors could easily check (I don’t think they mentioned surveyors checking this but the study is done by IPA in the same region I work in Kenya so I imagine it should have the high standard of work done by this organization and surveyors should check things when they can). In general economists don’t like to rely on survey data precisely because people may lie, and in developing countries when this is often inevitable we try hard to get around the problem and mention how we address them in papers (otherwise you would get a lot of questioning from presentation audience, referees, etc. so there’s plenty of incentive in academia to do that; e.g. I can imagine these two studies having survived such scrutiny).
Also regarding your point of how things on the ground could appear to be completely different from the truth if you don’t know the community well enough, in my experience typically local surveyors understand the context well enough to be able to explain to us foreigners what is happening. I agree that this could be a problem for some donors/organizations/studies that aren’t very careful, so I can only say that this won’t be a big problem if things are done well (which should be the case for charities recommended by GiveWell but I don’t know about others).
(I spent 5 months in a town in western Kenya, not super integrated into the local community or fluent in the local language so take this with a grain of salt if you want, but I do know very well how this kind of studies and surveys work since I did one myself.)
Thanks again. Here are my thoughts.
Certainly there are many different kinds of interventions that we can implement. Some interventions are mutually exclusive, others are not. Toms shoes comes and donates shoes to a community, where another organization is attempting to improve small businesses, particularly tailors, some of the few professionals in very small communities. The donation puts many local tailors out of business. Therefore, we either need to develop interventions which are not mutually exclusive or choose between interventions. If we want to do the first, organizations like AMF should help local businesses instead of destroying them. If we want to do the second, we should choose between interventions by asking the people what they need, not deciding what we think is best without ever talking to the people on the ground. If they don’t want us to save their lives at the expense of job opportunities, we should not. People should have the right to choose their own interventions, especially when those interventions can do their communities harm. If you want to go in without the community’s consent, the least that you can do first is do no harm.
As for the studies, at the end of the day I’m a philosophical skeptic all the way down (which means that I have so serious concerns about the relation between truth and the scientific method, for legitimate philosophical reasons, such as the problem of induction , and the problem of underdetermination, check out the Stanford Encyclopedia of Philosophy for more on that) but we can put that aside for now. As for the trust, I trust the people that live here about what they need, and what they are doing more than I trust foreigners who don’t spend much time with the people, famous economists or not. Your quote about nets fails to address a number of concerns, such as communities that put up nets whenever they see the inspectors coming (you would be surprise how many people do this), and the fact that most people here hang out outside until well after peak mosquito biting time, so even if they sleep under a mosquito net, and are fully truthful in an interview, since they don’t actually understand or appreciate the importance of bed nets, and someone values something much less when they get it for free. Checking for a metal roof seems like a clear method, but there are so many more which could be lied about. Livestock wanders freely here, and is rarely kept in any kind of pen, and only the people that live in a village really know who owns what. I’m not saying that all of their metrics are necessarily off, I’m saying that I’m not convinced that they are correct. As for the academic criticism, I’m concerned that the people that provide the scrutiny are, unfortunately, not the people with an in-depth experiential understanding of the cultural practices of these communities. I am also concerned that there is a gap between something being able to survive academic criticism, and providing a fully accurate picture of the state of affairs here.
As for the final point, why do you think that the local surveyors are able to solicit truthful responses any more than foreigners? If those local surveyors did not grow up in the same tiny area of the village, the families will lie through their teeth. I have lived here for years, and I am close to fluent in a local language and I have seen families lie to every census worker that comes through, local or foreign. I have also seen translators blatantly lie to researchers about what respondents say. Truth is not valued here. Unless you do speak the language and have lived with a family for quite some time, there’s no way for you to know if they actually sleep under a bed net, if they actually exclusively breast feed, or if they really own that goat. As noted before, I cannot say that this happens everywhere as Africa is a big place, and cultures differ drastically. Perhaps in East Africa there are different cultural practices which make it easier to conduct research, but here even the Government cannot correctly take a census.
Thanks for taking the time to engage in in this discussion. I hope that you reconsider donating to charities which ignore what communities actually want in favor of providing cheap, but ultimately harmful interventions.