I’m particularly keen on the mass media to prevent violence against women intervention. In fact in August, I sent a version of your webpage covering the same topic to the co-authors of the “one experimental study [that] measured effects on behavior” (my advisor and two grad school classmates, FWIW), and they are all positive about this project as well.
I’d like to know more about the thinking behind the claim that a mass media IPV campaign would be 50x as effective as cash transfers at improving welfare (which roughly follows from your claim that you want to fund things that are 5X as effective as GiveWell’s top charities, which they claim are all 10X as effective as GiveDirectly’s programs). For two reasons, this seems like a very high estimate to me:
As it happens, we can make an apples-to-apples comparison between cash transfers’ and mass media’s effects on IPV, because Haushofer et al. (2019) study the effects of cash transfers on IPV, and and find that transfers to “women averaging USD 709 reduced physical and sexual violence (-0:26, −0:22 standard deviations),” while “Transfers to men reduced physical violence (-0:18 SD).” In more detail, transfers led to “a decrease in being pushed or shaken by the husband by 7 percentage points relative to a control group mean of 27 percent (a 26 percent reduction); being slapped by the husband (11 percentage point decrease relative to 33 percent control group mean, a 33 percent reduction); being punched (6 percentage point decrease relative to 15 percent control group mean, a 39 percent reduction); and being kicked, dragged, or beaten (8 percentage point decrease relative to 15 percent control group mean, a 51 percent reduction).”
Green, Wilke, and Cooper, by contrast, report that “screenings reduced the average rate at which women in treatment communities experienced violence by roughly .15 to .34 of an incident, from a baseline of just over half an incident on average...[which] fall short of significance.” However, “the campaign reduces the village-level proportion of women respondents who report any violence in their household by seven percentage points.”
Reading these two papers side by side, my overall impression is that cash is probably better, for an individual woman, at reducing IPV than a mass media campaign. So I am wondering how you got to a conclusion that mass media is 50x more effective? I am assuming it has something to do with cost of delivery, but to get from a smaller effect size to a better cost benefit-analysis would require the intervention to be incredibly cheap to deliver on a per-person basis—I’m ballparking it at like $10. Perhaps this is possible once you’ve achieved massive scale and content you’re happy with, but both of those things are going to take time and money, and the project might fail along the way, all of which raise costs in expectation. Giving a dollar to GiveDirectly, by contrast, leads to about 89c going to a person in extreme poverty right away. So I’d love to hear your thinking on this question spelled out in more detail.
You write: “The evidence on the effectiveness of mass media in IPV is limited but cautiously optimistic: While only one experimental study measured effects on behavior, four other experimental and quasi-experimental studies found sizable shifts in attitudes and norms related to IPV.” How well do you think those norms are correlated with IPV, and how well do you think that changing one will change the other?
Another way to frame this: Going back to Haushofer et al., the authors posit a very different theory, which is that men who commit IPV in the sample do so mainly to extract financial resources from their spouses—in other words, because they’re poor, which is not an attitude problem. How do you reconcile these two perspectives?
Thanks in advance, I really look forward to seeing this all come together!
Thank you for your support and thoughtful comments! Apologies for the delay – it’s a really busy time for us.
To clarify, we aimed for 5x GiveWell bar for health policy ideas—the text is an example of our goals in our research process. For this round, the bar needed to be more formally established.
Our claim for the idea around IPV is that the modeled cost-effectiveness for a “hypothetical 5-year intervention in Lesotho, Rwanda, Angola, and Ethiopia revealed a cost per disability-adjusted life year (DALY) averted ranging from $28 to $1419”. The wide range is because it is highly unclear how one should model the DALY burden of IPV, among other reasons. Our full report is published on our website now, so you can further dig into our thinking behind this. Of note, GiveWell tends to use a cost-effectiveness bar of $100-150 per DALY averted (which they claim to be ~8X GiveDirectly, note this is no longer the case given OP updates), which means that our estimates for this charity fall within the range of 40-0.5x GiveDirectly in cost-effectiveness. However, we aren’t sure if GiveWell has directly included these IPV benefits in its evaluation of GiveDirectly’s program.
Regardless, thanks for bringing this Haushofer et al. (2019) paper to our attention, We hadn’t seen this paper during our research, which is a pity. I cannot look into the paper in detail now; hopefully, my above clarification shows that we aren’t directly claiming a “ 50x over GD” bar. What you say sounds roughly true—I’d expect cash transfers to be far more expensive per person (mass media would be cents per person reached) but more effective (if the effect holds—I haven’t looked at the paper in any detail).
On your second point. We write a bit more about our sense of what the mechanism behind the intervention is in the report. To summarise—there is no clear sense in the literature about the main accepted mechanism for reducing violence or the true causal link between attitudes/norms and the prevalence of violence. It’s plausible that finances are a part of this, but it is likewise plausible that patriarchal norms, acceptability, etc., play a role. Different papers point in different directions, and this is an area that would benefit a lot from further research so that we can understand what works best. The context will matter a lot as well in terms of what drives violence and what type of violence is committed.
Really excited to see this coming along!
I’m particularly keen on the mass media to prevent violence against women intervention. In fact in August, I sent a version of your webpage covering the same topic to the co-authors of the “one experimental study [that] measured effects on behavior” (my advisor and two grad school classmates, FWIW), and they are all positive about this project as well.
I’d like to know more about the thinking behind the claim that a mass media IPV campaign would be 50x as effective as cash transfers at improving welfare (which roughly follows from your claim that you want to fund things that are 5X as effective as GiveWell’s top charities, which they claim are all 10X as effective as GiveDirectly’s programs). For two reasons, this seems like a very high estimate to me:
As it happens, we can make an apples-to-apples comparison between cash transfers’ and mass media’s effects on IPV, because Haushofer et al. (2019) study the effects of cash transfers on IPV, and and find that transfers to “women averaging USD 709 reduced physical and sexual violence (-0:26, −0:22 standard deviations),” while “Transfers to men reduced physical violence (-0:18 SD).” In more detail, transfers led to “a decrease in being pushed or shaken by the husband by 7 percentage points relative to a control group mean of 27 percent (a 26 percent reduction); being slapped by the husband (11 percentage point decrease relative to 33 percent control group mean, a 33 percent reduction); being punched (6 percentage point decrease relative to 15 percent control group mean, a 39 percent reduction); and being kicked, dragged, or beaten (8 percentage point decrease relative to 15 percent control group mean, a 51 percent reduction).”
Green, Wilke, and Cooper, by contrast, report that “screenings reduced the average rate at which women in treatment communities experienced violence by roughly .15 to .34 of an incident, from a baseline of just over half an incident on average...[which] fall short of significance.” However, “the campaign reduces the village-level proportion of women respondents who report any violence in their household by seven percentage points.”
Reading these two papers side by side, my overall impression is that cash is probably better, for an individual woman, at reducing IPV than a mass media campaign. So I am wondering how you got to a conclusion that mass media is 50x more effective? I am assuming it has something to do with cost of delivery, but to get from a smaller effect size to a better cost benefit-analysis would require the intervention to be incredibly cheap to deliver on a per-person basis—I’m ballparking it at like $10. Perhaps this is possible once you’ve achieved massive scale and content you’re happy with, but both of those things are going to take time and money, and the project might fail along the way, all of which raise costs in expectation. Giving a dollar to GiveDirectly, by contrast, leads to about 89c going to a person in extreme poverty right away. So I’d love to hear your thinking on this question spelled out in more detail.
You write: “The evidence on the effectiveness of mass media in IPV is limited but cautiously optimistic: While only one experimental study measured effects on behavior, four other experimental and quasi-experimental studies found sizable shifts in attitudes and norms related to IPV.” How well do you think those norms are correlated with IPV, and how well do you think that changing one will change the other?
Another way to frame this: Going back to Haushofer et al., the authors posit a very different theory, which is that men who commit IPV in the sample do so mainly to extract financial resources from their spouses—in other words, because they’re poor, which is not an attitude problem. How do you reconcile these two perspectives?
Thanks in advance, I really look forward to seeing this all come together!
Dear Seth,
Thank you for your support and thoughtful comments! Apologies for the delay – it’s a really busy time for us.
To clarify, we aimed for 5x GiveWell bar for health policy ideas—the text is an example of our goals in our research process. For this round, the bar needed to be more formally established.
Our claim for the idea around IPV is that the modeled cost-effectiveness for a “hypothetical 5-year intervention in Lesotho, Rwanda, Angola, and Ethiopia revealed a cost per disability-adjusted life year (DALY) averted ranging from $28 to $1419”. The wide range is because it is highly unclear how one should model the DALY burden of IPV, among other reasons. Our full report is published on our website now, so you can further dig into our thinking behind this. Of note, GiveWell tends to use a cost-effectiveness bar of $100-150 per DALY averted (which they claim to be ~8X GiveDirectly, note this is no longer the case given OP updates), which means that our estimates for this charity fall within the range of 40-0.5x GiveDirectly in cost-effectiveness. However, we aren’t sure if GiveWell has directly included these IPV benefits in its evaluation of GiveDirectly’s program.
Regardless, thanks for bringing this Haushofer et al. (2019) paper to our attention, We hadn’t seen this paper during our research, which is a pity. I cannot look into the paper in detail now; hopefully, my above clarification shows that we aren’t directly claiming a “ 50x over GD” bar. What you say sounds roughly true—I’d expect cash transfers to be far more expensive per person (mass media would be cents per person reached) but more effective (if the effect holds—I haven’t looked at the paper in any detail).
On your second point. We write a bit more about our sense of what the mechanism behind the intervention is in the report. To summarise—there is no clear sense in the literature about the main accepted mechanism for reducing violence or the true causal link between attitudes/norms and the prevalence of violence. It’s plausible that finances are a part of this, but it is likewise plausible that patriarchal norms, acceptability, etc., play a role. Different papers point in different directions, and this is an area that would benefit a lot from further research so that we can understand what works best. The context will matter a lot as well in terms of what drives violence and what type of violence is committed.
Good stuff, thanks!
FWIW I am a co-author on a meta-analysis of interventions to reduce sexual violence, where we found a disappointing lack of correlation between changes in stated attitudes and change in behavior.
I look forward to seeing this come together!
P.S. another two studies on this subject: https://journals.sagepub.com/doi/abs/10.1177/00104140221139385 (found significant effects on attitudes towards early and forced marriage) and https://www.journals.uchicago.edu/doi/abs/10.1086/726964?journalCode=jop (found “sporadic” effects on attitudes)