I wrote this piece and wanted to offer my $0.02 on Hawthorne effects driving these consumption spillover results as it’s not covered in the report. I don’t think this is likely to be a key driver of the large spillovers reported, for two reasons:
To measure consumption spillovers, Egger et al. is essentially comparing consumption in nearby non-recipient households (e.g. <2km away) to consumption in further away non-recipient households (e.g. 10km). For this to produce biased results, you’d have to think the nearer non-recipients are gaming their answers in a way that the further away non-recipients aren’t. That seems plausible to me – but it also seems plausible that the further away non-recipients will still be aware of the program (so might have similar, counterbalancing incentives)
Even if you didn’t buy this, I’m not convinced the bias would be in the direction you’re implying. The program studied in Egger et al. was means-tested – cash transfers were only given to households with thatched roofs. If you think nearby non-recipients are more likely to be gaming the system, it seems plausible to me that they’d infer poorer households are more likely to get cash, so it makes sense for them to understate their consumption. This would downward bias the results
Hawthorne effects for recipient consumption gains seem more intuitively concerning to me, and I’ve been wondering whether this could be part of the story behind these large recipient consumption gains at 5-7 years we’ve been sent. We’re not putting much weight on these results at the moment as they’ve not been externally scrutinized, but it’s something I plan to think more about if/when we revisit these.
Thanks for that extra info. On Egger I agree that many of the effects found there don’t seem to be Hawthorne Effects—the bit that seems most credible to me is that non-recipients report working more hours on averagewhich seems consistent with a theory that they are getting real benefit from a cash injection into a cash constrained village). I also agree with your conclusion they’re unlikely to overstate their income/consumption, though there might be a stronger incentive for them to make their consumption choices seem sensible[1]
I have questions about how sustainable the gains are and who the losers are,[2] but also think the Egger characterisation of villages as small open economies with under-utilised resources (so the real welfare gains vastly outweigh local inflation) sounds plausible.
The roof methodology is probably a potentially Hawthorne Effect effect with more cause for concern: if potential recipients figure it out, the result might be certain households deferring a sensible investment in a tin roof in the hope this makes them eligible for future benefactor gifts. From a total welfare perspective those marginal poor families might be able to use their accumulated savings in other ways [nearly] as positive as the roof investment, but there are still arguments about perverse incentives
I realise people in villages in developing countries generally don’t have many opportunities to waste money on temptation goods benefactors would disapprove of, but at the margin...
an obvious potential example, given GD’s approach, would be future losses to the thatching industry (likely also composed of local poor people). Depending on how easily they can redeploy skills and resources, their losses from no longer being paid for rethatching might be comparable in magnitude to the welfare gains of families finally able to afford a roof that lasts.
Hi Nick & David,
I wrote this piece and wanted to offer my $0.02 on Hawthorne effects driving these consumption spillover results as it’s not covered in the report. I don’t think this is likely to be a key driver of the large spillovers reported, for two reasons:
To measure consumption spillovers, Egger et al. is essentially comparing consumption in nearby non-recipient households (e.g. <2km away) to consumption in further away non-recipient households (e.g. 10km). For this to produce biased results, you’d have to think the nearer non-recipients are gaming their answers in a way that the further away non-recipients aren’t. That seems plausible to me – but it also seems plausible that the further away non-recipients will still be aware of the program (so might have similar, counterbalancing incentives)
Even if you didn’t buy this, I’m not convinced the bias would be in the direction you’re implying. The program studied in Egger et al. was means-tested – cash transfers were only given to households with thatched roofs. If you think nearby non-recipients are more likely to be gaming the system, it seems plausible to me that they’d infer poorer households are more likely to get cash, so it makes sense for them to understate their consumption. This would downward bias the results
Hawthorne effects for recipient consumption gains seem more intuitively concerning to me, and I’ve been wondering whether this could be part of the story behind these large recipient consumption gains at 5-7 years we’ve been sent. We’re not putting much weight on these results at the moment as they’ve not been externally scrutinized, but it’s something I plan to think more about if/when we revisit these.
Hi Adam
Thanks for that extra info. On Egger I agree that many of the effects found there don’t seem to be Hawthorne Effects—the bit that seems most credible to me is that non-recipients report working more hours on average which seems consistent with a theory that they are getting real benefit from a cash injection into a cash constrained village). I also agree with your conclusion they’re unlikely to overstate their income/consumption, though there might be a stronger incentive for them to make their consumption choices seem sensible[1]
I have questions about how sustainable the gains are and who the losers are,[2] but also think the Egger characterisation of villages as small open economies with under-utilised resources (so the real welfare gains vastly outweigh local inflation) sounds plausible.
The roof methodology is probably a potentially Hawthorne Effect effect with more cause for concern: if potential recipients figure it out, the result might be certain households deferring a sensible investment in a tin roof in the hope this makes them eligible for future benefactor gifts. From a total welfare perspective those marginal poor families might be able to use their accumulated savings in other ways [nearly] as positive as the roof investment, but there are still arguments about perverse incentives
I realise people in villages in developing countries generally don’t have many opportunities to waste money on temptation goods benefactors would disapprove of, but at the margin...
an obvious potential example, given GD’s approach, would be future losses to the thatching industry (likely also composed of local poor people). Depending on how easily they can redeploy skills and resources, their losses from no longer being paid for rethatching might be comparable in magnitude to the welfare gains of families finally able to afford a roof that lasts.