It looks like this estimate comes from the proportion of countries the Bloomberg consortium and World Bank worked in that passed various policies over a decade without adjusting for the counterfactual chance of policy changes without their work.
I’m curious if CE had any luck trying to estimate the counterfactual (Eg by looking at other countries, trends before BB, or diving deep on individual case studies)?
Fwiw when I looked at this a few years ago (at GiveWell, not OP) I couldn’t find any evidence of a difference in policy change by comparing countries BB worked in more vs less.
I don’t think this is good evidence against impact (policy spillovers, selection, BB’s global work, and no data before BB make it difficult to do this comparison) but it made me think unfortunately we can’t learn much about the counterfactual from the high level quantitative comparison and made me want to fall back on going deep on qualitative case studies instead.
You can find CE’s Research Report here: https://3394c0c6-1f1a-4f86-a2db-df07ca1e24b2.filesusr.com/ugd/26c75f_2081c09f8f20405e89105ac88c01ec6d.pdf
Thanks, this looks like a helpful report!
It looks like this estimate comes from the proportion of countries the Bloomberg consortium and World Bank worked in that passed various policies over a decade without adjusting for the counterfactual chance of policy changes without their work.
I’m curious if CE had any luck trying to estimate the counterfactual (Eg by looking at other countries, trends before BB, or diving deep on individual case studies)?
Fwiw when I looked at this a few years ago (at GiveWell, not OP) I couldn’t find any evidence of a difference in policy change by comparing countries BB worked in more vs less.
I don’t think this is good evidence against impact (policy spillovers, selection, BB’s global work, and no data before BB make it difficult to do this comparison) but it made me think unfortunately we can’t learn much about the counterfactual from the high level quantitative comparison and made me want to fall back on going deep on qualitative case studies instead.