What a great study! I read through it and if I understand the PRESS statistic correctly, if you have a meta-analysis of impact evaluations, their average effect size predicts 33% of the variation of the next study run on the topic. So that’s not as great as I expected, but given the huge variations between countries and charities, it makes sense.
My takeaway is that since studies aren’t as generalizable as I would like, I should weight a study done on the charity I’m supporting much higher. This makes GiveDirectly a much stronger choice because it has a study run on it itself.
Another takeaway—studies done on studies are EA badass.
What a great study! I read through it and if I understand the PRESS statistic correctly, if you have a meta-analysis of impact evaluations, their average effect size predicts 33% of the variation of the next study run on the topic. So that’s not as great as I expected, but given the huge variations between countries and charities, it makes sense.
My takeaway is that since studies aren’t as generalizable as I would like, I should weight a study done on the charity I’m supporting much higher. This makes GiveDirectly a much stronger choice because it has a study run on it itself.
Another takeaway—studies done on studies are EA badass.