I think I would stick to my guns on the sample size point, although I think you would agree with it if I had expressed it better in the OP.
I agree with you sample sizes of 200 (or 20, or less) can be good enough depending on the context. My core claim is these contexts do not obtain for lots of EA problems: the units vary a lot, the variance is explained by other factors than the one we’re interested in, and the variance explained by the intervention/factor of interest will be much smaller (i.e. high variance across units, small effect sizes).
[My intuition driving the confounders point is the balancing these doesn’t look feasible if they are sufficiently heavy-tailed (e.g. take all countries starting with A-C, and randomly assign to two arms, these arms will tend to have very large differences in (say) mean GDP), and the implied premise being lots of EA problems will be ones where factors like these are expected to have greater effect than the upper bound on the intervention. I may be off-base.]
Thanks for responding. I’ve now reread your post (twice) and I feel comfortable in saying that I twisted myself up reading it the first time around. I don’t think my comment is directly relevant to the point you’re making, and I’ve retracted it. The point is well-taken, and I think it holds up.
I think I would stick to my guns on the sample size point, although I think you would agree with it if I had expressed it better in the OP.
I agree with you sample sizes of 200 (or 20, or less) can be good enough depending on the context. My core claim is these contexts do not obtain for lots of EA problems: the units vary a lot, the variance is explained by other factors than the one we’re interested in, and the variance explained by the intervention/factor of interest will be much smaller (i.e. high variance across units, small effect sizes).
[My intuition driving the confounders point is the balancing these doesn’t look feasible if they are sufficiently heavy-tailed (e.g. take all countries starting with A-C, and randomly assign to two arms, these arms will tend to have very large differences in (say) mean GDP), and the implied premise being lots of EA problems will be ones where factors like these are expected to have greater effect than the upper bound on the intervention. I may be off-base.]
Thanks for responding. I’ve now reread your post (twice) and I feel comfortable in saying that I twisted myself up reading it the first time around. I don’t think my comment is directly relevant to the point you’re making, and I’ve retracted it. The point is well-taken, and I think it holds up.