Interesting poll Ryan! I’m not sure how much to take away because I think epistemic / evidentiary standards is pretty fuzzy in the minds of most readers. But still, point taken that people probably expect high standards.
It’s also rough to see that if we project the Egger regression line back to the origin then the predicted effect when the SE is zero is basically zero.
I’m not sure about that. Here’s the output of the Egger test. If I’m interpreting it correctly then that’s smaller, but not zero. I’ll try to figure out how what the p-curve suggested correction says.
Edit: I’m also not sure how much to trust the Egger test to tell me what the corrected effect size should be, so this wasn’t an endorsement that I think the real effect size should be halfed. It seems different ways of making this correction give very different answers. I’ll add a further comment with more details.
I do think going forward it would be worth taking seriously community expectations about what underlies charity recommendations, and if something is tentative or rough then I hope that it gets clearly communicated as such, both originally and in downstream uses.
Interesting poll Ryan! I’m not sure how much to take away because I think epistemic / evidentiary standards is pretty fuzzy in the minds of most readers. But still, point taken that people probably expect high standards.
I’m not sure about that. Here’s the output of the Egger test. If I’m interpreting it correctly then that’s smaller, but not zero. I’ll try to figure out how what the p-curve suggested correction says.
Edit: I’m also not sure how much to trust the Egger test to tell me what the corrected effect size should be, so this wasn’t an endorsement that I think the real effect size should be halfed. It seems different ways of making this correction give very different answers. I’ll add a further comment with more details.
Seems reasonable.
Fair re: Egger. I just eyeballed the figure.