Thanks for this, a really nice write up. I like these heuristics, and will try to apply them.
On the intuition behind how to interpret statistical power, doesn’t a bayesian perspective help here?
If someone was conducting a statistical test to decide between two possibilities, and you knew nothing about their results except: (i) their calculated statistical power was B (ii) the statistical significance threshold they adopted was p and (iii) that they ultimately reported a positive result using that threshold, then how should you update on that, without knowing any more details about their data?
I think not having access to the data or reported effect sizes actually simplifies things a lot, and the Bayes factor you should update your priors by is just B/p (prob of observing this outcome if an effect / prob of observing this outcome if no effect). So if the test had half the power, the update to your prior odds of an effect should be half as big?
Thanks for this, a really nice write up. I like these heuristics, and will try to apply them.
On the intuition behind how to interpret statistical power, doesn’t a bayesian perspective help here?
If someone was conducting a statistical test to decide between two possibilities, and you knew nothing about their results except: (i) their calculated statistical power was B (ii) the statistical significance threshold they adopted was p and (iii) that they ultimately reported a positive result using that threshold, then how should you update on that, without knowing any more details about their data?
I think not having access to the data or reported effect sizes actually simplifies things a lot, and the Bayes factor you should update your priors by is just B/p (prob of observing this outcome if an effect / prob of observing this outcome if no effect). So if the test had half the power, the update to your prior odds of an effect should be half as big?
Yeah, I think a Bayesian perspective is really helpful here and this reply seems right.