NB: The Unjournal commissioned two expert evaluations of this work: see the full reports and ratings here.
From the first evaluation abstract (anonymous):
Pro:
Raises important points and brings them to wider attention in simple language.
Useful for considering individual RCTs.
Con:
Not clear enough about intended use cases and framing.
…
Guidelines need more clarity and precision before they can be genuinely used.
I think best to reframe this as a research note, rather than a ready-to-use ‘guideline’.
Unclear whether this is applicable to considering multiple studies and doing meta-analysis.
From the second evaluation abstract (by Max Maier):
The proposal makes an important practical contribution to the question of how to evaluate effect size estimates in RCTS. I also think overall the evaluation steps are plausible and well justified and will lead to a big improvement in comparison to using an unadjusted effect size. However, I am unsure whether they will lead to an improvement over simpler adjustment rules (e.g., dividing the effect size by 2) and see serious potential problems when applying this process in practice, especially related to the treatment of uncertainty.
I’d love to know if this work is being used or followed up on.
NB: The Unjournal commissioned two expert evaluations of this work: see the full reports and ratings here.
From the first evaluation abstract (anonymous):
From the second evaluation abstract (by Max Maier):
I’d love to know if this work is being used or followed up on.