I think all of these effects are plausible. However, I’m not a big fan of non-pre-registered RCTs, which appear to be most, if not all of the RCTs in that review paper. See: the section Our priors about sleep research should be weak
There is a question of how the evidence was created:
If someone flips an unknown amount of coins and decides to tell you about 100 of the results, and you know that this person has an incentive for you to believe most coins turn out heads, how do you update?
If you observe 6 coin flips (which are all you can find, but there is no force, probably, filtering which coin tosses you see towards heads/tails), how do you update?
Do you give the 100 coins vastly more weight and base your prior on them?
I think all of these effects are plausible. However, I’m not a big fan of non-pre-registered RCTs, which appear to be most, if not all of the RCTs in that review paper. See: the section Our priors about sleep research should be weak
I’m not sure why you accept as evidence non-pre-registered non-RCTs but do not accept non-pre-registered RCTs.
I don’t think I can explain this to you.
Here’s my attempt:
There is a question of how the evidence was created:
If someone flips an unknown amount of coins and decides to tell you about 100 of the results, and you know that this person has an incentive for you to believe most coins turn out heads, how do you update?
If you observe 6 coin flips (which are all you can find, but there is no force, probably, filtering which coin tosses you see towards heads/tails), how do you update?
Do you give the 100 coins vastly more weight and base your prior on them?
Thanks, makes sense.