Thanks for sharing your proposal Michael. The institute looks great. Finding ways to incentivise replication is something I consider to be really important.
A couple of questions. I am curious what probability you would place on the Institute significantly increasing acceptances of replications in top journals? More abstractly, I wonder if a dedicated instituted could help change social norms in academia around replication. Do you have any thoughts about this?
I’m not sure if top journals would publish replications. They seem to get prestige from publishing original research, but maybe if replication was higher status, they would do it. I mainly see the benefit of systematic replication in inducing researchers to improve the quality of their research, so we’d actually see fewer negative replications. (Another issue is that only negative replications are ‘interesting’.)
I think changing norms is possible. A lot of journals now have a data editor who ensures ‘push-button’ reproducibility: the data and code are available, and you can run a script that produces all of the results in the paper. This is a big improvement over 10-15 years ago when code wasn’t available, or didn’t reproduce results.
Thanks for sharing your proposal Michael. The institute looks great. Finding ways to incentivise replication is something I consider to be really important.
A couple of questions. I am curious what probability you would place on the Institute significantly increasing acceptances of replications in top journals? More abstractly, I wonder if a dedicated instituted could help change social norms in academia around replication. Do you have any thoughts about this?
Lastly, did you receive any feedback from FTX?
I’m not sure if top journals would publish replications. They seem to get prestige from publishing original research, but maybe if replication was higher status, they would do it. I mainly see the benefit of systematic replication in inducing researchers to improve the quality of their research, so we’d actually see fewer negative replications. (Another issue is that only negative replications are ‘interesting’.)
I think changing norms is possible. A lot of journals now have a data editor who ensures ‘push-button’ reproducibility: the data and code are available, and you can run a script that produces all of the results in the paper. This is a big improvement over 10-15 years ago when code wasn’t available, or didn’t reproduce results.
I didn’t get any feedback from FTX.