This is a great idea. The only tweak I’d suggest is to let people do data analysis that’s not in the pre-registration as long as it’s clearly marked. Exploratory data analysis doesn’t have to be a bad; if nothing else it can be a good way to get ideas for future studies.
Even though this seems like a pretty clear improvement over the status quo, it also feels like a band-aid. In the worst case I could imagine data falsification becoming more common if data analysis pre-registration became required. Funding-starved academics desperately need to create interesting publishable results (“publish or perish”), and you removed one of the main things that made this possible.
I would like to see more people write about what academia might look like if it was rethought from the ground up. OpenPhil has $10 billion, and funding is to academics what candy is to kids. If OpenPhil tells researchers they have to work a particular way, my guess is they will do it. This could be a really valuable opportunity to experiment with alternative models.
This is a great idea. The only tweak I’d suggest is to let people do data analysis that’s not in the pre-registration as long as it’s clearly marked. Exploratory data analysis doesn’t have to be a bad; if nothing else it can be a good way to get ideas for future studies.
Even though this seems like a pretty clear improvement over the status quo, it also feels like a band-aid. In the worst case I could imagine data falsification becoming more common if data analysis pre-registration became required. Funding-starved academics desperately need to create interesting publishable results (“publish or perish”), and you removed one of the main things that made this possible.
I would like to see more people write about what academia might look like if it was rethought from the ground up. OpenPhil has $10 billion, and funding is to academics what candy is to kids. If OpenPhil tells researchers they have to work a particular way, my guess is they will do it. This could be a really valuable opportunity to experiment with alternative models.