The most prestigious publications like Nature and Science love to publish breakthroughs, but this also leads to sloppiness, like the paper on arsenic-based life that was retracted and the paper on induced stem cells that was retracted. When we see really bold publications like that, we always ask, “How long do you think till this is retracted?” On the other hand, there are also incentives to produce research with social benefit. It’s odd to complain about both breakthrough (i.e. basic, high-risk/high-reward) research and translational (i.e. applied, incremental) research being underfunded. You can’t have it both ways here.
There’s a similar contradiction in this: “It takes too long for people to recognise paradigm-shifts. Replication is poor.” The reason science is slow to change theories is because replication is poor. Individual studies have an inherent stochasticity, so one has to consider a body of work as a whole before being willing to shift the paradigm. (Note that Galileo’s and Mendel’s studies didn’t replicate initially.)
What I think are neglected are science that is ambitious, with low probability of success (e.g. Hsu’s cognitive genomics work, Ioannidis’ statistics work), and work bridging new research to humanitarian applications (e.g. using machine learning to classify medical images, or to detect online fraud or risks to security). These are overlapping sets.
In paradigm shifts, I mean adaptation to different ways of doing things. e.g. working to develop BCI and brain implants, or to apply deep learning in machine learning. These things have occurred too slowly. The replication problems are mostly in soft sciences like psychology, and arise from systemic problems with study pre-registration. The causation of these problems is somewhat entangled, but they’re not the exact same problem. Both should be fixed.
My point is: there are a range of important structural changes that effective altruists might want to make in science.
The most prestigious publications like Nature and Science love to publish breakthroughs, but this also leads to sloppiness, like the paper on arsenic-based life that was retracted and the paper on induced stem cells that was retracted. When we see really bold publications like that, we always ask, “How long do you think till this is retracted?” On the other hand, there are also incentives to produce research with social benefit. It’s odd to complain about both breakthrough (i.e. basic, high-risk/high-reward) research and translational (i.e. applied, incremental) research being underfunded. You can’t have it both ways here.
There’s a similar contradiction in this: “It takes too long for people to recognise paradigm-shifts. Replication is poor.” The reason science is slow to change theories is because replication is poor. Individual studies have an inherent stochasticity, so one has to consider a body of work as a whole before being willing to shift the paradigm. (Note that Galileo’s and Mendel’s studies didn’t replicate initially.)
What I think are neglected are science that is ambitious, with low probability of success (e.g. Hsu’s cognitive genomics work, Ioannidis’ statistics work), and work bridging new research to humanitarian applications (e.g. using machine learning to classify medical images, or to detect online fraud or risks to security). These are overlapping sets.
In paradigm shifts, I mean adaptation to different ways of doing things. e.g. working to develop BCI and brain implants, or to apply deep learning in machine learning. These things have occurred too slowly. The replication problems are mostly in soft sciences like psychology, and arise from systemic problems with study pre-registration. The causation of these problems is somewhat entangled, but they’re not the exact same problem. Both should be fixed.
My point is: there are a range of important structural changes that effective altruists might want to make in science.