Thanks for writing this post and your efforts to address these issues! As someone who works in scientific research I have frustrations about the framework/criteria by which science is funded- so I’m really glad someone connected to effective altruism is looking into this, and I agree that this a really important cause area!
Often the criteria used to judge proposals is based on things like how innovative, novel, timely the science is, if it uses cutting-edge methodology and how suited the candidate is to that study area. Also, as a reviewer of proposals I am asked to judge the proposal’s ‘excellence’- a quite ambiguous quality- that accounts for significant randomness and bias too.
I think this approach to judge proposals is a limited as it means that a) potentially more important or impactful work will not be funded as it may lack novelty. b) people coming from a problem from a different discipline with differing expertise-yet many scientific breakthroughs come from those from other fields. c) it can lead funding of niche fields of science, to the detriment of neglected areas of greater importance and wider scale tractability.
Incorporation of the ITN framework would be beneficial, it may help fund more controversial or higher risk science and making decisions based on lotteries, after an initial sift, may also open up the field to more creativity and diversity, and counter some of the other biases that sadly occur. So I really agree with you here.
These issues above relate to the more open calls for proposals and how they are judged, but there are also many schemes one can apply to that are quite narrowly defined and have been decided by relatively few senior academics on a science board. It is not clear to many in the community how these grant calls are are decided, nor how rigorous and unbiased these are, making this process more transparent would be beneficial too. Your ideas based around cost-effectiveness and expected value might also provide more rigor to these decisions.
Anyway- really interested to see what you do with this, and let me know if I can be of help into the UK research council (UKRI) system. I’m currently part of the peer-review college for NERC and can find out more about specific protocols/decision making if useful.
Thanks for writing this post and your efforts to address these issues! As someone who works in scientific research I have frustrations about the framework/criteria by which science is funded- so I’m really glad someone connected to effective altruism is looking into this, and I agree that this a really important cause area!
Often the criteria used to judge proposals is based on things like how innovative, novel, timely the science is, if it uses cutting-edge methodology and how suited the candidate is to that study area. Also, as a reviewer of proposals I am asked to judge the proposal’s ‘excellence’- a quite ambiguous quality- that accounts for significant randomness and bias too.
I think this approach to judge proposals is a limited as it means that a) potentially more important or impactful work will not be funded as it may lack novelty. b) people coming from a problem from a different discipline with differing expertise-yet many scientific breakthroughs come from those from other fields. c) it can lead funding of niche fields of science, to the detriment of neglected areas of greater importance and wider scale tractability.
Incorporation of the ITN framework would be beneficial, it may help fund more controversial or higher risk science and making decisions based on lotteries, after an initial sift, may also open up the field to more creativity and diversity, and counter some of the other biases that sadly occur. So I really agree with you here.
These issues above relate to the more open calls for proposals and how they are judged, but there are also many schemes one can apply to that are quite narrowly defined and have been decided by relatively few senior academics on a science board. It is not clear to many in the community how these grant calls are are decided, nor how rigorous and unbiased these are, making this process more transparent would be beneficial too. Your ideas based around cost-effectiveness and expected value might also provide more rigor to these decisions.
Anyway- really interested to see what you do with this, and let me know if I can be of help into the UK research council (UKRI) system. I’m currently part of the peer-review college for NERC and can find out more about specific protocols/decision making if useful.
Thanks a lot for your comment and offer! I’ll send you a message =)