I’m generally in favour of experimenting with different granting models and am glad to hear that funders are starting to experiment with random allocation. However, I’d be a little bit cautious about moving to a system based solely on random grant assignment. Depending on the actual grant success rate per round (currently often <20%), it seems likely that one would get awarded grants quite infrequently, which would interrupt the continuity of research. For instance, if somebody gets a random grant and makes an interesting discovery, it seems silly to then expect to wait several years for another random grant assignment to follow up on it. So I feel that random assignment is probably better used for assigning funding for early-career researchers or pilot projects.
With respect to quality control, the Nature news article linked above notes:
assessment panels spend most of their time sorting out the specific order in which to place mid-ranking ideas. Low- and high-quality applications are easy to rank, she says. “But most applications are in the midfield, which is very big.”
The current modified lottery systems just remove the low-ranking applications, but if it’s easy to pick high-ranking applications, surely they should be given funding priority?
Its true that this is probably most suited to a funding scheme aimed at early researchers due to the limitations mentioned by you. However, I might think that the grant success might go up if you use a model were you sort out all bad research first, because your 20 % is probably relate to the overall number of applications. Or maybe you could give people more tickets in the lottery if they have proven they can produce good research. However, this might introduce new biases.
In addition, it might still be a good approach for intermediate researchers because the overall time for the whole grant process gets reduced dramatically if you can cut out most of the peer review, which might lead to more calls for research proposals.
Concerning the Nature article and the modified lottery system: I read conflicting opinions on this. While the Nature article states that very good research can be identified easily, there are also others that state that researchers can only reliably identify bad research, but have a hard time to sort good research in any reproducible way.
I’m generally in favour of experimenting with different granting models and am glad to hear that funders are starting to experiment with random allocation. However, I’d be a little bit cautious about moving to a system based solely on random grant assignment. Depending on the actual grant success rate per round (currently often <20%), it seems likely that one would get awarded grants quite infrequently, which would interrupt the continuity of research. For instance, if somebody gets a random grant and makes an interesting discovery, it seems silly to then expect to wait several years for another random grant assignment to follow up on it. So I feel that random assignment is probably better used for assigning funding for early-career researchers or pilot projects.
With respect to quality control, the Nature news article linked above notes:
The current modified lottery systems just remove the low-ranking applications, but if it’s easy to pick high-ranking applications, surely they should be given funding priority?
Its true that this is probably most suited to a funding scheme aimed at early researchers due to the limitations mentioned by you. However, I might think that the grant success might go up if you use a model were you sort out all bad research first, because your 20 % is probably relate to the overall number of applications. Or maybe you could give people more tickets in the lottery if they have proven they can produce good research. However, this might introduce new biases.
In addition, it might still be a good approach for intermediate researchers because the overall time for the whole grant process gets reduced dramatically if you can cut out most of the peer review, which might lead to more calls for research proposals.
Concerning the Nature article and the modified lottery system: I read conflicting opinions on this. While the Nature article states that very good research can be identified easily, there are also others that state that researchers can only reliably identify bad research, but have a hard time to sort good research in any reproducible way.