This is very neat, thanks for sharing! Some comments.
First, the term “academic research” is used a lot in the text. Does this modeling speak to a need for more academic research or more research and development more generally?
It’s not clear to me that academia (as opposed to say, the nonprofit sector or even for-profit business) can claim the best track record of creating effective interventions. Academia might be involved at early stages but interventions often transition out of the academy when they scale up.
In other sectors, development (after the basic science research is done) costs the most and makes a product commercially viable. Maybe a majority of R&D spending should go towards development of interventions after initial proof-of-concept research is done?
Secondly, this modeling seems sensitive to assumptions about the efficacy of new interventions.
″...we found that, on average, investing at least 50% of the initial annual budget in scientific research is optimal even if the new interventions are only about half as cost-effective as the best existing intervention, on average…”
One could argue that new interventions are unlikely to be, on average, even half as effective as the best existing interventions, given that the best current interventions are recognized to be outliers (maybe even extreme outliers). Could you use some historical data to model average effectiveness of new interventions? There is a lot of cost-effectiveness data out there for public health interventions.
The simulations do not distinguish between scientific research and R&D projects outside of academia. The relative usefulness of these two types of research is beyond the scope of the model. The main assumption of the simulations is that the research projects are selected strategically for their potential to enable or produce more cost-effective interventions.
I agree that the assumption about the cost-effectiveness of new interventions can and should be validated empirically. Estimating it from historical data is an important direction for future work, and I am planning to pursue it. I think the expected cost-effectiveness will be vastly different depending on the extent to which the research builds on established knowledge and techniques. In the extreme case of refining the best existing intervention, the expected cost-effectiveness of the new intervention would definitely be larger than 50%.
Thanks, your points make a lot of sense to me! The case does seem to be stronger for R&D generally and it’s helpful to know that you’re not arguing for investment in a specific stage of research. I also agree that targeting existing interventions for improvement could be very high yield :).
This is very neat, thanks for sharing! Some comments.
First, the term “academic research” is used a lot in the text. Does this modeling speak to a need for more academic research or more research and development more generally?
It’s not clear to me that academia (as opposed to say, the nonprofit sector or even for-profit business) can claim the best track record of creating effective interventions. Academia might be involved at early stages but interventions often transition out of the academy when they scale up.
In other sectors, development (after the basic science research is done) costs the most and makes a product commercially viable. Maybe a majority of R&D spending should go towards development of interventions after initial proof-of-concept research is done?
Secondly, this modeling seems sensitive to assumptions about the efficacy of new interventions.
One could argue that new interventions are unlikely to be, on average, even half as effective as the best existing interventions, given that the best current interventions are recognized to be outliers (maybe even extreme outliers). Could you use some historical data to model average effectiveness of new interventions? There is a lot of cost-effectiveness data out there for public health interventions.
Thank you for your insightful comments, Marshall!
The simulations do not distinguish between scientific research and R&D projects outside of academia. The relative usefulness of these two types of research is beyond the scope of the model. The main assumption of the simulations is that the research projects are selected strategically for their potential to enable or produce more cost-effective interventions.
I agree that the assumption about the cost-effectiveness of new interventions can and should be validated empirically. Estimating it from historical data is an important direction for future work, and I am planning to pursue it. I think the expected cost-effectiveness will be vastly different depending on the extent to which the research builds on established knowledge and techniques. In the extreme case of refining the best existing intervention, the expected cost-effectiveness of the new intervention would definitely be larger than 50%.
Thanks, your points make a lot of sense to me! The case does seem to be stronger for R&D generally and it’s helpful to know that you’re not arguing for investment in a specific stage of research. I also agree that targeting existing interventions for improvement could be very high yield :).