We had a hard time estimating the total impact of this research. It laid out an interesting case that modafinil is a reasonable alternative to caffeine, at least setting aside social factors. It did not seem to credibly address the main empirical questions that would motivate us to adopt either modafinil or caffeine, and we expect that most readers would be similarly skeptical (if they were sufficiently open-minded to plausibly take modafinil on the basis of analysis). We thought about evaluation by considering how hard it would be to produce a similar amount of value by paying for empirical research or critical review that would help clarify the benefits of modafinil and potentially push adoption.
Our very crude evaluation:
We had a hard time estimating the total impact of this research. It laid out an interesting case that modafinil is a reasonable alternative to caffeine, at least setting aside social factors. It did not seem to credibly address the main empirical questions that would motivate us to adopt either modafinil or caffeine, and we expect that most readers would be similarly skeptical (if they were sufficiently open-minded to plausibly take modafinil on the basis of analysis). We thought about evaluation by considering how hard it would be to produce a similar amount of value by paying for empirical research or critical review that would help clarify the benefits of modafinil and potentially push adoption.