My main critique of this process is that I have very little intuition for the ex post counterfactual impact of research unless I’ve thought about it deeply, because I think most of the impact of research is very heavy-tailed and depends on whether very specific lines of impact materialize (e.g. “this model of information theory coming out earlier than other models of information theory increases/decreases more dangerous paradigms in AI,” or “this specific funder made a counterfactually good/bad decision as a result of this research report” or “this specific set of people got more involved in EA as a result of this research blog post.”)
But of course this could just be copium for my own shitty orderings.
My main critique of this process is that I have very little intuition for the ex post counterfactual impact of research unless I’ve thought about it deeply, because I think most of the impact of research is very heavy-tailed and depends on whether very specific lines of impact materialize (e.g. “this model of information theory coming out earlier than other models of information theory increases/decreases more dangerous paradigms in AI,” or “this specific funder made a counterfactually good/bad decision as a result of this research report” or “this specific set of people got more involved in EA as a result of this research blog post.”)
But of course this could just be copium for my own shitty orderings.