some promising signs about their ability to produce work that well-established external reviewers consider to be very high-quality—most notably, the acceptance of one of their decision theory papers to a top philosophy journal, The Journal of Philosophy.
I get that this is not the main case for the grant, and that MIRI generally avoids dealing with academia so this is not a great criterion to evaluate them on, but getting a paper accepted does not imply “very high-quality”, and having a single paper accepted in (I assume) a couple of years is an extremely low bar (e.g. many PhD students exceed this in terms of their solo output).
I’d heard that the particular journal had quite a high quality bar. Do you have a sense of whether that’s true or how hard it is to get into that journal? I guess we could just check the number of PhD students who get published in an edition of the journal to check the comparison.
I’d say most PhD students don’t publish in the Journal of Philosophy or other journals of a similar or better quality (it’s the fourth best general philosophy journal according to a poll by Brian Leiter).
This blog post seems to suggest it has an acceptance rate of about 5%.
I don’t know for sure, but at least in most areas of Computer Science it is pretty typical for at least Berkeley PhD students to publish in the top conferences in their area. (And they could publish in top journals; that just happens not to be as incentivized in CS.)
I generally dislike using acceptance rates—I don’t see strong reasons that they should correlate strongly with quality or difficulty—but top CS conferences have maybe ~25% acceptance rates, suggesting this journal would be 5x “harder”. This is more than I thought, but I don’t think it brings me to the point of thinking it should be a significant point in favor in an outside evaluation, given the size of the organization and the time period over which we’re talking.
Do you mean CS or ML? Because (I believe) ML is an especially new and ‘flat’ field where it doesn’t take as long to get to the cutting edge, so it probably isn’t representative.
I do mean CS and not just ML. (E.g. PLDI and OSDI are top conferences with acceptance rates of 27% and 18% respectively according to the first Google result, and Berkeley students do publish there.)
I get that this is not the main case for the grant, and that MIRI generally avoids dealing with academia so this is not a great criterion to evaluate them on, but getting a paper accepted does not imply “very high-quality”, and having a single paper accepted in (I assume) a couple of years is an extremely low bar (e.g. many PhD students exceed this in terms of their solo output).
I’d heard that the particular journal had quite a high quality bar. Do you have a sense of whether that’s true or how hard it is to get into that journal? I guess we could just check the number of PhD students who get published in an edition of the journal to check the comparison.
I’d say most PhD students don’t publish in the Journal of Philosophy or other journals of a similar or better quality (it’s the fourth best general philosophy journal according to a poll by Brian Leiter).
This blog post seems to suggest it has an acceptance rate of about 5%.
I don’t know for sure, but at least in most areas of Computer Science it is pretty typical for at least Berkeley PhD students to publish in the top conferences in their area. (And they could publish in top journals; that just happens not to be as incentivized in CS.)
I generally dislike using acceptance rates—I don’t see strong reasons that they should correlate strongly with quality or difficulty—but top CS conferences have maybe ~25% acceptance rates, suggesting this journal would be 5x “harder”. This is more than I thought, but I don’t think it brings me to the point of thinking it should be a significant point in favor in an outside evaluation, given the size of the organization and the time period over which we’re talking.
Not an expert but, fwiw, my impression is that this is more common in CS than philosophy and the social science areas I know best.
Do you mean CS or ML? Because (I believe) ML is an especially new and ‘flat’ field where it doesn’t take as long to get to the cutting edge, so it probably isn’t representative.
I do mean CS and not just ML. (E.g. PLDI and OSDI are top conferences with acceptance rates of 27% and 18% respectively according to the first Google result, and Berkeley students do publish there.)