Thank you, Aaron. This is an important point. I agree that it is worth putting out any work you have made out there. This is exactly the spirit of draft amnesty week! In this case, though, I had only conducted preparatory work to delineate the project. Continuing as if the other paper did not exist was not a realistic path, and my contribution would not have been meaningfully distinct. When I start a research project independently, I do it without much mentorship or institutional feedback, which makes it genuinely hard to assess whether I am working on the right question, using the right methods, or producing something useful to the field.
As someone reviewing applications, how do you evaluate independent research produced outside of a fellowship or academic context? Is there a threshold of rigor or novelty below which it hurts more than it helps to include it? And do you have suggestions for how early-career people in this transition can get lightweight feedback on research directions before investing weeks into a project? I ask because the fellowship application process often requires demonstrating prior work to get the mentorship needed to produce prior work, and I am trying to figure out how to navigate this as efficiently as possible.
As someone reviewing applications, how do you evaluate independent research produced outside of a fellowship or academic context?
I’m not sure how to answer this. I try to evaluate all work based on its quality; whether a project was single-authored also matters a fair amount (and high quality single-author work is an especially strong signal).
Is there a threshold of rigor or novelty below which it hurts more than it helps to include it?
Maybe for rigor, probably not for novelty. Applicants and other researchers should of course be up front about what are their contributions.
And do you have suggestions for how early-career people in this transition can get lightweight feedback on research directions before investing weeks into a project?
Weeks sound like it might be a lot. I encourage people to do Apart Research Sprints or other Hackathon-style things which are shorter. I’m not really sure about getting lightweight feedback. In my experience, when junior people ask me for feedback on a project idea, the project idea is usually too broad or vague for me to know if it’s a good project, and they have usually put less than 30 min of effort into it. So maybe my advice there is something like “if you are going to ask a more established researcher for feedback on your project plan, you should have already put a couple hours into the project, including surveying the relevant literature, coming up with a detailed project plan, and doing a little bit of de-risking”. I’m not sure, maybe that’s more intense than I endorse. Fortunately, even without the goal of getting feedback from somebody else, these are useful steps to begin a project with.
I will also note that prior work does not always have to be extremely relevant. Academia exists and is by far the place where the most people learn research skills.
Thank you, Aaron. This is an important point. I agree that it is worth putting out any work you have made out there. This is exactly the spirit of draft amnesty week! In this case, though, I had only conducted preparatory work to delineate the project. Continuing as if the other paper did not exist was not a realistic path, and my contribution would not have been meaningfully distinct. When I start a research project independently, I do it without much mentorship or institutional feedback, which makes it genuinely hard to assess whether I am working on the right question, using the right methods, or producing something useful to the field.
As someone reviewing applications, how do you evaluate independent research produced outside of a fellowship or academic context? Is there a threshold of rigor or novelty below which it hurts more than it helps to include it? And do you have suggestions for how early-career people in this transition can get lightweight feedback on research directions before investing weeks into a project? I ask because the fellowship application process often requires demonstrating prior work to get the mentorship needed to produce prior work, and I am trying to figure out how to navigate this as efficiently as possible.
I’m not sure how to answer this. I try to evaluate all work based on its quality; whether a project was single-authored also matters a fair amount (and high quality single-author work is an especially strong signal).
Maybe for rigor, probably not for novelty. Applicants and other researchers should of course be up front about what are their contributions.
Weeks sound like it might be a lot. I encourage people to do Apart Research Sprints or other Hackathon-style things which are shorter. I’m not really sure about getting lightweight feedback. In my experience, when junior people ask me for feedback on a project idea, the project idea is usually too broad or vague for me to know if it’s a good project, and they have usually put less than 30 min of effort into it. So maybe my advice there is something like “if you are going to ask a more established researcher for feedback on your project plan, you should have already put a couple hours into the project, including surveying the relevant literature, coming up with a detailed project plan, and doing a little bit of de-risking”. I’m not sure, maybe that’s more intense than I endorse. Fortunately, even without the goal of getting feedback from somebody else, these are useful steps to begin a project with.
I will also note that prior work does not always have to be extremely relevant. Academia exists and is by far the place where the most people learn research skills.