As someone reviewing applications, how do you evaluate independent research produced outside of a fellowship or academic context?
I’m not sure how to answer this. I try to evaluate all work based on its quality; whether a project was single-authored also matters a fair amount (and high quality single-author work is an especially strong signal).
Is there a threshold of rigor or novelty below which it hurts more than it helps to include it?
Maybe for rigor, probably not for novelty. Applicants and other researchers should of course be up front about what are their contributions.
And do you have suggestions for how early-career people in this transition can get lightweight feedback on research directions before investing weeks into a project?
Weeks sound like it might be a lot. I encourage people to do Apart Research Sprints or other Hackathon-style things which are shorter. I’m not really sure about getting lightweight feedback. In my experience, when junior people ask me for feedback on a project idea, the project idea is usually too broad or vague for me to know if it’s a good project, and they have usually put less than 30 min of effort into it. So maybe my advice there is something like “if you are going to ask a more established researcher for feedback on your project plan, you should have already put a couple hours into the project, including surveying the relevant literature, coming up with a detailed project plan, and doing a little bit of de-risking”. I’m not sure, maybe that’s more intense than I endorse. Fortunately, even without the goal of getting feedback from somebody else, these are useful steps to begin a project with.
I will also note that prior work does not always have to be extremely relevant. Academia exists and is by far the place where the most people learn research skills.
I’m not sure how to answer this. I try to evaluate all work based on its quality; whether a project was single-authored also matters a fair amount (and high quality single-author work is an especially strong signal).
Maybe for rigor, probably not for novelty. Applicants and other researchers should of course be up front about what are their contributions.
Weeks sound like it might be a lot. I encourage people to do Apart Research Sprints or other Hackathon-style things which are shorter. I’m not really sure about getting lightweight feedback. In my experience, when junior people ask me for feedback on a project idea, the project idea is usually too broad or vague for me to know if it’s a good project, and they have usually put less than 30 min of effort into it. So maybe my advice there is something like “if you are going to ask a more established researcher for feedback on your project plan, you should have already put a couple hours into the project, including surveying the relevant literature, coming up with a detailed project plan, and doing a little bit of de-risking”. I’m not sure, maybe that’s more intense than I endorse. Fortunately, even without the goal of getting feedback from somebody else, these are useful steps to begin a project with.
I will also note that prior work does not always have to be extremely relevant. Academia exists and is by far the place where the most people learn research skills.