I think the type of application you have in mind is when you’re hiring for a specific role? I think you’re right that there are circumstances where good evaluation is possible. Looking at what I wrote, I am making a too strong claim.
The type of applications I had in mind for that paragraph is things like accepting people to an AI Safety research program, or grants, and stuff like that. Although probably there are some lessens from hiring for specific roles that can be generalised to those situations.
Hmmmm. I’m wondering what part of the “selecting people for a job” model is transferrable and applicable to the “selecting people for a research program, grant, etc.”
In those circumstances, I’m guessing that there are specific criteria you are looking for, and it might just be a matter of shifting away from vibes & gut feelings and towards trying to verbalize/clarify what the criteria are. I’m guessing that even if you won’t have performance reviews for these people (like you would with employees), you still have an idea as to what counts as success.
Here is a hypothetical that might be worth exploring (this is very rough and was written in only a few minutes fairly off the top of my head, so don’t take t too seriously):
The next cohort for the AI Safety Camp is very large (large enough to be a good sample size for social science research), and X years in the future you look at all the individuals from that cohort to see what they are doing. The goals of AI Safety Camp are to provide people with both the motivation to work on AI safety and the skills to work on AI safety, so let’s see A) how many people in the cohort are working on AI safety, and B) how much they are contributing or how much of a positive impact they are having. Then we look at the applications that they submitted X years ago to join AI Safety camp, and see what criteria those applications have that they have.
I’m not good enough at data analysis to be able to pull much info, but there likely would be differences (of course, in reality it would have to be a pretty big sample size in order for any effects to not be overwhelmed by the random noise of life that has happened in the intervening X years). So although this little thought experiment is a bit silly and simplistic, we can still imagine the idea.
I think the type of application you have in mind is when you’re hiring for a specific role? I think you’re right that there are circumstances where good evaluation is possible. Looking at what I wrote, I am making a too strong claim.
The type of applications I had in mind for that paragraph is things like accepting people to an AI Safety research program, or grants, and stuff like that. Although probably there are some lessens from hiring for specific roles that can be generalised to those situations.
Hmmmm. I’m wondering what part of the “selecting people for a job” model is transferrable and applicable to the “selecting people for a research program, grant, etc.”
In those circumstances, I’m guessing that there are specific criteria you are looking for, and it might just be a matter of shifting away from vibes & gut feelings and towards trying to verbalize/clarify what the criteria are. I’m guessing that even if you won’t have performance reviews for these people (like you would with employees), you still have an idea as to what counts as success.
Here is a hypothetical that might be worth exploring (this is very rough and was written in only a few minutes fairly off the top of my head, so don’t take t too seriously):
I’m not good enough at data analysis to be able to pull much info, but there likely would be differences (of course, in reality it would have to be a pretty big sample size in order for any effects to not be overwhelmed by the random noise of life that has happened in the intervening X years). So although this little thought experiment is a bit silly and simplistic, we can still imagine the idea.