Filtering for obvious misfits, I think the majority reason is that I don’t think the project proposal will be sufficiently valuable for the long-term future, even if executed well. The minority reason is that there isn’t strong enough evidence that the project will be executed well.
Sorry if this is an unsatisfying answer—I think our applications are different enough that it’s hard to think of common reasons for rejection that are more granular. Also, often the bottom line is “this seems like it could be good, but isn’t as good as other things we want to fund”. Here are some more concrete kinds of reasons that I think have come up at least more than once:
Project seems good for the medium-term future, but not for the long-term future
Applicant wants to learn the answer to X, but X doesn’t seem like an important question to me
Applicant wants to learn about X via doing Y, but I think Y is not a promising approach for learning about X
Applicant proposes a solution to some problem, but I think the real bottleneck in the problem lies elsewhere
Applicant wants to write something for a particular audience, but I don’t think that writing will be received well by that audience
Project would be good if executed exceptionally well, but applicant doesn’t have a track record in this area, and there are no references that I trust to be calibrated to vouch for their ability
Applicant wants to do research on some topic, but their previous research on similar topics doesn’t seem very good
Applicant wants money to do movement-building, but several people have reported negative interactions with them
Hey Asya!
I’ve seen that you’ve received a comment prize on this. Congratulations! I have found it interesting.
I was wondering: you give these two reasons for rejecting a funding application
Project would be good if executed exceptionally well, but applicant doesn’t have a track record in this area, and there are no references that I trust to be calibrated to vouch for their ability.
Applicant wants to do research on some topic, but their previous research on similar topics doesn’t seem very good.
My question is: what method would you use to evaluate the track record of someone who has not done a Ph.D. in AI Safety, but rather on something like Physics (my case :) )? Do you expect the applicant to have some track record in AI Safety research?
I do not plan on applying for funding on the short term, but I think I would find some intuition on this valuable. I also ask because I find it hard to calibrate myself on the quality of my own research.
Hey! I definitely don’t expect people starting AI safety research to have a track record doing AI safety work—in fact, I think some of our most valuable grants are paying for smart people to transition into AI safety from other fields. I don’t know the details of your situation, but in general I don’t think “former physics student starting AI safety work” fits into the category of “project would be good if executed exceptionally well”. In that case, I think most of the value would come from supporting the transition of someone who could potentially be really good, rather than from the object-level work itself.
In the case of other technical Ph.D.s, I generally check whether their work is impressive in the context of their field, whether their academic credentials are impressive, what their references have to say. I also place a lot of weight on whether their proposal makes sense and shows an understanding of the topic, and on my own impressions of the person after talking to them.
I do want to emphasize that “paying a smart person to test their fit for AI safety” is a really good use of money from my perspective—if the person turns out to be good, I’ve in some sense paid for a whole lifetime of high-quality AI safety research. So I think my bar is not as high as it is when evaluating grant proposals for object-level work from people I already know.
Filtering for obvious misfits, I think the majority reason is that I don’t think the project proposal will be sufficiently valuable for the long-term future, even if executed well. The minority reason is that there isn’t strong enough evidence that the project will be executed well.
Sorry if this is an unsatisfying answer—I think our applications are different enough that it’s hard to think of common reasons for rejection that are more granular. Also, often the bottom line is “this seems like it could be good, but isn’t as good as other things we want to fund”. Here are some more concrete kinds of reasons that I think have come up at least more than once:
Project seems good for the medium-term future, but not for the long-term future
Applicant wants to learn the answer to X, but X doesn’t seem like an important question to me
Applicant wants to learn about X via doing Y, but I think Y is not a promising approach for learning about X
Applicant proposes a solution to some problem, but I think the real bottleneck in the problem lies elsewhere
Applicant wants to write something for a particular audience, but I don’t think that writing will be received well by that audience
Project would be good if executed exceptionally well, but applicant doesn’t have a track record in this area, and there are no references that I trust to be calibrated to vouch for their ability
Applicant wants to do research on some topic, but their previous research on similar topics doesn’t seem very good
Applicant wants money to do movement-building, but several people have reported negative interactions with them
Hey Asya! I’ve seen that you’ve received a comment prize on this. Congratulations! I have found it interesting. I was wondering: you give these two reasons for rejecting a funding application
Project would be good if executed exceptionally well, but applicant doesn’t have a track record in this area, and there are no references that I trust to be calibrated to vouch for their ability.
Applicant wants to do research on some topic, but their previous research on similar topics doesn’t seem very good.
My question is: what method would you use to evaluate the track record of someone who has not done a Ph.D. in AI Safety, but rather on something like Physics (my case :) )? Do you expect the applicant to have some track record in AI Safety research? I do not plan on applying for funding on the short term, but I think I would find some intuition on this valuable. I also ask because I find it hard to calibrate myself on the quality of my own research.
Hey! I definitely don’t expect people starting AI safety research to have a track record doing AI safety work—in fact, I think some of our most valuable grants are paying for smart people to transition into AI safety from other fields. I don’t know the details of your situation, but in general I don’t think “former physics student starting AI safety work” fits into the category of “project would be good if executed exceptionally well”. In that case, I think most of the value would come from supporting the transition of someone who could potentially be really good, rather than from the object-level work itself.
In the case of other technical Ph.D.s, I generally check whether their work is impressive in the context of their field, whether their academic credentials are impressive, what their references have to say. I also place a lot of weight on whether their proposal makes sense and shows an understanding of the topic, and on my own impressions of the person after talking to them.
I do want to emphasize that “paying a smart person to test their fit for AI safety” is a really good use of money from my perspective—if the person turns out to be good, I’ve in some sense paid for a whole lifetime of high-quality AI safety research. So I think my bar is not as high as it is when evaluating grant proposals for object-level work from people I already know.