Thanks, this is useful to hear. I think intuitively I lean towards your view of this being helpful and good to do. But the received wisdom among grantmakers as far as I can tell (that I have mostly been deferring to) is that it is bad to do this due to grantees overupdating. I think if grantees/applicants were perfectly even-keeled and rational this would be a great idea. And many applicants probably would benefit from it. But there will likely be a minority of cases where people (unreasonably) get upset if they don’t get funded after seeing an initial high probability. Not sure what to do about this, maybe we can be just like ‘tough luck, we put a disclaimer saying not to rely on this probability estimate overmuch’ but phrased more nicely.
I think the received wisdom is both outdated and perhaps based on other ecosystems like traditional mainstream grant-making outside of AI
Some thoughts
Even if some grantees overupdate, it may still have a benefit in net; I suspect it would do in the rationalist/EA/AI safety/quantitative spaces
That said, would be nice to tally and publish your calibration in this—e.g., what share of cases assessed at 90% actually get funded. Previously, this tracking was annoying, but with AI agents it should be prety easy
People are less likely to overupdate if you explicitly say that these probabilities were generated by an AI agent.
FWIW, I think interested potential grantees can guesstime these probabilities & the likely feedback themselves. They can just ask their favorite LLM. This has the advantage that the org doesn’t have to support this themselves, where it would give a false sense of confidence to possible grantees.
I agree that they applicants can and will compute the ex-ante probabilities in this way. But what I’m suggesting is that the grantmakers could share probabilities informed by the granters’
own internal discussions of priority areas and of the grant applications as as well as as, interim grant decisions (who have they already decided to fund).
The grant makers might reasonably be reluctant to share that internal discussion, but might be willing to share estimated probabilities implied by these.
I suspect there’s a lot of this private information that the applicants are unaware of.
Similarly for “which part of the applicants case are granters is already convinced by in which part needs more justification” — I suspect that applicants has major blindspots, and a tiny bit of hints and steering could help a lot
(I am a new-ish AI policy grantmaker)
Thanks, this is useful to hear. I think intuitively I lean towards your view of this being helpful and good to do. But the received wisdom among grantmakers as far as I can tell (that I have mostly been deferring to) is that it is bad to do this due to grantees overupdating. I think if grantees/applicants were perfectly even-keeled and rational this would be a great idea. And many applicants probably would benefit from it. But there will likely be a minority of cases where people (unreasonably) get upset if they don’t get funded after seeing an initial high probability. Not sure what to do about this, maybe we can be just like ‘tough luck, we put a disclaimer saying not to rely on this probability estimate overmuch’ but phrased more nicely.
I think the received wisdom is both outdated and perhaps based on other ecosystems like traditional mainstream grant-making outside of AI
Some thoughts
Even if some grantees overupdate, it may still have a benefit in net; I suspect it would do in the rationalist/EA/AI safety/quantitative spaces
That said, would be nice to tally and publish your calibration in this—e.g., what share of cases assessed at 90% actually get funded. Previously, this tracking was annoying, but with AI agents it should be prety easy
People are less likely to overupdate if you explicitly say that these probabilities were generated by an AI agent.
Thanks for sharing this.
FWIW, I think interested potential grantees can guesstime these probabilities & the likely feedback themselves. They can just ask their favorite LLM. This has the advantage that the org doesn’t have to support this themselves, where it would give a false sense of confidence to possible grantees.
I agree that they applicants can and will compute the ex-ante probabilities in this way. But what I’m suggesting is that the grantmakers could share probabilities informed by the granters’ own internal discussions of priority areas and of the grant applications as as well as as, interim grant decisions (who have they already decided to fund).
The grant makers might reasonably be reluctant to share that internal discussion, but might be willing to share estimated probabilities implied by these.
I suspect there’s a lot of this private information that the applicants are unaware of.
Similarly for “which part of the applicants case are granters is already convinced by in which part needs more justification” — I suspect that applicants has major blindspots, and a tiny bit of hints and steering could help a lot