Here’s an idea on how funders in AI safety and governance could help applicants improve their applications and projects: Share statistics!
(What follows is a text I’m also sending directly to a funder.)
I understand you aren’t really able to give individualized feedback. Though, as applicants, it would be really helpful to have some more clarity. I think you’d like to give more feedback, if you were able. In thinking about this, here’s an idea I had.
You could create a score chart, where for every application you keep numerical or binary scores on the reasons it did well or not in the evaluation. Then you can release some statistics publicly.
You could for example be able to say things like:
“30% of applications were filtered out because they were not pursuing our AI safety objectives”
“20% we would fund if we had more money”
“40% we’d investigate more deeply if we had more money”
“x% were too hard to understand”
“x% had some flaw in the theory of change”
“x% …”
(I picked more negative than positive slices in these examples, but the positive side is just as interesting.)
In fact, some of these statistics would be helpful for donors to <funder>.
The scores should track some pretty different kinds of dimensions.
Would this be hard to do? I think there can be some pretty different levels of ambition here and some are pretty easy. Releasing just a few pieces of statistics for example might be possible to do based on your current tracking.
Potentially, you could even let people request access to some subset of these scores for their application specifically. Something like, the email used to send the application can send an email to request an automated response with the scores you’d be prepared to divulge.
Could this create more opportunities for gaming? Well yes, but assuming your criteria are actually good proxies for value, then you also achieve: (1) Better applications (so you get to grant valuable things you might filter otherwise), and (2) Better projects (people make their projects have better theories of change etc).
The lack of two-way communication in funding seems like a large missed opportunity to me!
Grantmakers, even when you don’t grant, wield a lot of influence. You shape incentives in the ecosystem.
Here’s an idea on how funders in AI safety and governance could help applicants improve their applications and projects: Share statistics! (What follows is a text I’m also sending directly to a funder.)
I understand you aren’t really able to give individualized feedback. Though, as applicants, it would be really helpful to have some more clarity. I think you’d like to give more feedback, if you were able. In thinking about this, here’s an idea I had.
You could create a score chart, where for every application you keep numerical or binary scores on the reasons it did well or not in the evaluation. Then you can release some statistics publicly.
You could for example be able to say things like:
“30% of applications were filtered out because they were not pursuing our AI safety objectives”
“20% we would fund if we had more money”
“40% we’d investigate more deeply if we had more money”
“x% were too hard to understand”
“x% had some flaw in the theory of change”
“x% …”
(I picked more negative than positive slices in these examples, but the positive side is just as interesting.)
In fact, some of these statistics would be helpful for donors to <funder>. The scores should track some pretty different kinds of dimensions.
Would this be hard to do? I think there can be some pretty different levels of ambition here and some are pretty easy. Releasing just a few pieces of statistics for example might be possible to do based on your current tracking.
Potentially, you could even let people request access to some subset of these scores for their application specifically. Something like, the email used to send the application can send an email to request an automated response with the scores you’d be prepared to divulge.
Could this create more opportunities for gaming? Well yes, but assuming your criteria are actually good proxies for value, then you also achieve: (1) Better applications (so you get to grant valuable things you might filter otherwise), and (2) Better projects (people make their projects have better theories of change etc).
The lack of two-way communication in funding seems like a large missed opportunity to me!
Grantmakers, even when you don’t grant, wield a lot of influence. You shape incentives in the ecosystem.