Agree with this—it’s impossible to give constructive feedback on thousands of applications. The decision is between not giving grants, or accepting that most grant applications won’t get much feedback from us. We chose the latter.
I’d like to challenge this. There are simultaneous claims that:
it’s impossible to give constructive feedback on thousands of applications
It is possible to effectively (in an expected value sense) allocate $100m - $1b a year using this process which evaluates thousands of applications from a broad range of applicants related to a broad spectrum of ideas over just a two week period
I don’t think both can be true in the long run. Like others in the comments suggested both may be a question of further investment in and improvement of the process. There is a lot of room for improvement: any feedback is better than no feedback, it doesn’t have to be super constructive -just knowing if anyone even spent more than a minute looking at your application is useful info that applicants currently don’t have.
Wanting to be constructive: would there be arguments against hiring an extra person whose job is to observe the decision making process (I assume there is a kind of internal log of decisions/opinions), and formulate non-zero feedback on applications?
It would be very surprising if there weren’t an opportunity cost to providing feedback. Those might include:
Senior management time to oversee the project, bottlenecking other plans
PR firefighting and morale counselling when 1 in ~100 people get angry at what you say and cause you grief (this will absolutely happen)
Any hires capable of thinking up and communicating helpful feedback (this is difficult!) could otherwise use that time to read and make decisions on more grant proposals in more areas — or just improve the decision-making among the same pool of applicants.
That there’s an opportunity cost doesn’t show it’s not worth it but my guess is right now it would be huge mistake for Future Fund to provide substantial feedback except in rare cases.
That could change in future if their other streams of successful applicants dry up and improving the projects of people who were previously rejected becomes the best way to find new things they want to fund.
huge mistake for Future Fund to provide substantial feedback except in rare cases.
Yep, I’d imagine what makes sense is between ‘highly involved and coordinated attempt to provide feedback at scale’ and ‘zero’. I think it’s tempting to look away from how harmful ‘zero’ can be at scale
> That could change in future if their other streams of successful applicants dry up and improving the projects of people who were previously rejected becomes the best way to find new things they want to fund.
Agreed – this seems like a way to pick up easy wins and should be a good go-to for grant makers to circle back. However, banking on this as handling the concerns that were raised doesn’t account for all the things that come with unqualified rejection and people deciding to do other things, leave EA, incur critical stakeholder instability etc. as a result.
In other words, for the consequentialist-driven among us, I don’t think that community health is a nice-to-have if we’re serious about having a community of highly effective people working urgently on hard/complex things
“However, banking on this as handling the concerns that were raised doesn’t account for all the things that come with unqualified rejection and people deciding to do other things, leave EA, incur critical stakeholder instability etc. as a result. ”
I mean I think people are radically underestimating the opportunity cost of doing feedback properly at the moment. If I’m right then getting feedback might reduce people’s chances of getting funded by say, 30%, or 50%, because the throughput for grants will be much reduced.
I would probably rather have a 20% chance of getting funding for my project without feedback than a 10% chance with feedback, though people’s preferences may vary.
(Alternatively all the time spent explaining and writing and corresponding will mean worse projects get funded as there’s not much time left to actually think through which projects are most impactful.)
Rob, I think you’re consistently arguing against a point few people are making. You talk about ongoing correspondence with projects, or writing (potentially paragraphs of) feedback. Several people in this thread have suggested that pre-written categories of feedback would be a huge improvement from the status quo, and I can’t see anything you’ve said that actually argues against that.
Also, as someone who semi-regularly gives feedback to 80+ people, I’ve never found it to make my thinking worse, but I’ve sometimes found it makes my thinking better.
I’m not saying there’s no cost to feedback. Of course there’s a cost! But these exaggerations are really frustrating to read, because I actually do this kind of work and the cost of what I’m proposing is a lot lower than you keep suggesting.
I’ve got a similar feeling to Khorton. Happy to have been pre-empted there.
It could be helpful to consider what it is that legibility in the grant application process (for which post-application feedback is only one sort) is meant to achieve. Depending on the grant maker’s aims, this can non-exhaustively include developing and nurturing talent, helping future applicants self-select, orienting projects on whether they are doing a good job, being a beacon and marketing instrument, clarifying and staking out an epistemic position, serving an orientation function for the community etc.
And depending on the basket of things the grant maker is trying to achieve, different pieces of legibility affect ‘efficiency’ in the process. For example, case studies and transparent reasoning about accepted and rejected projects, published evaluations, criteria for projects to consider before applying, hazard disclaimers, risk profile declarations, published work on the grant makers theory of change, etc. can give grant makers ‘published’ content to invoke during the post-application process that allows for the scaling of feedback. (e.g. our website states that we don’t invest in projects that rapidly accelerate ‘x’). There are other forms of pro-active communication and stratifying applicant journeys that would make things even more efficient.
FTX did what they did, and there is definitely a strong case for why they did it that way. In moving forward , I’d be curious to see if they acknowledge and make adjustments in light of the fact that different forms and degrees of legibility can affect the community.
Okay, upon review, that was a little bit too much of a rhetorical flourish at the end. Basically, I think there’s something seriously important to consider here about how process can negatively affect community health and alignment, which I believe to be important for this community in achieving the plurality of ambitious goals we’re shooting for. I believe FTX could definitely affect in a very positive way if they wanted to
Thanks for your comment! I wanted to try to clarify a few things regarding the two claims you see us as making. I agree there are major benefits to providing feedback to applicants. But there are also significant costs, too, and I want to explain why it’s at least a non-obvious decision what the right choice is here.
On (1), I agree with Sam that it wouldn’t be the right prioritization for our team right now to give detailed feedback to >1600 applications we rejected, and would cut into our total output for the year significantly. I think it could be done if need be, but it would be really hard and require an innovative approach. So I don’t think we should be doing this now, but I’m not saying that we won’t try to find ways to give more feedback in the future (see below).
On (2), although we want to effectively allocate at least $100M this year, we don’t plan to do 100% of this using this particular process without growing our team. In our announcement post, we said we would try four different processes and see what works best. We could continue all, some, or none of them. We have given out considerably less than $100M via the open call (more in our progress update in a month or so); and, as I mentioned in another comment, for larger and/or more complex grants the investigation process often takes longer than two weeks.
On hiring someone to do this: I think there are good reasons for us not to hire an extra person whose job is to give feedback to everyone. Most importantly: there are lots of things we could hire for, I take early hiring decisions very seriously because they affect the culture and long-term trajectory of the organization, and we want to take those decisions slowly and deliberately. I also think it’s important to maintain a certain quality bar for this kind of feedback, and this would likely require significant oversight from the existing team.
Will we provide feedback to rejected applicants in the future? Possibly, but I think this involves complex tradeoffs and isn’t a no-brainer. I’ll try to explain some of the reasons I see it this way, even at scale. A simple and unfortunate reason is that there are a lot of opportunities for angry rejected applicants—most of whom we do not know at all and aren’t part of the effective altruism community—to play “gotcha” on Twitter (or with lawsuit threats) in response to badly worded feedback, and even if the chances of this happening are small for any single rejected application, the cumulative chances of this happening once are substantial if you’re giving feedback to thousands of people. (I think this may be why even many public-spirited employers and major funders don’t provide such feedback.) I could imagine a semi-standardized process that gave more feedback to people who wanted it and very nearly got funded. (A model that I heard TripleByte used sounds interesting to me.) We’ll have to revisit these questions the next time we have an open call, and we’ll take the conversation here into account—we really appreciate your feedback!
A model that I heard TripleByte used sounds interesting to me.
I wrote a comment about TripleByte’s feedback process here; this blog post is great too. In our experience, the fear of lawsuits and PR disasters from giving feedback to rejected candidates was much overblown, even at a massive scale. (We gave every candidate feedback regardless of how well they performed on our interview.)
Something I didn’t mention in my comment is that much of TripleByte’s feedback email was composed of prewritten text blocks carefully optimized to be helpful and non-offensive. While interviewing a candidate, I would check boxes for things like “this candidate used their debugger poorly”, and then their feedback email would automatically include a prewritten spiel with links on how to use a debugger well (or whatever). I think this model could make a lot of sense for the fund:
It makes giving feedback way more scalable. There’s a one-time setup cost of prewriting some text blocks, and probably a minor ongoing cost of gradually improving your blocks over time, but the marginal cost of giving a candidate feedback is just 30 seconds of checking some boxes. (IIRC our approach was to tell candidates “here are some things we think it might be helpful for you to read” and then when in doubt, err on the side of checking more boxes. For funding, I’d probably take it a step further, and rank or score the text blocks according to their importance to your decision. At TripleByte, we would score the candidate on different facets of their interview performance and send them their scores—if you’re already scoring applications according to different facets, this could be a cheap way to provide feedback.)
Minimize lawsuit risk. It’s not that costly to have a lawyer vet a few pages of prewritten text that will get reused over and over. (We didn’t have a lawyer look over our feedback emails, and it turned out fine, so this is a conservative recommendation.)
Minimize PR risk. Someone who posts their email to Twitter can expect bored replies like “yeah, they wrote the exact same thing in my email.” (Again, PR risk didn’t seem to be an issue in practice despite giving lots of freeform feedback along with the prewritten blocks, so this seems like a conservative approach to me.)
If I were you, I think I’d experiment with hiring one of the writers of the TripleByte feedback emails as a contractor or consultant. Happy to make an intro.
A few final thoughts:
Without feedback, a rejectee is likely to come up with their own theory of why they were rejected. You have no way to observe this theory or vet its quality. So I think it’s a mistake to hold yourself to a high bar. You just have to beat the rejectee’s theory. (BTW, most of the EA rejectee theories I’ve heard have been very cynical.)
You might look into liability insurance if you don’t have it already; it probably makes sense to get it for other reasons anyway. I’d be curious how the cost of insurance changes depending on the feedback you’re giving.
Will we provide feedback to rejected applicants in the future? Possibly, but I think this involves complex tradeoffs and isn’t a no-brainer
So I don’t think we should be doing this now, but I’m not saying that we won’t try to find ways to give more feedback in the future (see below).
Very much appreciate the considerate engagement with this. Wanted to flag that my primary response to your initial comment can be found here.
All this makes a lot of sense to me. I suspect some people got value out of the presentation of this reasoning. My goal here was to bring this set of consideration to yours and Sam’s attention and upvote its importance, hopefully it’s factored into what is definitely non-obvious and complex to decide moving forward. Great to see how thoughtful you all have been and thanks again!
Thanks for the response, and thanks for being open to improving your process, and I agree with many of your points about the importance of scaling teams cautiously.
I disagree that it’s impossible to give constructive feedback on 1700 applications.
I could imagine FTX Future Fund having a couple of standardized responses, rather than just one. For example:
Your application was rejected because based on the information provided it did not appear to be in scope for what we fund (link to the page that sets out what you fund)
Your application appears to be in scope for what we fund. We weren’t currently confident in the information provided about [theory of change / founding team / etc]. It might still be a good fit for another grantmaker. If you do decide to update that section, feel free to re-apply to a future round of funding.
potentially a response for applications you think are an especially bad idea?
It seems many of the downsides of giving feedback would also apply to this.
I think lower resolution feedback introduces new issues too. For example, people might become aware of the schema and over-index on getting a “1. Reject” versus getting a “2. Revise and resubmit”.
A major consideration is that I think some models of very strong projects and founders says that these people wouldn’t be harmed by rejections.
Further considerations related to this (that are a little sensitive) is that there are other ways of getting feedback, and that extremely impactful granting and funding is relationship based, not based on an instance of one proposal or project. This makes sense once you consider that grantees are EAs and should have very high knowledge of their domains in EA cause areas.
Thanks to Sam and Nick for getting to this. I think it’s very cool that you two are taking the time to engage. In light of the high esteem that I regard both of you and the value of your time, I’ll try to close the loop of this interaction by leaving you with one main idea.
I was pointing at something different than what I think was addressed. To distill what I was saying: >> Were FTX to encounter a strong case for non-negligible harms/externalities to community health that could result from the grant making process, what would your response to that evidence be? <<
The response would likely depend on a hard-to-answer question about how FTX conceives of its responsibilities within the community given that it is now the largest funder by far.
Personally, I was hoping for a response more along the lines of “Oh, we hadn’t thought about it that way. Can you tell us more? How do you think we get more information about how this could be important?”
I was grateful for Nick’s thoughtful answer about what’s happening over there. I think we all hear what you’re saying about chosen priorities, complexity of project, and bandwidth issues. Also the future is hard to predict. I get all that and can feel how authentically you feel proud about how hard the team has been working and the great work that’s been done already. I’m sure that’s an amazing place to be.
My question marks are around how you conceive of responsibility and choose to take responsibility moving forward in light of new information about the reality on the ground. Given the resources at your disposal, I’d be inclined to view your answer within the lens of prioritization of options, rather than simply making the best of constraints.
As the largest funder in the space by far, it’s a choice to be open to discovering and uncovering risk and harms that they didn’t account for previously. It’s a choice to devote time and resources to investigate them. It’s a choice to think through how context shifts and your relationship to responsibility evolves. It’s a choice to not do all those things.
A few things that seem hard waive away:
1) 1600 −1650 (?) rejected applications from the largest and most exciting new funder with no feedback could be disruptive to community health
Live example: Established organization(s) got rejected and/or far less than asked for with no feedback. Stakeholders asked the project leaders “What does it mean that you got rejected/less than you asked for from FTX? What does that say about the impact potential of your project, quality of your project, fitness to lead it, etc.” This can cause great instability. Did FTX foresee this? Probably not, for understandable reasons. Is this the effect that FTX wants to have? Probably not. Is it FTX’s responsibility to address this? Uncertain.
2) Opaque reasoning for where large amounts of money goes and why could be disruptive to community health
3) (less certain regarding your M&E plans) Little visibility on M&E given to applicants puts them in a place of not only not knowing what is good, but also how they know they’re doing well. Also potentially disruptive
In regards to the approach moving forward for FTX, I wouldn’t be surprised if more reflection among the staff yielded more than ‘we’re trying hard + it’s complex + bandwidth issues so what do you want us to do?’ My hope with this comment is to nudge internal discussions to be more expansive and reflective. Maybe you can let me know if that happened or not. Insofar as I delivered this in a way that hopefully didn’t feel like an attack, if you feel including me in a discussion would be helpful, I’d love to be a part of it.
And finally, I’m not sure where the ‘we couldn’t possibly give feedback on 1700 applications’ response came from. I mentioned feedback, but there’s innumerable ways to construct a feedback apparatus that isn’t (what seemed to be assumed) the same level of care and complexity for each application. A quick example – ‘stratified feedback’ – FTX considers who the applicant is and gives varying levels of feedback depth depending on who they are. This could be important for established EA entities (as I mentioned above), where for various reasons, you think leaving them completely in the dark would be actively harmful for a subnetwork of stakeholders. My ideal version of this would also include promising individuals who you don’t want to discourage, but for whatever reason their application wasn’t successful.
Thanks for taking the time. I hope this is received well.
Agree with this—it’s impossible to give constructive feedback on thousands of applications. The decision is between not giving grants, or accepting that most grant applications won’t get much feedback from us. We chose the latter.
I’d like to challenge this. There are simultaneous claims that:
it’s impossible to give constructive feedback on thousands of applications
It is possible to effectively (in an expected value sense) allocate $100m - $1b a year using this process which evaluates thousands of applications from a broad range of applicants related to a broad spectrum of ideas over just a two week period
I don’t think both can be true in the long run. Like others in the comments suggested both may be a question of further investment in and improvement of the process. There is a lot of room for improvement: any feedback is better than no feedback, it doesn’t have to be super constructive -just knowing if anyone even spent more than a minute looking at your application is useful info that applicants currently don’t have.
Wanting to be constructive: would there be arguments against hiring an extra person whose job is to observe the decision making process (I assume there is a kind of internal log of decisions/opinions), and formulate non-zero feedback on applications?
It would be very surprising if there weren’t an opportunity cost to providing feedback. Those might include:
Senior management time to oversee the project, bottlenecking other plans
PR firefighting and morale counselling when 1 in ~100 people get angry at what you say and cause you grief (this will absolutely happen)
Any hires capable of thinking up and communicating helpful feedback (this is difficult!) could otherwise use that time to read and make decisions on more grant proposals in more areas — or just improve the decision-making among the same pool of applicants.
That there’s an opportunity cost doesn’t show it’s not worth it but my guess is right now it would be huge mistake for Future Fund to provide substantial feedback except in rare cases.
That could change in future if their other streams of successful applicants dry up and improving the projects of people who were previously rejected becomes the best way to find new things they want to fund.
Yep, I’d imagine what makes sense is between ‘highly involved and coordinated attempt to provide feedback at scale’ and ‘zero’. I think it’s tempting to look away from how harmful ‘zero’ can be at scale
> That could change in future if their other streams of successful applicants dry up and improving the projects of people who were previously rejected becomes the best way to find new things they want to fund.
Agreed – this seems like a way to pick up easy wins and should be a good go-to for grant makers to circle back. However, banking on this as handling the concerns that were raised doesn’t account for all the things that come with unqualified rejection and people deciding to do other things, leave EA, incur critical stakeholder instability etc. as a result.
In other words, for the consequentialist-driven among us, I don’t think that community health is a nice-to-have if we’re serious about having a community of highly effective people working urgently on hard/complex things
I mean I think people are radically underestimating the opportunity cost of doing feedback properly at the moment. If I’m right then getting feedback might reduce people’s chances of getting funded by say, 30%, or 50%, because the throughput for grants will be much reduced.
I would probably rather have a 20% chance of getting funding for my project without feedback than a 10% chance with feedback, though people’s preferences may vary.
(Alternatively all the time spent explaining and writing and corresponding will mean worse projects get funded as there’s not much time left to actually think through which projects are most impactful.)
Rob, I think you’re consistently arguing against a point few people are making. You talk about ongoing correspondence with projects, or writing (potentially paragraphs of) feedback. Several people in this thread have suggested that pre-written categories of feedback would be a huge improvement from the status quo, and I can’t see anything you’ve said that actually argues against that.
Also, as someone who semi-regularly gives feedback to 80+ people, I’ve never found it to make my thinking worse, but I’ve sometimes found it makes my thinking better.
I’m not saying there’s no cost to feedback. Of course there’s a cost! But these exaggerations are really frustrating to read, because I actually do this kind of work and the cost of what I’m proposing is a lot lower than you keep suggesting.
If it’s just a form where the main reason for rejection is chosen from a list then that’s probably fine/good.
I’ve seen people try to do written feedback before and find it a nightmare so I guess people’s mileage varies a fair bit.
I’ve got a similar feeling to Khorton. Happy to have been pre-empted there.
It could be helpful to consider what it is that legibility in the grant application process (for which post-application feedback is only one sort) is meant to achieve. Depending on the grant maker’s aims, this can non-exhaustively include developing and nurturing talent, helping future applicants self-select, orienting projects on whether they are doing a good job, being a beacon and marketing instrument, clarifying and staking out an epistemic position, serving an orientation function for the community etc.
And depending on the basket of things the grant maker is trying to achieve, different pieces of legibility affect ‘efficiency’ in the process. For example, case studies and transparent reasoning about accepted and rejected projects, published evaluations, criteria for projects to consider before applying, hazard disclaimers, risk profile declarations, published work on the grant makers theory of change, etc. can give grant makers ‘published’ content to invoke during the post-application process that allows for the scaling of feedback. (e.g. our website states that we don’t invest in projects that rapidly accelerate ‘x’). There are other forms of pro-active communication and stratifying applicant journeys that would make things even more efficient.
FTX did what they did, and there is definitely a strong case for why they did it that way. In moving forward , I’d be curious to see if they acknowledge and make adjustments in light of the fact that different forms and degrees of legibility can affect the community.
Okay, upon review, that was a little bit too much of a rhetorical flourish at the end. Basically, I think there’s something seriously important to consider here about how process can negatively affect community health and alignment, which I believe to be important for this community in achieving the plurality of ambitious goals we’re shooting for. I believe FTX could definitely affect in a very positive way if they wanted to
Thanks for your comment! I wanted to try to clarify a few things regarding the two claims you see us as making. I agree there are major benefits to providing feedback to applicants. But there are also significant costs, too, and I want to explain why it’s at least a non-obvious decision what the right choice is here.
On (1), I agree with Sam that it wouldn’t be the right prioritization for our team right now to give detailed feedback to >1600 applications we rejected, and would cut into our total output for the year significantly. I think it could be done if need be, but it would be really hard and require an innovative approach. So I don’t think we should be doing this now, but I’m not saying that we won’t try to find ways to give more feedback in the future (see below).
On (2), although we want to effectively allocate at least $100M this year, we don’t plan to do 100% of this using this particular process without growing our team. In our announcement post, we said we would try four different processes and see what works best. We could continue all, some, or none of them. We have given out considerably less than $100M via the open call (more in our progress update in a month or so); and, as I mentioned in another comment, for larger and/or more complex grants the investigation process often takes longer than two weeks.
On hiring someone to do this: I think there are good reasons for us not to hire an extra person whose job is to give feedback to everyone. Most importantly: there are lots of things we could hire for, I take early hiring decisions very seriously because they affect the culture and long-term trajectory of the organization, and we want to take those decisions slowly and deliberately. I also think it’s important to maintain a certain quality bar for this kind of feedback, and this would likely require significant oversight from the existing team.
Will we provide feedback to rejected applicants in the future? Possibly, but I think this involves complex tradeoffs and isn’t a no-brainer. I’ll try to explain some of the reasons I see it this way, even at scale. A simple and unfortunate reason is that there are a lot of opportunities for angry rejected applicants—most of whom we do not know at all and aren’t part of the effective altruism community—to play “gotcha” on Twitter (or with lawsuit threats) in response to badly worded feedback, and even if the chances of this happening are small for any single rejected application, the cumulative chances of this happening once are substantial if you’re giving feedback to thousands of people. (I think this may be why even many public-spirited employers and major funders don’t provide such feedback.) I could imagine a semi-standardized process that gave more feedback to people who wanted it and very nearly got funded. (A model that I heard TripleByte used sounds interesting to me.) We’ll have to revisit these questions the next time we have an open call, and we’ll take the conversation here into account—we really appreciate your feedback!
I wrote a comment about TripleByte’s feedback process here; this blog post is great too. In our experience, the fear of lawsuits and PR disasters from giving feedback to rejected candidates was much overblown, even at a massive scale. (We gave every candidate feedback regardless of how well they performed on our interview.)
Something I didn’t mention in my comment is that much of TripleByte’s feedback email was composed of prewritten text blocks carefully optimized to be helpful and non-offensive. While interviewing a candidate, I would check boxes for things like “this candidate used their debugger poorly”, and then their feedback email would automatically include a prewritten spiel with links on how to use a debugger well (or whatever). I think this model could make a lot of sense for the fund:
It makes giving feedback way more scalable. There’s a one-time setup cost of prewriting some text blocks, and probably a minor ongoing cost of gradually improving your blocks over time, but the marginal cost of giving a candidate feedback is just 30 seconds of checking some boxes. (IIRC our approach was to tell candidates “here are some things we think it might be helpful for you to read” and then when in doubt, err on the side of checking more boxes. For funding, I’d probably take it a step further, and rank or score the text blocks according to their importance to your decision. At TripleByte, we would score the candidate on different facets of their interview performance and send them their scores—if you’re already scoring applications according to different facets, this could be a cheap way to provide feedback.)
Minimize lawsuit risk. It’s not that costly to have a lawyer vet a few pages of prewritten text that will get reused over and over. (We didn’t have a lawyer look over our feedback emails, and it turned out fine, so this is a conservative recommendation.)
Minimize PR risk. Someone who posts their email to Twitter can expect bored replies like “yeah, they wrote the exact same thing in my email.” (Again, PR risk didn’t seem to be an issue in practice despite giving lots of freeform feedback along with the prewritten blocks, so this seems like a conservative approach to me.)
If I were you, I think I’d experiment with hiring one of the writers of the TripleByte feedback emails as a contractor or consultant. Happy to make an intro.
A few final thoughts:
Without feedback, a rejectee is likely to come up with their own theory of why they were rejected. You have no way to observe this theory or vet its quality. So I think it’s a mistake to hold yourself to a high bar. You just have to beat the rejectee’s theory. (BTW, most of the EA rejectee theories I’ve heard have been very cynical.)
You might look into liability insurance if you don’t have it already; it probably makes sense to get it for other reasons anyway. I’d be curious how the cost of insurance changes depending on the feedback you’re giving.
Very much appreciate the considerate engagement with this. Wanted to flag that my primary response to your initial comment can be found here.
All this makes a lot of sense to me. I suspect some people got value out of the presentation of this reasoning. My goal here was to bring this set of consideration to yours and Sam’s attention and upvote its importance, hopefully it’s factored into what is definitely non-obvious and complex to decide moving forward. Great to see how thoughtful you all have been and thanks again!
Thanks for the response, and thanks for being open to improving your process, and I agree with many of your points about the importance of scaling teams cautiously.
I disagree that it’s impossible to give constructive feedback on 1700 applications.
I could imagine FTX Future Fund having a couple of standardized responses, rather than just one. For example:
Your application was rejected because based on the information provided it did not appear to be in scope for what we fund (link to the page that sets out what you fund)
Your application appears to be in scope for what we fund. We weren’t currently confident in the information provided about [theory of change / founding team / etc]. It might still be a good fit for another grantmaker. If you do decide to update that section, feel free to re-apply to a future round of funding.
potentially a response for applications you think are an especially bad idea?
It seems many of the downsides of giving feedback would also apply to this.
I think lower resolution feedback introduces new issues too. For example, people might become aware of the schema and over-index on getting a “1. Reject” versus getting a “2. Revise and resubmit”.
A major consideration is that I think some models of very strong projects and founders says that these people wouldn’t be harmed by rejections.
Further considerations related to this (that are a little sensitive) is that there are other ways of getting feedback, and that extremely impactful granting and funding is relationship based, not based on an instance of one proposal or project. This makes sense once you consider that grantees are EAs and should have very high knowledge of their domains in EA cause areas.
Thanks to Sam and Nick for getting to this. I think it’s very cool that you two are taking the time to engage. In light of the high esteem that I regard both of you and the value of your time, I’ll try to close the loop of this interaction by leaving you with one main idea.
I was pointing at something different than what I think was addressed. To distill what I was saying: >> Were FTX to encounter a strong case for non-negligible harms/externalities to community health that could result from the grant making process, what would your response to that evidence be? <<
The response would likely depend on a hard-to-answer question about how FTX conceives of its responsibilities within the community given that it is now the largest funder by far.
Personally, I was hoping for a response more along the lines of “Oh, we hadn’t thought about it that way. Can you tell us more? How do you think we get more information about how this could be important?”
I was grateful for Nick’s thoughtful answer about what’s happening over there. I think we all hear what you’re saying about chosen priorities, complexity of project, and bandwidth issues. Also the future is hard to predict. I get all that and can feel how authentically you feel proud about how hard the team has been working and the great work that’s been done already. I’m sure that’s an amazing place to be.
My question marks are around how you conceive of responsibility and choose to take responsibility moving forward in light of new information about the reality on the ground. Given the resources at your disposal, I’d be inclined to view your answer within the lens of prioritization of options, rather than simply making the best of constraints.
As the largest funder in the space by far, it’s a choice to be open to discovering and uncovering risk and harms that they didn’t account for previously. It’s a choice to devote time and resources to investigate them. It’s a choice to think through how context shifts and your relationship to responsibility evolves. It’s a choice to not do all those things.
A few things that seem hard waive away:
1) 1600 −1650 (?) rejected applications from the largest and most exciting new funder with no feedback could be disruptive to community health
Live example: Established organization(s) got rejected and/or far less than asked for with no feedback. Stakeholders asked the project leaders “What does it mean that you got rejected/less than you asked for from FTX? What does that say about the impact potential of your project, quality of your project, fitness to lead it, etc.” This can cause great instability. Did FTX foresee this? Probably not, for understandable reasons. Is this the effect that FTX wants to have? Probably not. Is it FTX’s responsibility to address this? Uncertain.
2) Opaque reasoning for where large amounts of money goes and why could be disruptive to community health
3) (less certain regarding your M&E plans) Little visibility on M&E given to applicants puts them in a place of not only not knowing what is good, but also how they know they’re doing well. Also potentially disruptive
In regards to the approach moving forward for FTX, I wouldn’t be surprised if more reflection among the staff yielded more than ‘we’re trying hard + it’s complex + bandwidth issues so what do you want us to do?’ My hope with this comment is to nudge internal discussions to be more expansive and reflective. Maybe you can let me know if that happened or not. Insofar as I delivered this in a way that hopefully didn’t feel like an attack, if you feel including me in a discussion would be helpful, I’d love to be a part of it.
And finally, I’m not sure where the ‘we couldn’t possibly give feedback on 1700 applications’ response came from. I mentioned feedback, but there’s innumerable ways to construct a feedback apparatus that isn’t (what seemed to be assumed) the same level of care and complexity for each application. A quick example – ‘stratified feedback’ – FTX considers who the applicant is and gives varying levels of feedback depth depending on who they are. This could be important for established EA entities (as I mentioned above), where for various reasons, you think leaving them completely in the dark would be actively harmful for a subnetwork of stakeholders. My ideal version of this would also include promising individuals who you don’t want to discourage, but for whatever reason their application wasn’t successful.
Thanks for taking the time. I hope this is received well.