Nothing I have seen makes me thinks the EAIF should change the decision criteria. It seems to be working very well and good stuff is getting funded. So don’t change that to address a comparatively very minor issue like this, would be throwing the baby out with the bathwater!!
-- If you showed me the list here and said ‘Which EA Fund should fund each of these?’ I would have put the Lohmar and the CLTR grants (which both look like v good grants and glad they are getting funded) in the longtermist fund. Based on your comments above you might have made the same call as well.
From an outside view the actual cost of making the grants from the pot of another fund seems incredibly small. At minimum it could just be having someone to look over the end decisions and see if any feel like they belong in a different fund and then quickly double checking with the other fund’s grantmakers that they have no strong objections and then granting the money from a different pot. (You could even do that after the decision to grant has been communicated to applicants, no reason to hold up, if the second fund objects then can still be given by the first fund).
And then all those dogmatic donors to the EAIF who don’t like longtermist stuff can go to bed happy and all those dogmatic donors to the LTFF who don’t like meta stuff can go to bed happy and everyone feels like there money is going to where they expect it to go, etc. Which does matter a little bit because as a donor you feel that you really need to trust that the money is going to where it says on the tin and not to something else.
(But sure if the admin costs here are actually really high or something then not a big deal, it matters a little bit to some donors but is not the most important thing to get right)
From an outside view the actual cost of making the grants from the pot of another fund seems incredibly small. At minimum it could just be having someone to look over the end decisions and see if any feel like they belong in a different fund and then quickly double checking with the other fund’s grantmakers that they have no strong objections and then granting the money from a different pot. (You could even do that after the decision to grant has been communicated to applicants, no reason to hold up, if the second fund objects then can still be given by the first fund).
Thank you for this suggestion. It makes sense to me that this is how the situation looks from the outside.
I’ll think about the general issue and suggestions like this one a bit more, but currently don’t expect large changes to how we operate. I do think this might mean that in future rounds there may be a similar fraction of grants that some donors perceive to better fit with another fund. I acknowledge that this is not ideal, but I currently expect it will seem best after considering the cost and benefits of alternatives.
So please view the following points of me trying to explain why I don’t expect to adopt what may sound like a good suggestion, while still being appreciative of the feedback and suggestions.
I think based on my EA Funds experience so far, I’m less optimistic that the cost would be incredibly small. E.g., I would expect less correlation between “EAIF managers think something is good to fund from a longtermist perspective” and “LTFF managers think something is good to fund from a longtermist perspective” (and vice versa for ‘meta’ grants) than you seem to expect.
This is because grantmaking decisions in these areas rely a lot on judgment calls that different people might make differently even if they’re aligned on broad “EA principes” and other fundamental views. I have this view both because of some cases I’ve seen where we actually discussed (aspects of) grants across both the EAIF and LTFF managers and because within the EAIF committee large disagreements are not uncommon (and I have no reasons to believe that disagreements would be smaller between LTFF and EAIF managers than just within EAIF managers).
To be clear, I would expect decision-relevant disagreements for a minority of grants—but not a sufficiently clear minority that I’d be comfortable acting on “the other fund is going to make this grant” as a default assumption.
Your suggestion of retaining the option to make the grant through the ‘original’ fund would help with this, but not with the two following points.
I think another issue is duplication of time cost. If the LTFF told me “here is a grant we want to make, but we think it fits better for the EAIF—can you fund it?”, then I would basically always want to have a look at it. In maybe 50% [?, unsure] of cases this would only take me like 10 minutes, though the real attention + time cost would be higher. In the other 50% of cases I would want to invest at least another hour—and sometimes significantly more—assessing the grant myself. E.g., I might want to talk to the grantee myself or solicit additional references. This is because I expect that donors and grantees would hold me accountable for that decision, and I’d feel uncomfortable saying “I don’t really have an independent opinion on this grant, we just made it b/c it was recommended by the LTFF”.
(In general, I worry that “quickly double-checking” something is close to impossible between two groups of 4 or so people, all of whom are very opinionated and can’t necessarily predict each other’s views very well, are in parallel juggling dozens of grant assessments, and most of whom are very time-constrained and are doing all of this next to their main jobs.)
A third issue is that increasing the delay between the time of a grant application and the time of a grant payout is somewhat costly. So, e.g., inserting another ‘review allocation of grants to funds’ step somewhere would somewhat help with the time & attention cost by bundling all scoping decisions together; but it would also mean a delay of potentially a few days or even more given fund managers’ constrained availabilities. This is not clearly prohibitive, but significant since I think that some grantees care about the time window between application and potential payments being short.
However, there may be some cases where grants could be quickly transferred (e.g., if for some reason managers from different funds had been involved in a discussion anyway), or there may be other, less costly processes for how to organize transfers. This is definitely something I will be paying a bit more attention to going forward, but for the reasons explained in this and other comments I currently don’t expect significant changes to how we operate.
Thank you so much for your thoughtful and considered reply.
I think based on my EA Funds experience so far, I’m less optimistic that the cost would be incredibly small. E.g., I would expect less correlation between “EAIF managers think something is good to fund from a longtermist perspective” and “LTFF managers think something is good to fund from a longtermist perspective” (and vice versa for ‘meta’ grants) than you seem to expect.
This is because grantmaking decisions in these areas rely a lot on judgment calls that different people might make differently even if they’re aligned on broad “EA principes” and other fundamental views. I have this view both because of some cases I’ve seen where we actually discussed (aspects of) grants across both the EAIF and LTFF managers and because within the EAIF committee large disagreements are not uncommon (and I have no reasons to believe that disagreements would be smaller between LTFF and EAIF managers than just within EAIF managers).
Sorry to change topic but this is super fascinating and more interesting to me than questions of fund admin time (however much I like discussing organisational design I am happy to defer to you / Jonas / etc on if the admin cost is too high – ultimately only you know that).
Why would there be so much disagreement (so much that you would routinely want to veto each others decisions if you had the option)? It seems plausible that if there is such levels of disagreement maybe:
One fund is making quite poor decisions AND/OR
There is significant potential to use consensus decisions making tools as a large group to improve decision quality AND/OR
There are some particularly interesting lessons to be learned by identifying the cruxes of these disagreements.
Just curious and typing up my thoughts. Not expecting good answers to this.
I think all funds are generally making good decisions.
I think a lot of the effect is just that making these decisions is hard, and so that variance between decision-makers is to some extent unavoidable. I think some of the reasons are quite similar to why, e.g., hiring decisions, predicting startup success, high-level business strategy, science funding decisions, or policy decisions are typically considered to be hard/unreliable. Especially for longtermist grants, on top of this we have issues around cluelessness, potentially missing crucial considerations, sign uncertainty, etc.
I think you are correct that both of the following are true:
There is potential of improving decision quality by spending time on discussing diverging views, improving the way we aggregate opinions to the extent they still differ after the amount of discussion that is possible, and maybe by using specific ‘decision making tools’ (e.g., certain ways of a structured discussion + voting).
There are interesting lessons to be learned by identifying cruxes. Some of these lessons might directly improve future decisions, others might be valuable for other reasons—e.g., generating active grantmaking ideas or cruxes/results being shareable and thereby being a tiny bit epistemically helpful to many people.
I think a significant issue is that both of these cost time—both identifying how to improve in these areas and then implementing the improvements -, which is a very scarce resource for fund managers.
I don’t think it’s obvious whether at the margin the EAIF committee should spend more or less time to get more or fewer benefits in these areas. Hopefully this means we’re not too far away from the optimum.
I think there are different views on this within EA Funds (both within the EAIF committee, and potentially between the average view of the EAIF committee and the average view of the LTFF committee—or at least this is suggested by revealed preferences as my loose impression is that LTFF fund managers spend more time in discussions with each other). Personally, I actually lean toward spending less time and less aggregation of opinions across fund managers—but I think currently this view isn’t sufficiently widely shared that I expect it to be reflected in how we’re going to make decisions in the future.
But I also feel a bit confused because some people (e.g., some LTFF fund managers, Jonas) have told me that spending more time discussing disagreements seemed really helpful to them, while I feel like my experience with this and my inside-view prediction of how spending more time on discussions would look like make me expect less value. I don’t really know why that is—it could be that I’m just bad at getting value out of discussions, or updating my views, or something like that.
think a significant issue is that both of these cost time
I am always amazed at how much you fund managers all do given this isn’t your paid job!
I don’t think it’s obvious whether at the margin the EAIF committee should spend more or less time to get more or fewer benefits in these areas
Fair enough. FWIW my general approach to stuff like this is not to aim for perfection but to aim for each iteration/round to be a little bit better than the last.
… it could be that I’m just bad at getting value out of discussions, or updating my views, or something like that.
That is possible. But also possible that you are particularly smart and have well thought-out views and people learn more from talking to you than you do from talking to them! (And/or just that everyone is different and different ways of learning work for different people)
I think based on my EA Funds experience so far, I’m less optimistic that the cost would be incredibly small. E.g., I would expect less correlation between “EAIF managers think something is good to fund from a longtermist perspective” and “LTFF managers think something is good to fund from a longtermist perspective” (and vice versa for ‘meta’ grants) than you seem to expect.
This is because grantmaking decisions in these areas rely a lot on judgment calls that different people might make differently even if they’re aligned on broad “EA principes” and other fundamental views. I have this view both because of some cases I’ve seen where we actually discussed (aspects of) grants across both the EAIF and LTFF managers and because within the EAIF committee large disagreements are not uncommon (and I have no reasons to believe that disagreements would be smaller between LTFF and EAIF managers than just within EAIF managers).
Thanks for writing up this detailed response. I agree with your intuition here that ‘review, refer, and review again’ could be quite time consuming.
However, I think it’s worth considering why this is the case. Do we think that the EAIF evaluators are similarly qualified to judge primarily-longtermist activities as the LTFF people, and the differences of views is basically noise? If so, it seems plausible to me that the EAIF evaluators should be able to unilaterally make disbursements from the LTFF money. In this setup, the specific fund you apply to is really about your choice of evaluator, not about your choice of donor, and the fund you donate to is about your choice of cause area, not your choice of evaluator-delegate.
In contrast, if the EAIF people are not as qualified to judge primarily-longtermist (or primarily animal rights, etc.) projects as the specialised funds’ evaluators, then they should probably refer the application early on in the process, prior to doing detailed due diligence etc.
If you showed me the list here and said ‘Which EA Fund should fund each of these?’ I would have put the Lohmar and the CLTR grants (which both look like v good grants and glad they are getting funded) in the longtermist fund. Based on your comments above you might have made the same call as well.
Thank you for sharing—as I mentioned I find this concrete feedback spelled out in terms of particular grants particularly useful.
[ETA: btw I do think part of the issue here is an “object-level” disagreement about where the grants best fit—personally, I definitely see why among the grants we’ve made they are among the ones that seem ‘closest’ to the LTFF’s scope; but I don’t personally view them as clearly being more in scope for the LTFF than for the EAIF.]
[ETA: btw I do think part of the issue here is an “object-level” disagreement about where the grants best fit—personally, I definitely see why among the grants we’ve made they are among the ones that seem ‘closest’ to the LTFF’s scope; but I don’t personally view them as clearly being more in scope for the LTFF than for the EAIF.]
Thank you Max. A guess the interesting question then is why do we think different things. Is it just a natural case of different people thinking differently or have I made a mistake or is there some way the funds could better communicate.
One way to consider this might be to looking at juts the basic info / fund scope on the both EAIF and LTFF pages and ask: “if the man on the Clapham omnibus only read this information here and the description of these funds where do they think these grants would sit?”
Nothing I have seen makes me thinks the EAIF should change the decision criteria. It seems to be working very well and good stuff is getting funded. So don’t change that to address a comparatively very minor issue like this, would be throwing the baby out with the bathwater!!
--
If you showed me the list here and said ‘Which EA Fund should fund each of these?’ I would have put the Lohmar and the CLTR grants (which both look like v good grants and glad they are getting funded) in the longtermist fund. Based on your comments above you might have made the same call as well.
From an outside view the actual cost of making the grants from the pot of another fund seems incredibly small. At minimum it could just be having someone to look over the end decisions and see if any feel like they belong in a different fund and then quickly double checking with the other fund’s grantmakers that they have no strong objections and then granting the money from a different pot. (You could even do that after the decision to grant has been communicated to applicants, no reason to hold up, if the second fund objects then can still be given by the first fund).
And then all those dogmatic donors to the EAIF who don’t like longtermist stuff can go to bed happy and all those dogmatic donors to the LTFF who don’t like meta stuff can go to bed happy and everyone feels like there money is going to where they expect it to go, etc. Which does matter a little bit because as a donor you feel that you really need to trust that the money is going to where it says on the tin and not to something else.
(But sure if the admin costs here are actually really high or something then not a big deal, it matters a little bit to some donors but is not the most important thing to get right)
Thank you for this suggestion. It makes sense to me that this is how the situation looks from the outside.
I’ll think about the general issue and suggestions like this one a bit more, but currently don’t expect large changes to how we operate. I do think this might mean that in future rounds there may be a similar fraction of grants that some donors perceive to better fit with another fund. I acknowledge that this is not ideal, but I currently expect it will seem best after considering the cost and benefits of alternatives.
So please view the following points of me trying to explain why I don’t expect to adopt what may sound like a good suggestion, while still being appreciative of the feedback and suggestions.
I think based on my EA Funds experience so far, I’m less optimistic that the cost would be incredibly small. E.g., I would expect less correlation between “EAIF managers think something is good to fund from a longtermist perspective” and “LTFF managers think something is good to fund from a longtermist perspective” (and vice versa for ‘meta’ grants) than you seem to expect.
This is because grantmaking decisions in these areas rely a lot on judgment calls that different people might make differently even if they’re aligned on broad “EA principes” and other fundamental views. I have this view both because of some cases I’ve seen where we actually discussed (aspects of) grants across both the EAIF and LTFF managers and because within the EAIF committee large disagreements are not uncommon (and I have no reasons to believe that disagreements would be smaller between LTFF and EAIF managers than just within EAIF managers).
To be clear, I would expect decision-relevant disagreements for a minority of grants—but not a sufficiently clear minority that I’d be comfortable acting on “the other fund is going to make this grant” as a default assumption.
Your suggestion of retaining the option to make the grant through the ‘original’ fund would help with this, but not with the two following points.
I think another issue is duplication of time cost. If the LTFF told me “here is a grant we want to make, but we think it fits better for the EAIF—can you fund it?”, then I would basically always want to have a look at it. In maybe 50% [?, unsure] of cases this would only take me like 10 minutes, though the real attention + time cost would be higher. In the other 50% of cases I would want to invest at least another hour—and sometimes significantly more—assessing the grant myself. E.g., I might want to talk to the grantee myself or solicit additional references. This is because I expect that donors and grantees would hold me accountable for that decision, and I’d feel uncomfortable saying “I don’t really have an independent opinion on this grant, we just made it b/c it was recommended by the LTFF”.
(In general, I worry that “quickly double-checking” something is close to impossible between two groups of 4 or so people, all of whom are very opinionated and can’t necessarily predict each other’s views very well, are in parallel juggling dozens of grant assessments, and most of whom are very time-constrained and are doing all of this next to their main jobs.)
A third issue is that increasing the delay between the time of a grant application and the time of a grant payout is somewhat costly. So, e.g., inserting another ‘review allocation of grants to funds’ step somewhere would somewhat help with the time & attention cost by bundling all scoping decisions together; but it would also mean a delay of potentially a few days or even more given fund managers’ constrained availabilities. This is not clearly prohibitive, but significant since I think that some grantees care about the time window between application and potential payments being short.
However, there may be some cases where grants could be quickly transferred (e.g., if for some reason managers from different funds had been involved in a discussion anyway), or there may be other, less costly processes for how to organize transfers. This is definitely something I will be paying a bit more attention to going forward, but for the reasons explained in this and other comments I currently don’t expect significant changes to how we operate.
Thank you so much for your thoughtful and considered reply.
Sorry to change topic but this is super fascinating and more interesting to me than questions of fund admin time (however much I like discussing organisational design I am happy to defer to you / Jonas / etc on if the admin cost is too high – ultimately only you know that).
Why would there be so much disagreement (so much that you would routinely want to veto each others decisions if you had the option)? It seems plausible that if there is such levels of disagreement maybe:
One fund is making quite poor decisions AND/OR
There is significant potential to use consensus decisions making tools as a large group to improve decision quality AND/OR
There are some particularly interesting lessons to be learned by identifying the cruxes of these disagreements.
Just curious and typing up my thoughts. Not expecting good answers to this.
I think all funds are generally making good decisions.
I think a lot of the effect is just that making these decisions is hard, and so that variance between decision-makers is to some extent unavoidable. I think some of the reasons are quite similar to why, e.g., hiring decisions, predicting startup success, high-level business strategy, science funding decisions, or policy decisions are typically considered to be hard/unreliable. Especially for longtermist grants, on top of this we have issues around cluelessness, potentially missing crucial considerations, sign uncertainty, etc.
I think you are correct that both of the following are true:
There is potential of improving decision quality by spending time on discussing diverging views, improving the way we aggregate opinions to the extent they still differ after the amount of discussion that is possible, and maybe by using specific ‘decision making tools’ (e.g., certain ways of a structured discussion + voting).
There are interesting lessons to be learned by identifying cruxes. Some of these lessons might directly improve future decisions, others might be valuable for other reasons—e.g., generating active grantmaking ideas or cruxes/results being shareable and thereby being a tiny bit epistemically helpful to many people.
I think a significant issue is that both of these cost time—both identifying how to improve in these areas and then implementing the improvements -, which is a very scarce resource for fund managers.
I don’t think it’s obvious whether at the margin the EAIF committee should spend more or less time to get more or fewer benefits in these areas. Hopefully this means we’re not too far away from the optimum.
I think there are different views on this within EA Funds (both within the EAIF committee, and potentially between the average view of the EAIF committee and the average view of the LTFF committee—or at least this is suggested by revealed preferences as my loose impression is that LTFF fund managers spend more time in discussions with each other). Personally, I actually lean toward spending less time and less aggregation of opinions across fund managers—but I think currently this view isn’t sufficiently widely shared that I expect it to be reflected in how we’re going to make decisions in the future.
But I also feel a bit confused because some people (e.g., some LTFF fund managers, Jonas) have told me that spending more time discussing disagreements seemed really helpful to them, while I feel like my experience with this and my inside-view prediction of how spending more time on discussions would look like make me expect less value. I don’t really know why that is—it could be that I’m just bad at getting value out of discussions, or updating my views, or something like that.
I am always amazed at how much you fund managers all do given this isn’t your paid job!
Fair enough. FWIW my general approach to stuff like this is not to aim for perfection but to aim for each iteration/round to be a little bit better than the last.
That is possible. But also possible that you are particularly smart and have well thought-out views and people learn more from talking to you than you do from talking to them!
(And/or just that everyone is different and different ways of learning work for different people)
Thanks for writing up this detailed response. I agree with your intuition here that ‘review, refer, and review again’ could be quite time consuming.
However, I think it’s worth considering why this is the case. Do we think that the EAIF evaluators are similarly qualified to judge primarily-longtermist activities as the LTFF people, and the differences of views is basically noise? If so, it seems plausible to me that the EAIF evaluators should be able to unilaterally make disbursements from the LTFF money. In this setup, the specific fund you apply to is really about your choice of evaluator, not about your choice of donor, and the fund you donate to is about your choice of cause area, not your choice of evaluator-delegate.
In contrast, if the EAIF people are not as qualified to judge primarily-longtermist (or primarily animal rights, etc.) projects as the specialised funds’ evaluators, then they should probably refer the application early on in the process, prior to doing detailed due diligence etc.
Thank you for sharing—as I mentioned I find this concrete feedback spelled out in terms of particular grants particularly useful.
[ETA: btw I do think part of the issue here is an “object-level” disagreement about where the grants best fit—personally, I definitely see why among the grants we’ve made they are among the ones that seem ‘closest’ to the LTFF’s scope; but I don’t personally view them as clearly being more in scope for the LTFF than for the EAIF.]
Thank you Max. A guess the interesting question then is why do we think different things. Is it just a natural case of different people thinking differently or have I made a mistake or is there some way the funds could better communicate.
One way to consider this might be to looking at juts the basic info / fund scope on the both EAIF and LTFF pages and ask: “if the man on the Clapham omnibus only read this information here and the description of these funds where do they think these grants would sit?”