I think many applicants who we reject could apply with different proposals that I’d be more excited to fund—rejecting an application doesn’t mean I think there’s no good direct work the applicant could do.
I would guess some people would be better off earning to give, but I don’t know that I could say which ones just from looking at one application they’ve sent us.
I see. That suggests you think the LTFF would have much more room for funding with some not-super-large changes to your processes, such as encouraging applicants to submit multiple project proposals, or doing calls with applicants to talk about other projects they could do, or modifications to their original proposal which would make it more appealing to you.
Sadly, I think those changes would in fact be fairly large and would take up a lot of fund manager time. I think small modifications to original proposals wouldn’t be enough, and it would require suggesting new projects or assessing applicants holistically and seeing if a career change made sense.
In my mind, this relates to ways in which mentorship is a bottleneck in longtermist work right now-- there are probably lots of people who could be doing useful direct work, but they would require resources and direction that we as a community don’t have the capacity for. I don’t think the LTFF is well-placed to provide this kind of mentorship, though we do offer to give people one-off feedback on their applications.
there are probably lots of people who could be doing useful direct work, but they would require resources and direction that we as a community don’t have the capacity for.
I imagine this could be one of the highest-leverage places to apply additional resources and direction though. People who are applying for funding for independent projects are people who desire to operate autonomously and execute on their own vision. So I imagine they’d require much less direction than marginal employees at an EA organization, for instance.
I also think there’s an epistemic humility angle here. It’s very likely that the longtermist movement as it currently exists is missing important perspectives. To some degree, as a funder, you are diffing your perspective against that of applicants and rejecting applicants whose projects make sense according to their perspective and not yours. It seems easy for this to result in the longtermist movement developing more homogenous perspectives over time, as people Goodhart on whatever metrics are related to getting funding/career advancement. I’m actually not convinced that direction is a good thing! I personally would be more inclined to fund anyone who meets a particular talent bar. That also makes your job easier because you can focus on just the person/people and worry less about their project.
we do offer to give people one-off feedback on their applications.
Huh. I understood your rejection email says the fund was unable to provide further feedback due to high volume of applications.
I imagine this could be one of the highest-leverage places to apply additional resources and direction though. People who are applying for funding for independent projects are people who desire to operate autonomously and execute on their own vision. So I imagine they’d require much less direction than marginal employees at an EA organization, for instance.
I don’t have a strong take on whether people rejected from the LTFF are the best use of mentorship resources. I think many employees at EA organizations are also selected for being self-directed. I know of cases where mentorship made a big difference to both existing employees and independent LTFF applicants.
I personally would be more inclined to fund anyone who meets a particular talent bar. That also makes your job easier because you can focus on just the person/people and worry less about their project.
We do weigh individual talent heavily when deciding what to fund, i.e., sometimes we will fund someone to do work we’re less excited about because we’re interested in supporting the applicant’s career. I’m not in favor of funding exclusively based on talent, because I think a lot of the impact of our grants is in how they affect the surrounding field, and low-quality work dilutes the quality of those fields and attracts other low-quality work.
Huh. I understood your rejection email says the fund was unable to provide further feedback due to high volume of applications.
Whoops, yeah—we were previously overwhelmed with requests for feedback, so we now only offer feedback on a subset of applications where fund managers are actively interested in providing it.
I’m not in favor of funding exclusively based on talent, because I think a lot of the impact of our grants is in how they affect the surrounding field, and low-quality work dilutes the quality of those fields and attracts other low-quality work.
Let’s compare the situation of the Long-Term Future Fund evaluating the quality of a grant proposal to that of the academic community evaluating the quality of a published paper. Compared to the LTFF evaluating a grant proposal, the academic community evaluating the quality of a published paper has big advantages: The work is being evaluated retrospectively instead of prospectively (i.e. it actually exists, it is not just a hypothetical project). The academic community has more time and more eyeballs. The academic community has people who are very senior in their field, and your team is relatively junior—plus, “longtermism” is a huge area that’s really hard to be an expert in all of.
Even so, the academic community doesn’t seem very good at their task. “Sleeping beauty” papers, whose quality is only recognized long after publication, seem common. Breakthroughs are denounced by scientists, or simply underappreciated, at first (often ‘correctly’ due to being less fleshed out than existing theories). This paper contains a list of 34 examples of Nobel Prize-winning work being rejected by peer review. “Science advances one funeral at a time”, they say.
Problems compound when the question of first-order quality is replaced by the question of what others will consider to be high quality. You’re funding researchers to do work that you consider to be work that others will consider to be good—based on relatively superficial assessments due to time limitations, it sounds like.
Seems like a recipe for herd behavior. But breakthroughscomefrommavericks. This funding strategy could have a negative effect by stifling innovation (filtering out contrarian thinking and contrarian researchers from the field).
Keep longtermism weird?
(I’m also a little skeptical of your “low-quality work dilutes the quality of those fields and attracts other low-quality work” fear—since high citation count is often thought of as an ipso facto measure of quality in academia, it would seem that if work attracts additional related work, it is probably not low quality. I think the most likely fate of low-quality work is to be forgotten. If people are too credulous of work which is actually low-quality, it’s unclear to me why the fund managers would be immune to this, and having more contrarians seems like the best solution to me. The general approach of “fund many perspectives and let them determine what constitutes quality through discussion” has the advantage of offloading work from the LTFF team.)
I’m also a little skeptical of your “low-quality work dilutes the quality of those fields and attracts other low-quality work” fear—since high citation count is often thought of as an ipso facto measure of quality in academia, it would seem that if work attracts additional related work, it is probably not low quality.
The difference here is that most academic fields are pretty well-established, whereas AI safety, longtermism, and longtermist subparts of most academic fields are very new. The mechanism for attracting low-quality work I’m imagining is that smart people look at existing work and think “these people seem amateurish, and I’m not interested in engaging with them”. Luke Muelhauser’s report on case studies in early field growth gives the case of cryonics, which “failed to grow [...] is not part of normal medical practice, it is regarded with great skepticism by the mainstream scientific community, and it has not been graced with much funding or scientific attention.” I doubt most low-quality work we could fund would cripple the surrounding fields this way, but I do think it would have an effect on the kind of people who were interested in doing longtermist work.
I will also say that I think somewhat different perspectives do get funded through the LTFF, partially because we’ve intentionally selected fund managers with different views, and we weigh it strongly if one fund manager is really excited about something. We’ve made many grants that didn’t cross the funding bar for one or more fund managers.
Sure. I guess I don’t have a lot of faith in your team’s ability to do this, since you/people you are funding are already saying things that seem amateurish to me. But I’m not sure that is a big deal.
I think many applicants who we reject could apply with different proposals that I’d be more excited to fund—rejecting an application doesn’t mean I think there’s no good direct work the applicant could do.
I would guess some people would be better off earning to give, but I don’t know that I could say which ones just from looking at one application they’ve sent us.
I see. That suggests you think the LTFF would have much more room for funding with some not-super-large changes to your processes, such as encouraging applicants to submit multiple project proposals, or doing calls with applicants to talk about other projects they could do, or modifications to their original proposal which would make it more appealing to you.
Sadly, I think those changes would in fact be fairly large and would take up a lot of fund manager time. I think small modifications to original proposals wouldn’t be enough, and it would require suggesting new projects or assessing applicants holistically and seeing if a career change made sense.
In my mind, this relates to ways in which mentorship is a bottleneck in longtermist work right now-- there are probably lots of people who could be doing useful direct work, but they would require resources and direction that we as a community don’t have the capacity for. I don’t think the LTFF is well-placed to provide this kind of mentorship, though we do offer to give people one-off feedback on their applications.
I imagine this could be one of the highest-leverage places to apply additional resources and direction though. People who are applying for funding for independent projects are people who desire to operate autonomously and execute on their own vision. So I imagine they’d require much less direction than marginal employees at an EA organization, for instance.
I also think there’s an epistemic humility angle here. It’s very likely that the longtermist movement as it currently exists is missing important perspectives. To some degree, as a funder, you are diffing your perspective against that of applicants and rejecting applicants whose projects make sense according to their perspective and not yours. It seems easy for this to result in the longtermist movement developing more homogenous perspectives over time, as people Goodhart on whatever metrics are related to getting funding/career advancement. I’m actually not convinced that direction is a good thing! I personally would be more inclined to fund anyone who meets a particular talent bar. That also makes your job easier because you can focus on just the person/people and worry less about their project.
Huh. I understood your rejection email says the fund was unable to provide further feedback due to high volume of applications.
I don’t have a strong take on whether people rejected from the LTFF are the best use of mentorship resources. I think many employees at EA organizations are also selected for being self-directed. I know of cases where mentorship made a big difference to both existing employees and independent LTFF applicants.
We do weigh individual talent heavily when deciding what to fund, i.e., sometimes we will fund someone to do work we’re less excited about because we’re interested in supporting the applicant’s career. I’m not in favor of funding exclusively based on talent, because I think a lot of the impact of our grants is in how they affect the surrounding field, and low-quality work dilutes the quality of those fields and attracts other low-quality work.
Whoops, yeah—we were previously overwhelmed with requests for feedback, so we now only offer feedback on a subset of applications where fund managers are actively interested in providing it.
Let’s compare the situation of the Long-Term Future Fund evaluating the quality of a grant proposal to that of the academic community evaluating the quality of a published paper. Compared to the LTFF evaluating a grant proposal, the academic community evaluating the quality of a published paper has big advantages: The work is being evaluated retrospectively instead of prospectively (i.e. it actually exists, it is not just a hypothetical project). The academic community has more time and more eyeballs. The academic community has people who are very senior in their field, and your team is relatively junior—plus, “longtermism” is a huge area that’s really hard to be an expert in all of.
Even so, the academic community doesn’t seem very good at their task. “Sleeping beauty” papers, whose quality is only recognized long after publication, seem common. Breakthroughs are denounced by scientists, or simply underappreciated, at first (often ‘correctly’ due to being less fleshed out than existing theories). This paper contains a list of 34 examples of Nobel Prize-winning work being rejected by peer review. “Science advances one funeral at a time”, they say.
Problems compound when the question of first-order quality is replaced by the question of what others will consider to be high quality. You’re funding researchers to do work that you consider to be work that others will consider to be good—based on relatively superficial assessments due to time limitations, it sounds like.
Seems like a recipe for herd behavior. But breakthroughs come from mavericks. This funding strategy could have a negative effect by stifling innovation (filtering out contrarian thinking and contrarian researchers from the field).
Keep longtermism weird?
(I’m also a little skeptical of your “low-quality work dilutes the quality of those fields and attracts other low-quality work” fear—since high citation count is often thought of as an ipso facto measure of quality in academia, it would seem that if work attracts additional related work, it is probably not low quality. I think the most likely fate of low-quality work is to be forgotten. If people are too credulous of work which is actually low-quality, it’s unclear to me why the fund managers would be immune to this, and having more contrarians seems like the best solution to me. The general approach of “fund many perspectives and let them determine what constitutes quality through discussion” has the advantage of offloading work from the LTFF team.)
The difference here is that most academic fields are pretty well-established, whereas AI safety, longtermism, and longtermist subparts of most academic fields are very new. The mechanism for attracting low-quality work I’m imagining is that smart people look at existing work and think “these people seem amateurish, and I’m not interested in engaging with them”. Luke Muelhauser’s report on case studies in early field growth gives the case of cryonics, which “failed to grow [...] is not part of normal medical practice, it is regarded with great skepticism by the mainstream scientific community, and it has not been graced with much funding or scientific attention.” I doubt most low-quality work we could fund would cripple the surrounding fields this way, but I do think it would have an effect on the kind of people who were interested in doing longtermist work.
I will also say that I think somewhat different perspectives do get funded through the LTFF, partially because we’ve intentionally selected fund managers with different views, and we weigh it strongly if one fund manager is really excited about something. We’ve made many grants that didn’t cross the funding bar for one or more fund managers.
Sure. I guess I don’t have a lot of faith in your team’s ability to do this, since you/people you are funding are already saying things that seem amateurish to me. But I’m not sure that is a big deal.