How does the team weigh the interests of non-humans (such as animals, extraterrestrials, and digital sentience) relative to humans? What do you folks think of the value of interventions to help non-humans in the long-term future specifically relative to that of interventions to reduce x-risk?
I don’t think there is a team-wide answer, and there certainly isn’t an institutional answer that I’m aware of. My own position is a pretty-standard-within-EA form of cosmopolitanism, where a) we should have a strong prior in favor of moral value being substrate-independent, and b) we should naively expect people to (wrongly) underestimate the moral value of beings that look different from ourselves. Also as an empirical belief, I do expect the majority of moral value in the future to be held in minds that are very different from my own. The human brain is just such a narrow target in the space of possible designs, it’d be quite surprising to me if a million years from now the most effective way to achieve value is via minds-designed-just-like-2023-humans, even by the lights of typical 2023-humans.
There are some second-order concerns like cooperativeness (I have a stronger presumption in favor of believing it’s correct to cooperate with other humans than with ants, or with aliens), but I think cosmopolitanism is mostly correct.
However, I want to be careful in distinguishing the moral value or moral patiency of other beings from their interests. It is at least theoretically possible to imagine agents (eg designed digital beings) with strong preferences and optimization ability but not morally relevant experiences. In those cases, I think there are cooperative reasons to care about their preferences, but not altruistic reasons. In particular, I think the case for optimizing for the preferences of non-existent beings is fairly weak, but the case for optimizing for their experiences (eg making sure future beings aren’t tortured) is very strong.
That said, in practice I don’t think we often (ever?) get competitive grant applications that specialize in helping non-humans in the LT future; most of our applications are about reducing risks of extinction or other catastrophic outcomes, with a smattering of applications that are about helping individuals and organizations think better (eg via forecasting or rationality training or improving mechanism/institutional design), with flow-through effects that I expect to be both positive for reducing global catastrophic risks and for other long-term outcomes.
I don’t think we often (ever?) get competitive grant applications that specialize in helping non-humans in the LT future
I think we’ve funded some work on digital sentience before. I would personally be excited about seeing some more applications in this area. I think marginal work in this area could be competitive with AIS grants if the bar reduces (as I expect).
There are several organizations that work on helping non-humans in the long-term future, such as Sentience Institute and Center on Long-Term Risk; do you think that their activities could be competitive with the typical grant applications that LTFF gets?
Also, in general, how do you folks decide how to prioritize between causes and how to compare projects?
I’m confused about the prudence of publicly discussing specific organizations in the context of being potential grantees, especially ones that we haven’t (AFAIK) given money to.
Okay, giving entirely my own professional view as I see it, absolutely not speaking for anybody else or the fund writ large:
There are several organizations that work on helping non-humans in the long-term future[...]; do you think that their activities could be competitive with the typical grant applications that LTFF gets?
To be honest, I’m not entirely sure what most of these organizations actually do research on, on a day-to-day basis. Here are some examples of what I understand to be the one-sentence pitch for many of these projects
figure out models of digital sentience
research on cooperation in large worlds
how to design AIs to reduce the risk that unaligned AIs will lead to hyperexistential catastrophes
moral circle expansion
etc,
Intuitively, they all sound plausible enough to me. I can definitely imagine projects in those categories being competitive with our other grants, especially if and when our bar lowers to where I think the longtermist bar overall “should” be. That said, the specific details of those projects, individual researchers, and organizational structure and leadership matters as well[1], so it’s hard to give an answer writ large.
From a community building angle, I think junior researchers who try to work on these topics have a reasonably decent hit rate of progressing to doing important work in other longtermist areas. So I can imagine a reasonable community-building case to fund some talent development programs as well[2], though I haven’t done a BOTEC and again the specific details matter a lot.
For example, I’m rather hesitant to recommend funding to organizations where I view the leadership as having substantially higher-than-baseline rate of being interpersonally dangerous.
How does the team weigh the interests of non-humans (such as animals, extraterrestrials, and digital sentience) relative to humans? What do you folks think of the value of interventions to help non-humans in the long-term future specifically relative to that of interventions to reduce x-risk?
I don’t think there is a team-wide answer, and there certainly isn’t an institutional answer that I’m aware of. My own position is a pretty-standard-within-EA form of cosmopolitanism, where a) we should have a strong prior in favor of moral value being substrate-independent, and b) we should naively expect people to (wrongly) underestimate the moral value of beings that look different from ourselves. Also as an empirical belief, I do expect the majority of moral value in the future to be held in minds that are very different from my own. The human brain is just such a narrow target in the space of possible designs, it’d be quite surprising to me if a million years from now the most effective way to achieve value is via minds-designed-just-like-2023-humans, even by the lights of typical 2023-humans.
There are some second-order concerns like cooperativeness (I have a stronger presumption in favor of believing it’s correct to cooperate with other humans than with ants, or with aliens), but I think cosmopolitanism is mostly correct.
However, I want to be careful in distinguishing the moral value or moral patiency of other beings from their interests. It is at least theoretically possible to imagine agents (eg designed digital beings) with strong preferences and optimization ability but not morally relevant experiences. In those cases, I think there are cooperative reasons to care about their preferences, but not altruistic reasons. In particular, I think the case for optimizing for the preferences of non-existent beings is fairly weak, but the case for optimizing for their experiences (eg making sure future beings aren’t tortured) is very strong.
That said, in practice I don’t think we often (ever?) get competitive grant applications that specialize in helping non-humans in the LT future; most of our applications are about reducing risks of extinction or other catastrophic outcomes, with a smattering of applications that are about helping individuals and organizations think better (eg via forecasting or rationality training or improving mechanism/institutional design), with flow-through effects that I expect to be both positive for reducing global catastrophic risks and for other long-term outcomes.
I think we’ve funded some work on digital sentience before. I would personally be excited about seeing some more applications in this area. I think marginal work in this area could be competitive with AIS grants if the bar reduces (as I expect).
Thanks for the responses, @Linch and @calebp!
There are several organizations that work on helping non-humans in the long-term future, such as Sentience Institute and Center on Long-Term Risk; do you think that their activities could be competitive with the typical grant applications that LTFF gets?
Also, in general, how do you folks decide how to prioritize between causes and how to compare projects?
I’m confused about the prudence of publicly discussing specific organizations in the context of being potential grantees, especially ones that we haven’t (AFAIK) given money to.
Okay, giving entirely my own professional view as I see it, absolutely not speaking for anybody else or the fund writ large:
To be honest, I’m not entirely sure what most of these organizations actually do research on, on a day-to-day basis. Here are some examples of what I understand to be the one-sentence pitch for many of these projects
figure out models of digital sentience
research on cooperation in large worlds
how to design AIs to reduce the risk that unaligned AIs will lead to hyperexistential catastrophes
moral circle expansion
etc,
Intuitively, they all sound plausible enough to me. I can definitely imagine projects in those categories being competitive with our other grants, especially if and when our bar lowers to where I think the longtermist bar overall “should” be. That said, the specific details of those projects, individual researchers, and organizational structure and leadership matters as well[1], so it’s hard to give an answer writ large.
From a community building angle, I think junior researchers who try to work on these topics have a reasonably decent hit rate of progressing to doing important work in other longtermist areas. So I can imagine a reasonable community-building case to fund some talent development programs as well[2], though I haven’t done a BOTEC and again the specific details matter a lot.
For example, I’m rather hesitant to recommend funding to organizations where I view the leadership as having substantially higher-than-baseline rate of being interpersonally dangerous.
I happen to have a small COI with one of the groups so were they to apply, I will likely recuse myself from the evaluation.