Earlier this year, Carina Prunkl (FHI) and Jess Whittlestone (Leverhulme Centre for the Future of Intelligence) put out a paper called Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Society. The paper itself is 6 pages long and easy to read, and I thought it made several interesting and probably valuable points, so you may wish to just read the whole thing. But here I’ll summarise what I saw as the key points, and then provide my own commentary on some specific parts of the paper.
The abstract
One way of carving up the broad ‘AI ethics and society’ research space that has emerged in recent years is to distinguish between ‘near-term’ and ‘long-term’ research. While such ways of breaking down the research space can be useful, we put forward several concerns about the near/long-term distinction gaining too much prominence in how research questions and priorities are framed. We highlight some ambiguities and inconsistencies in how the distinction is used, and argue that while there are differing priorities within this broad research community, these differences are not well-captured by the near/long-term distinction. We unpack the near/long-term distinction into four different dimensions, and propose some ways that researchers can communicate more clearly about their work and priorities using these dimensions. We suggest that moving towards a more nuanced conversation about research priorities can help establish new opportunities for collaboration, aid the development of more consistent and coherent research agendas, and enable identification of previously neglected research areas.
The four dimensions
To me, a lot of the paper’s value came from the following subsection about what those “four different dimensions” actually are:
As commonly used, the terms ‘near-term’ and ‘long-term’ in fact appear to capture four different dimensions of differing priorities within the AI [ethics and society] research community:
-Capabilities: whether to focus on the impacts and challenges of current AI systems, or those relating to much more advanced AI systems
-Impacts: whether to focus mostly on the immediate impacts of AI for society, or whether to consider possible impacts much further into the future.
-Certainty: whether to focus on impacts and issues that are relatively certain and well-understood, or those that are more uncertain and speculative.
-Extremity: whether to focus on impacts at all scales, or to prioritise focusing on those that may be particularly large in scale.
None of these four dimensions are binary: one can choose research questions that focus on AI systems that are more or less advanced, exploring impacts on varying time horizons, with varying degrees of certainty and extremity. [...]
Of course, these dimensions are not entirely independent of one another: for example, if one wants to focus on particularly large-scale impacts of AI, one may have to be willing to consider more speculative questions. However, it is still useful to unpack these different dimensions, particularly because there are many possible views and research questions which are not easily captured by the near/long-term distinction as commonly used.
Table 1 and Figure 1 also interestingly represent how these dimensions can be unpacked and disentangled, so I’d suggest checking those out.
My commentary
Long-term impacts from near-term capabilities?
Prunkl and Whittlestone write:
Of course the timescale of technological advances and their impacts on society will be related. However, conflating long-term capabilities and impacts may mean the research community neglect important questions about the potential long-term impacts of current AI systems and their applications in society. For example, in what ways could injustices perpetuated by increased use of current algorithmic systems have very long-lasting and irreversible consequences for inequality in society? What are the possible longer-term implications of how data-intensive AI applications are beginning to change norms around privacy and personal data? [23]
I think it’s interesting and valuable to note that timescales “of technological advances and their impacts on society” can be disaggregated. I also appreciate that the authors note that these timescales will be related.
But I think I suspect the relationship between these timescales to be stronger than the authors do. Personally, I don’t see the questions they raise as compelling examples of potential long-term impacts from near-term capabilities. It’s not apparent to me how “injustices perpetuated by increased use of current algorithmic systems” could “have very long-lasting and irreversible consequences for inequality in society”. Nor is it apparent to me how similarly long-lasting effects could result from “how data-intensive AI applications are beginning to change norms around privacy and personal data”.
One potential explanation for this: In the context of AI and longtermism, I take “very long-lasting” to mean something like “lasting at least centuries and perhaps millions of years”. The authors may instead mean the phrase as something like “at least decades, and upwards of a century or two”. But if that’s the case, then I don’t think this is merely a linguistic confusion, because then what they mean by “long-term impacts of AI systems” is not really the key thing I’m concerned about, so I won’t necessarily be very concerned if those questions get limited attention.
Another potential explanation for this: Perhaps there actually is some mechanism by which those current AI systems and applications could have impacts that last at substantial strength for at least centuries and perhaps millions of years. Or perhaps there’s at least a high enoughchance that there’s such a mechanism for it to indeed be worth giving those questions more attention, even from longtermists.
As noted, this seems quite unlikely to me. But the authors’ point is precisely that the research community may have neglected these sorts of questions, and it’s true that I haven’t spent much time learning or thinking about them. So I’m open to being convinced on this point, though I don’t see that as likely.
Each dimension is continuous, not binary
I think that this point is true and important. I’d like to highlight in particular one important implication the authors draw from this point:
it is important to recognise that both capabilities and impacts lie on a spectrum between near- and long-term. [...] there are many ways in which AI systems could become more advanced over the coming years, before reaching anything close to ‘superintelligence’ or ‘AGI’. We need to consider what issues these intermediate advances might raise, and what kinds of intermediate advances in capabilities are mostly likely to be of harm or benefit for society.
“AI ethics and society”
There is a growing community of researchers focused on ensuring that advances in AI are safe and beneficial for humanity (which we call the AI ethics and society community, or AI E&S for short.) The AI E&S research space is very broad, encompassing many different questions: from how AI is likely to impact society in practice to what ethical issues that might raise, and what technical and governance solutions might be needed.
It makes sense to me to label a conference as a “Conference on AI, Ethics, and Society”. But it doesn’t seem to me that “the AI ethics and society community” is a good label for the “growing community of researchers focused on ensuring that advances in AI are safe and beneficial for humanity”. (Note that I’m unsure whether the authors are coining or just adopting that usage of this term.)
My core objection is that that label doesn’t seem to neatly capture technical AI safety work.
But I also feel like it’s a bit strange to explicitly include “society” in the label. It does seem true that AI’s impacts will be partly a function of how AI interacts with and is used by society. But that seems like it can be taken as implied, as it’s also true of a very wide range of things, and we don’t use labels like “bioengineering and society” or “climate change and society”. (Though one could reply that in fact we should use labels like that for those cases as well, to highlight how those issues interact with society.)
It might be that the authors are intentionally coining/adopting a separate term in order to point at a somewhat distinct community, which has less emphasis on the technical safety work and/or less emphasis on uncertain, extreme impacts from long-term capabilities. But it seems to me that people focused on the more technical and longtermist work are clearly part of the “growing community of researchers focused on ensuring that advances in AI are safe and beneficial for humanity”.
Prioritarianism/egalitarianism and longtermism
(Tangential philosophical point I may be wrong about.)
Further dimensions of disagreement may help explain why the AI E&S community has divided in certain ways. Disagreement on normative issues may be relevant here, such as around whether we have a special moral obligation to help those who are alive today over those who will live in future [26], or to prioritise helping those worst off in society [25]. Someone who holds the more fundamental philosophical belief that we should prioritise helping the worst off in society, for example, is likely to choose to work on the implications of AI for global fairness or social justice, regardless of their position on the certainty/extremity tension as outlined above.
I think that all of the above statements are true. But I disagree with one claim that could be inferred from that last sentence (and which may or may not also have been what the authors meant): The claim that a belief in the intrinsic value of promoting equality, or of improving the wellbeing of worse-off rather than better-off people, would itself or necessarily lead people to a focus on global fairness or social justice (as opposed to a focus on things like existential risk reduction).
I haven’t really thought about this before, but it seems to me that a focus on existential risk reduction could well be compatible with, or possibly even follow from, such an egalitarian or prioritarian ethic, as long as you also hold some combination of the following views:
rejection of person-affecting views (though see also this post)
rejection of pure time discounting for moral matters
belief that death counts as “worse for a person” than the issues that could be alleviated by work towards global fairness or social justice
belief that some existential risk reduction efforts have a non-negligible chance of preventing a “suffering risk”, long-lasting totalitarianism, or something like that, in which there would be many people who’d suffer with similar or greater intensity as the people who are currently worst-off
Making beliefs and disagreements more clear and explicit
Based on this analysis, we have a few concrete recommendations for how the AI E&S research community can encourage more nuanced and productive discussion about different priorities and assumptions:
[...]
-Make an effort to understand the underlying motivations and assumptions of others with different research priorities, again using some of the questions outlined in the previous section as a starting point. Conferences, workshops and journals could potentially help support this by providing fora for researchers to debate such fundamental questions and disagreements, improving mutual understanding. (emphasis in original)
I strongly agree that this would be valuable; in fact, a decent portion of the work I’ve recently been involved with has had something similar as one of its primary goals (e.g., this and this). I also think that this paper by Prunkl and Whittlestone should indeed help with that goal.
See also Rohin Shah’s summary of and commentary on the paper here, which I found after writing this but before posting it.
[Link and commentary] Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Society
Earlier this year, Carina Prunkl (FHI) and Jess Whittlestone (Leverhulme Centre for the Future of Intelligence) put out a paper called Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Society. The paper itself is 6 pages long and easy to read, and I thought it made several interesting and probably valuable points, so you may wish to just read the whole thing. But here I’ll summarise what I saw as the key points, and then provide my own commentary on some specific parts of the paper.
The abstract
The four dimensions
To me, a lot of the paper’s value came from the following subsection about what those “four different dimensions” actually are:
Table 1 and Figure 1 also interestingly represent how these dimensions can be unpacked and disentangled, so I’d suggest checking those out.
My commentary
Long-term impacts from near-term capabilities?
Prunkl and Whittlestone write:
I think it’s interesting and valuable to note that timescales “of technological advances and their impacts on society” can be disaggregated. I also appreciate that the authors note that these timescales will be related.
But I think I suspect the relationship between these timescales to be stronger than the authors do. Personally, I don’t see the questions they raise as compelling examples of potential long-term impacts from near-term capabilities. It’s not apparent to me how “injustices perpetuated by increased use of current algorithmic systems” could “have very long-lasting and irreversible consequences for inequality in society”. Nor is it apparent to me how similarly long-lasting effects could result from “how data-intensive AI applications are beginning to change norms around privacy and personal data”.
One potential explanation for this: In the context of AI and longtermism, I take “very long-lasting” to mean something like “lasting at least centuries and perhaps millions of years”. The authors may instead mean the phrase as something like “at least decades, and upwards of a century or two”. But if that’s the case, then I don’t think this is merely a linguistic confusion, because then what they mean by “long-term impacts of AI systems” is not really the key thing I’m concerned about, so I won’t necessarily be very concerned if those questions get limited attention.
Another potential explanation for this: Perhaps there actually is some mechanism by which those current AI systems and applications could have impacts that last at substantial strength for at least centuries and perhaps millions of years. Or perhaps there’s at least a high enough chance that there’s such a mechanism for it to indeed be worth giving those questions more attention, even from longtermists.
As noted, this seems quite unlikely to me. But the authors’ point is precisely that the research community may have neglected these sorts of questions, and it’s true that I haven’t spent much time learning or thinking about them. So I’m open to being convinced on this point, though I don’t see that as likely.
Each dimension is continuous, not binary
I think that this point is true and important. I’d like to highlight in particular one important implication the authors draw from this point:
“AI ethics and society”
It makes sense to me to label a conference as a “Conference on AI, Ethics, and Society”. But it doesn’t seem to me that “the AI ethics and society community” is a good label for the “growing community of researchers focused on ensuring that advances in AI are safe and beneficial for humanity”. (Note that I’m unsure whether the authors are coining or just adopting that usage of this term.)
My core objection is that that label doesn’t seem to neatly capture technical AI safety work.
But I also feel like it’s a bit strange to explicitly include “society” in the label. It does seem true that AI’s impacts will be partly a function of how AI interacts with and is used by society. But that seems like it can be taken as implied, as it’s also true of a very wide range of things, and we don’t use labels like “bioengineering and society” or “climate change and society”. (Though one could reply that in fact we should use labels like that for those cases as well, to highlight how those issues interact with society.)
So I think I’d prefer to stick with something like “AI safety and governance”, or “AI alignment” (see also MacAskill’s three-part breakdown of “the alignment problem”).
It might be that the authors are intentionally coining/adopting a separate term in order to point at a somewhat distinct community, which has less emphasis on the technical safety work and/or less emphasis on uncertain, extreme impacts from long-term capabilities. But it seems to me that people focused on the more technical and longtermist work are clearly part of the “growing community of researchers focused on ensuring that advances in AI are safe and beneficial for humanity”.
Prioritarianism/egalitarianism and longtermism
(Tangential philosophical point I may be wrong about.)
I think that all of the above statements are true. But I disagree with one claim that could be inferred from that last sentence (and which may or may not also have been what the authors meant): The claim that a belief in the intrinsic value of promoting equality, or of improving the wellbeing of worse-off rather than better-off people, would itself or necessarily lead people to a focus on global fairness or social justice (as opposed to a focus on things like existential risk reduction).
I haven’t really thought about this before, but it seems to me that a focus on existential risk reduction could well be compatible with, or possibly even follow from, such an egalitarian or prioritarian ethic, as long as you also hold some combination of the following views:
rejection of person-affecting views (though see also this post)
rejection of pure time discounting for moral matters
belief that death counts as “worse for a person” than the issues that could be alleviated by work towards global fairness or social justice
belief that some existential risk reduction efforts have a non-negligible chance of preventing a “suffering risk”, long-lasting totalitarianism, or something like that, in which there would be many people who’d suffer with similar or greater intensity as the people who are currently worst-off
Making beliefs and disagreements more clear and explicit
I strongly agree that this would be valuable; in fact, a decent portion of the work I’ve recently been involved with has had something similar as one of its primary goals (e.g., this and this). I also think that this paper by Prunkl and Whittlestone should indeed help with that goal.
See also Rohin Shah’s summary of and commentary on the paper here, which I found after writing this but before posting it.