Hi brekels, I think these are fair points. In particular, I think we may be able to agree on the following statement as well as more precise versions of it:
We may not ever be able to align our priors and sufficiently agree on the future, but for the purposes of planning and allocating resources, the discussion around climate change seems significantly more grounded [than the one about e.g. AI safety].
In my view, the key point is that, say, climate change and AI safety differ in degree but not in kind regarding whether we can make probabilistic predictions, should take action now, etc.
In particular, consider the following similarities:
I agree that for climate change we utilize extrapolations of current trends such as “if current data projected forward with no notable intervention, the Earth would be uninhabitable in x years.”—But in principle we can do the same for AI safety, e.g. “if Moore’s Law continued, we could buy a brain-equivalent of compute for $X in Y years.”
Yes, it’s not straightforward to say what a “brain-equivalent of compute” is, or why this matters. But neither is it straightforward to e.g. determine when the Earth becomes “uninhabitable”. (Again, I might concede that the latter notion is in some sense easier to define—my point it just that I don’t see a qualitative difference.)
You say we haven’t yet observed human-level AI. But neither have we observed (at least not directly an on a planetary scale), say, +6 degrees of global warming compared to pre-industrial times. Yes, we have observed anthropogenic climate change, but we’ve also observed AI systems developed by humans including specific failure modes (e.g. misspecified rewards, biased training data, or lack of desired generalization in response to distributional shift).
In various ways it sounds right to me that we have “more data” on climate change, or that the problem of more severe climate change is “more similar” to current climate change than the problem of misaligned transformative AI is to current AI failure modes. But again, to me this seems like “merely” a difference in degree.
Separately, I think that if we try hard to find the most effective intervention to avoid some distant harm (say, one we think would occur in the year 2100, or even 2050), we will have to confront the “less well-defined” and “more uncertain” aspects of the future anyway, no matter whether the harm we’re considering has some relatively well-understood core (such as climate change).
This is because, whether we like it or not, these less well-defined issues such as the future of technology, governance, economic and political systems, etc., as well as interactions with other, less predictable, issues (e.g. migration, war, inequality, …) will make a massive difference to how some apparently predictable harm will in fact affect different people, how we in fact might be able to prevent or respond to it etc.
E.g. it’s not that much use if I can predict how much warming we’d get by 2100 conditional on a certain amount of emissions (though note that even in this seemingly “well-defined” case a lot hinges on which prior over climate sensitivity we use, since that has a large affect on the a posteriori probability of bad tail scenarios—and how to determine that prior isn’t something we can just “read off” from any current observation) if I don’t know for even the year 2050 the state of nuclear fusion, carbon capture and storage, geoengineering, solar cell efficiency, batteries, US-China relations, or whether in the meantime a misaligned AI system killed everyone.
It seems to me that the alternative, i.e. planning based on just those aspects of the future that seem “well-defined” or “predictable”, leads to things like the Population Bomb or Limits to Growth, i.e. things that have a pretty bad track record.
Hi brekels, I think these are fair points. In particular, I think we may be able to agree on the following statement as well as more precise versions of it:
In my view, the key point is that, say, climate change and AI safety differ in degree but not in kind regarding whether we can make probabilistic predictions, should take action now, etc.
In particular, consider the following similarities:
I agree that for climate change we utilize extrapolations of current trends such as “if current data projected forward with no notable intervention, the Earth would be uninhabitable in x years.”—But in principle we can do the same for AI safety, e.g. “if Moore’s Law continued, we could buy a brain-equivalent of compute for $X in Y years.”
Yes, it’s not straightforward to say what a “brain-equivalent of compute” is, or why this matters. But neither is it straightforward to e.g. determine when the Earth becomes “uninhabitable”. (Again, I might concede that the latter notion is in some sense easier to define—my point it just that I don’t see a qualitative difference.)
You say we haven’t yet observed human-level AI. But neither have we observed (at least not directly an on a planetary scale), say, +6 degrees of global warming compared to pre-industrial times. Yes, we have observed anthropogenic climate change, but we’ve also observed AI systems developed by humans including specific failure modes (e.g. misspecified rewards, biased training data, or lack of desired generalization in response to distributional shift).
In various ways it sounds right to me that we have “more data” on climate change, or that the problem of more severe climate change is “more similar” to current climate change than the problem of misaligned transformative AI is to current AI failure modes. But again, to me this seems like “merely” a difference in degree.
Separately, I think that if we try hard to find the most effective intervention to avoid some distant harm (say, one we think would occur in the year 2100, or even 2050), we will have to confront the “less well-defined” and “more uncertain” aspects of the future anyway, no matter whether the harm we’re considering has some relatively well-understood core (such as climate change).
This is because, whether we like it or not, these less well-defined issues such as the future of technology, governance, economic and political systems, etc., as well as interactions with other, less predictable, issues (e.g. migration, war, inequality, …) will make a massive difference to how some apparently predictable harm will in fact affect different people, how we in fact might be able to prevent or respond to it etc.
E.g. it’s not that much use if I can predict how much warming we’d get by 2100 conditional on a certain amount of emissions (though note that even in this seemingly “well-defined” case a lot hinges on which prior over climate sensitivity we use, since that has a large affect on the a posteriori probability of bad tail scenarios—and how to determine that prior isn’t something we can just “read off” from any current observation) if I don’t know for even the year 2050 the state of nuclear fusion, carbon capture and storage, geoengineering, solar cell efficiency, batteries, US-China relations, or whether in the meantime a misaligned AI system killed everyone.
It seems to me that the alternative, i.e. planning based on just those aspects of the future that seem “well-defined” or “predictable”, leads to things like the Population Bomb or Limits to Growth, i.e. things that have a pretty bad track record.