Hi Max_Daniel! I’m sympathetic to both your and Vaden’s arguments, so I may try to bridge the gap on climate change vs. your Christmas party vs. longtermism.
Climate change is a problem now, and we have past data to support projecting already-observed effects into the future. So statements of the sort “if current data projected forward with no notable intervention, the Earth would be uninhabitable in x years.” This statement is reliant on some assumptions about future data vs. past data, but we can be reasonably clear about them and debate them.
Future knowledge will undoubtedly help things and reframe certain problems, but a key point is that we know where to start gathering data on some of the aspects you raise: “how ppl will adapt”, “how can we develop renewable energy or batteries”, etc, because climate change is already a well defined problem. We have current knowledge that will help us get off the ground.
I agree the measure theoretic arguments may prove too much, but the number of people at your Christmas party is an unambiguously posed question for which you have data on how many people you invited, how flaky your friends are, etc.
In both cases, you may use probabilistic predictions, based on a set of assumptions, to compel others to act on climate change or compel yourself to invite more people.
the key question is whether longtermism requires the kind of predictions that aren’t feasible
At the risk of oversimplification by using AI Safety example as a representative longtermist argument, the key difference is that we haven’t created or observed human-level AI, or even those which can adaptively set their own goals.
There are meaningful arguments we can use to compel others to discuss issues of safety (in algorithm development, government regulation, etc). After all, it will be a human process to develop and deploy these AI, and we can set guardrails by focused discussion today.
Vaden’s point seems to be that arguments that rely on expected values or probabilities are of significantly less value in this case. We are not operating in a well-defined problem, with already-available or easily -collectable data, because we haven’t even created the AI.
This seems to be the key point about “predicting future knowledge” being fundamentally infeasible (just as people in 1900 couldn’t meaningfully reason about the internet, let alone make expected utility calculations). Again, we’re not as ignorant as ppl in 1900 and may have a sense this problem is important, but can we actually make concrete progress with respect to killer robots today?
Everyone on this forum may have their own assumptions about the future AI, or climate change for that matter. We may not ever be able to align our priors and sufficiently agree on the future, but for the purposes of planning and allocating resources, the discussion around climate change seems significantly more grounded.
Hi brekels, I think these are fair points. In particular, I think we may be able to agree on the following statement as well as more precise versions of it:
We may not ever be able to align our priors and sufficiently agree on the future, but for the purposes of planning and allocating resources, the discussion around climate change seems significantly more grounded [than the one about e.g. AI safety].
In my view, the key point is that, say, climate change and AI safety differ in degree but not in kind regarding whether we can make probabilistic predictions, should take action now, etc.
In particular, consider the following similarities:
I agree that for climate change we utilize extrapolations of current trends such as “if current data projected forward with no notable intervention, the Earth would be uninhabitable in x years.”—But in principle we can do the same for AI safety, e.g. “if Moore’s Law continued, we could buy a brain-equivalent of compute for $X in Y years.”
Yes, it’s not straightforward to say what a “brain-equivalent of compute” is, or why this matters. But neither is it straightforward to e.g. determine when the Earth becomes “uninhabitable”. (Again, I might concede that the latter notion is in some sense easier to define—my point it just that I don’t see a qualitative difference.)
You say we haven’t yet observed human-level AI. But neither have we observed (at least not directly an on a planetary scale), say, +6 degrees of global warming compared to pre-industrial times. Yes, we have observed anthropogenic climate change, but we’ve also observed AI systems developed by humans including specific failure modes (e.g. misspecified rewards, biased training data, or lack of desired generalization in response to distributional shift).
In various ways it sounds right to me that we have “more data” on climate change, or that the problem of more severe climate change is “more similar” to current climate change than the problem of misaligned transformative AI is to current AI failure modes. But again, to me this seems like “merely” a difference in degree.
Separately, I think that if we try hard to find the most effective intervention to avoid some distant harm (say, one we think would occur in the year 2100, or even 2050), we will have to confront the “less well-defined” and “more uncertain” aspects of the future anyway, no matter whether the harm we’re considering has some relatively well-understood core (such as climate change).
This is because, whether we like it or not, these less well-defined issues such as the future of technology, governance, economic and political systems, etc., as well as interactions with other, less predictable, issues (e.g. migration, war, inequality, …) will make a massive difference to how some apparently predictable harm will in fact affect different people, how we in fact might be able to prevent or respond to it etc.
E.g. it’s not that much use if I can predict how much warming we’d get by 2100 conditional on a certain amount of emissions (though note that even in this seemingly “well-defined” case a lot hinges on which prior over climate sensitivity we use, since that has a large affect on the a posteriori probability of bad tail scenarios—and how to determine that prior isn’t something we can just “read off” from any current observation) if I don’t know for even the year 2050 the state of nuclear fusion, carbon capture and storage, geoengineering, solar cell efficiency, batteries, US-China relations, or whether in the meantime a misaligned AI system killed everyone.
It seems to me that the alternative, i.e. planning based on just those aspects of the future that seem “well-defined” or “predictable”, leads to things like the Population Bomb or Limits to Growth, i.e. things that have a pretty bad track record.
Hi Max_Daniel! I’m sympathetic to both your and Vaden’s arguments, so I may try to bridge the gap on climate change vs. your Christmas party vs. longtermism.
Climate change is a problem now, and we have past data to support projecting already-observed effects into the future. So statements of the sort “if current data projected forward with no notable intervention, the Earth would be uninhabitable in x years.” This statement is reliant on some assumptions about future data vs. past data, but we can be reasonably clear about them and debate them.
Future knowledge will undoubtedly help things and reframe certain problems, but a key point is that we know where to start gathering data on some of the aspects you raise: “how ppl will adapt”, “how can we develop renewable energy or batteries”, etc, because climate change is already a well defined problem. We have current knowledge that will help us get off the ground.
I agree the measure theoretic arguments may prove too much, but the number of people at your Christmas party is an unambiguously posed question for which you have data on how many people you invited, how flaky your friends are, etc.
In both cases, you may use probabilistic predictions, based on a set of assumptions, to compel others to act on climate change or compel yourself to invite more people.
At the risk of oversimplification by using AI Safety example as a representative longtermist argument, the key difference is that we haven’t created or observed human-level AI, or even those which can adaptively set their own goals.
There are meaningful arguments we can use to compel others to discuss issues of safety (in algorithm development, government regulation, etc). After all, it will be a human process to develop and deploy these AI, and we can set guardrails by focused discussion today.
Vaden’s point seems to be that arguments that rely on expected values or probabilities are of significantly less value in this case. We are not operating in a well-defined problem, with already-available or easily -collectable data, because we haven’t even created the AI.
This seems to be the key point about “predicting future knowledge” being fundamentally infeasible (just as people in 1900 couldn’t meaningfully reason about the internet, let alone make expected utility calculations). Again, we’re not as ignorant as ppl in 1900 and may have a sense this problem is important, but can we actually make concrete progress with respect to killer robots today?
Everyone on this forum may have their own assumptions about the future AI, or climate change for that matter. We may not ever be able to align our priors and sufficiently agree on the future, but for the purposes of planning and allocating resources, the discussion around climate change seems significantly more grounded.
Hi brekels, I think these are fair points. In particular, I think we may be able to agree on the following statement as well as more precise versions of it:
In my view, the key point is that, say, climate change and AI safety differ in degree but not in kind regarding whether we can make probabilistic predictions, should take action now, etc.
In particular, consider the following similarities:
I agree that for climate change we utilize extrapolations of current trends such as “if current data projected forward with no notable intervention, the Earth would be uninhabitable in x years.”—But in principle we can do the same for AI safety, e.g. “if Moore’s Law continued, we could buy a brain-equivalent of compute for $X in Y years.”
Yes, it’s not straightforward to say what a “brain-equivalent of compute” is, or why this matters. But neither is it straightforward to e.g. determine when the Earth becomes “uninhabitable”. (Again, I might concede that the latter notion is in some sense easier to define—my point it just that I don’t see a qualitative difference.)
You say we haven’t yet observed human-level AI. But neither have we observed (at least not directly an on a planetary scale), say, +6 degrees of global warming compared to pre-industrial times. Yes, we have observed anthropogenic climate change, but we’ve also observed AI systems developed by humans including specific failure modes (e.g. misspecified rewards, biased training data, or lack of desired generalization in response to distributional shift).
In various ways it sounds right to me that we have “more data” on climate change, or that the problem of more severe climate change is “more similar” to current climate change than the problem of misaligned transformative AI is to current AI failure modes. But again, to me this seems like “merely” a difference in degree.
Separately, I think that if we try hard to find the most effective intervention to avoid some distant harm (say, one we think would occur in the year 2100, or even 2050), we will have to confront the “less well-defined” and “more uncertain” aspects of the future anyway, no matter whether the harm we’re considering has some relatively well-understood core (such as climate change).
This is because, whether we like it or not, these less well-defined issues such as the future of technology, governance, economic and political systems, etc., as well as interactions with other, less predictable, issues (e.g. migration, war, inequality, …) will make a massive difference to how some apparently predictable harm will in fact affect different people, how we in fact might be able to prevent or respond to it etc.
E.g. it’s not that much use if I can predict how much warming we’d get by 2100 conditional on a certain amount of emissions (though note that even in this seemingly “well-defined” case a lot hinges on which prior over climate sensitivity we use, since that has a large affect on the a posteriori probability of bad tail scenarios—and how to determine that prior isn’t something we can just “read off” from any current observation) if I don’t know for even the year 2050 the state of nuclear fusion, carbon capture and storage, geoengineering, solar cell efficiency, batteries, US-China relations, or whether in the meantime a misaligned AI system killed everyone.
It seems to me that the alternative, i.e. planning based on just those aspects of the future that seem “well-defined” or “predictable”, leads to things like the Population Bomb or Limits to Growth, i.e. things that have a pretty bad track record.