I want to very briefly argue that given the complexity of long-term trajectories, the lack of empirical evidence, and the difficulty of identifying robust interventions, efforts to improve future value are significantly less tractable than reducing existential risk.
[...]
And compared to existential risk, where specific interventions may have clear leverage points, such as biosecurity or AI safety, increasing the quality of long-term futures is a vast and nebulous goal.
I guess, there is a misunderstanding in your analysis. Please correct me if I am wrong.
“Increasing the quality of long-term futures” reduces existential risks. When longtermists talk about “increasing the quality of long-term futures,” they include progress on aligning AIs as one of the best interventions they have in mind.
To compare their relative tractability, let’s look at the best intervention to reduce Extinction-Risks and, on the other hand, at the best interventions for “increasing the quality of long-term futures”, what I call reducing Alignment-Risks.
Illustrative best PTIs for Extinction-Risks reduction: Improving AI control, Reducing AI misuses. These reduce the chance of AI destroying future Earth-originating intelligent agents.
Illustrative best PTIs for Alignment-Risks reduction: Technical AI alignment, improving AI governance. These improve the quality of the long-term futures.
Now, let’s compare their tractability. How these interventions differ in tractability is not clear. These interventions actually overlap significantly. It is not clear if reducing misuse risks is actually harder than improving alignment or than improving AI governance.
Interestingly, this leads us to a plausible contradiction in arguments against Alignment-Risks: Some will say that the interventions to reduce Alignment-Risks and Extinction-Risks are the same, and some will say they have vastly different tractability. One of the two groups is incorrect. Interventions can’t be the same and have different tractability.
You make a dichotomy not present in my post, then conflate the two types of interventions while focusing only on AI risk—so that you’re saying that two different kinds of what most people would call extinction reduction efforts are differently tractable—and conclude that there’s a definition confusion.
To respond, first, that has little to do with my argument, but if it’s correct, your problem is with the entire debate week framing, which you think doesn’t present two distinct options, not with my post! And second, look at the other comments which bring up other types of change as quality increasing, and try to do the same analysis, without creating new categories, and you’ll understand what I was saying better.
I guess, there is a misunderstanding in your analysis. Please correct me if I am wrong.
“Increasing the quality of long-term futures” reduces existential risks. When longtermists talk about “increasing the quality of long-term futures,” they include progress on aligning AIs as one of the best interventions they have in mind.
To compare their relative tractability, let’s look at the best intervention to reduce Extinction-Risks and, on the other hand, at the best interventions for “increasing the quality of long-term futures”, what I call reducing Alignment-Risks.
Illustrative best PTIs for Extinction-Risks reduction: Improving AI control, Reducing AI misuses. These reduce the chance of AI destroying future Earth-originating intelligent agents.
Illustrative best PTIs for Alignment-Risks reduction: Technical AI alignment, improving AI governance. These improve the quality of the long-term futures.
Now, let’s compare their tractability. How these interventions differ in tractability is not clear. These interventions actually overlap significantly. It is not clear if reducing misuse risks is actually harder than improving alignment or than improving AI governance.
Interestingly, this leads us to a plausible contradiction in arguments against Alignment-Risks: Some will say that the interventions to reduce Alignment-Risks and Extinction-Risks are the same, and some will say they have vastly different tractability. One of the two groups is incorrect. Interventions can’t be the same and have different tractability.
You make a dichotomy not present in my post, then conflate the two types of interventions while focusing only on AI risk—so that you’re saying that two different kinds of what most people would call extinction reduction efforts are differently tractable—and conclude that there’s a definition confusion.
To respond, first, that has little to do with my argument, but if it’s correct, your problem is with the entire debate week framing, which you think doesn’t present two distinct options, not with my post! And second, look at the other comments which bring up other types of change as quality increasing, and try to do the same analysis, without creating new categories, and you’ll understand what I was saying better.