(After writing this I thought of one example where the goals are in conflict: permanent surveillance that stops the development of advanced AI systems. Thought I’d still post this in case others have similar thoughts. Would also be interested in hearing other examples.)
I’m assuming a reasonable interpretation of the proxy goal of safety means roughly this: “be reasonably sure that we can prevent AI systems we expect to be built from causing harm”. Is this a good interpretation? If so, when is this proxy goal in conflict with the goal of having “things go great in the long run”?
I agree that it’s epistemically good for people to not confuse proxy goals with goals, but in practice I have trouble thinking of situations where these two are in conflict. If we’ve ever succeeded in the first goal, it seems like making progress in the second goal should be much easier, and at that point it would make more sense to advocate using-AI-to-bring-a-good-future-ism.
Focusing on the proxy goal of AI safety seems also good for the reason that it makes sense across many moral views, while people are going to have different thoughts on what it means for things to “go great in the long run”. Fleshing out those disagreements is important, but I would think there’s time to do that when we’re in a period of lower existential risk.
(After writing this I thought of one example where the goals are in conflict: permanent surveillance that stops the development of advanced AI systems. Thought I’d still post this in case others have similar thoughts. Would also be interested in hearing other examples.)
I’m assuming a reasonable interpretation of the proxy goal of safety means roughly this: “be reasonably sure that we can prevent AI systems we expect to be built from causing harm”. Is this a good interpretation? If so, when is this proxy goal in conflict with the goal of having “things go great in the long run”?
I agree that it’s epistemically good for people to not confuse proxy goals with goals, but in practice I have trouble thinking of situations where these two are in conflict. If we’ve ever succeeded in the first goal, it seems like making progress in the second goal should be much easier, and at that point it would make more sense to advocate using-AI-to-bring-a-good-future-ism.
Focusing on the proxy goal of AI safety seems also good for the reason that it makes sense across many moral views, while people are going to have different thoughts on what it means for things to “go great in the long run”. Fleshing out those disagreements is important, but I would think there’s time to do that when we’re in a period of lower existential risk.
Economic degrowth/stagnation is another example of something that prevents AI doom but will be very bad to have in the long run.