I feel like this goes against the principle of not leaving your footprint on the future, no?
Like, a large part of what I believe to be the danger with AI is that we donāt have any reflective framework for morality. I also donāt believe the standard path for AGI is one of moral reflection. This would then to me say that we leave the value of the future up to market dynamics and this doesnāt seem good with all the traps there are in such a situation? (Moloch for example)
If we want a shot at a long reflection or similar, I donāt think full sending AGI is the best thing to do.
I feel like this goes against the principle of not leaving your footprint on the future, no?
A major reason that I got into longtermism in the first place is that Iām quite interested in āleaving a footprintā on the future (albeit a good one). In other words, Iām not sure I understand the intuition for why we wouldnāt deliberately try to leave our footprints on the future, if we want to have an impact. But perhaps Iām misunderstanding the nature of this metaphor. Can you elaborate?
I also donāt believe the standard path for AGI is one of moral reflection.
I think itās worth being more specific about why you think AGI will not do moral reflection? In the post, I carefully consider arguments about whether future AIs will be alien-like and have morally arbitrary goals, in a respect that you seem to be imagining. I think itās possible that I addressed some of the intuitions behind your argument here.
I guess I felt that a lot of the post was arguing under a frame of utilitarianism which is generally fair I think. When it comes to ānot leaving a footprint on the futureā what Iām referring to is epistemic humility about the correct moral theories. Iām quite uncertain myself about what is correct when it comes to morality with extra weight on utilitarianism. From this, we should be worried about being wrong and therefore try our best to not lock in whatever weāre currently thinking. (The classic example being if we did this 200 years ago we might still have slaves in the future)
Iām a believer that virtue ethics and deontology are imperfect information approximations of utilitarianism. Like Kantās categorical imperative is a way of looking at the long-term future and asking, how do we optimise society to be the best that it can be?
I guess a core crux here for me is that it seems like youāre arguing a bit for naive utilitarianism here. I actually donāt really believe the idea that we will have the AGI follow the VNM-axioms that is being fully rational. I think it will be an internal dynamic system that are weighing for different things that it wants and that it wonāt fully maximise utility because it wonāt be internally aligned. Therefore we need to get it right or weāre going to have weird and idiosyncratic values that are not optimal for the long-term future of the world.
I hope that makes sense, I liked your post in general.
I see. After briefly skimming that post, I think I pretty strongly disagree with just about every major point in it (along with many of its empirical background assumptions), although admittedly I did not spend much time reading through it. If someone thinks that post provides good reasons to doubt the arguments in my post, Iād likely be happy to discuss the specific ideas within it in more detail.
I feel like this goes against the principle of not leaving your footprint on the future, no?
Like, a large part of what I believe to be the danger with AI is that we donāt have any reflective framework for morality. I also donāt believe the standard path for AGI is one of moral reflection. This would then to me say that we leave the value of the future up to market dynamics and this doesnāt seem good with all the traps there are in such a situation? (Moloch for example)
If we want a shot at a long reflection or similar, I donāt think full sending AGI is the best thing to do.
A major reason that I got into longtermism in the first place is that Iām quite interested in āleaving a footprintā on the future (albeit a good one). In other words, Iām not sure I understand the intuition for why we wouldnāt deliberately try to leave our footprints on the future, if we want to have an impact. But perhaps Iām misunderstanding the nature of this metaphor. Can you elaborate?
I think itās worth being more specific about why you think AGI will not do moral reflection? In the post, I carefully consider arguments about whether future AIs will be alien-like and have morally arbitrary goals, in a respect that you seem to be imagining. I think itās possible that I addressed some of the intuitions behind your argument here.
I guess I felt that a lot of the post was arguing under a frame of utilitarianism which is generally fair I think. When it comes to ānot leaving a footprint on the futureā what Iām referring to is epistemic humility about the correct moral theories. Iām quite uncertain myself about what is correct when it comes to morality with extra weight on utilitarianism. From this, we should be worried about being wrong and therefore try our best to not lock in whatever weāre currently thinking. (The classic example being if we did this 200 years ago we might still have slaves in the future)
Iām a believer that virtue ethics and deontology are imperfect information approximations of utilitarianism. Like Kantās categorical imperative is a way of looking at the long-term future and asking, how do we optimise society to be the best that it can be?
I guess a core crux here for me is that it seems like youāre arguing a bit for naive utilitarianism here. I actually donāt really believe the idea that we will have the AGI follow the VNM-axioms that is being fully rational. I think it will be an internal dynamic system that are weighing for different things that it wants and that it wonāt fully maximise utility because it wonāt be internally aligned. Therefore we need to get it right or weāre going to have weird and idiosyncratic values that are not optimal for the long-term future of the world.
I hope that makes sense, I liked your post in general.
The āfootprints on the futureā thing could be referencing this post.
(Edit: to be clear, this link is not an endorsement.)
I see. After briefly skimming that post, I think I pretty strongly disagree with just about every major point in it (along with many of its empirical background assumptions), although admittedly I did not spend much time reading through it. If someone thinks that post provides good reasons to doubt the arguments in my post, Iād likely be happy to discuss the specific ideas within it in more detail.
Yes, I was on my phone, and you canāt link things there easily; that was what I was referring to.