Because there is a much higher correlation between the value of the current generation of humans and the next one than there is between the values of humans and arbitrary AI entities
I’m not talking about “arbitrary AI entities” in this context, but instead, the AI entities who will actually exist in the future, who will presumably be shaped by our training data, as well as our training methods. From this perspective, it’s not clear to me that your claim is true. But even if your claim is true, I was actually making a different point. My point was instead that it isn’t clear that future generations of AIs would be much worse than future generations of humans from an impartial utilitarian point of view.
(That said, it sounds like the real crux between us might instead be about whether pausing AI would be very costly to people who currently exist. If indeed you disagree with me about this point, I’d prefer you reply to my other comment rather than replying to this one, as I perceive that discussion as likely to be more productive.)
Because there is a much higher correlation between the value of the current generation of humans and the next one than there is between the values of humans and arbitrary AI entities
I’m not talking about “arbitrary AI entities” in this context, but instead, the AI entities who will actually exist in the future, who will presumably be shaped by our training data, as well as our training methods. From this perspective, it’s not clear to me that your claim is true. But even if your claim is true, I was actually making a different point. My point was instead that it isn’t clear that future generations of AIs would be much worse than future generations of humans from an impartial utilitarian point of view.
(That said, it sounds like the real crux between us might instead be about whether pausing AI would be very costly to people who currently exist. If indeed you disagree with me about this point, I’d prefer you reply to my other comment rather than replying to this one, as I perceive that discussion as likely to be more productive.)