This essentially refers to the the implicit value system that would emerge if, when advanced AI is eventually created, you gave the then-currently existing set of humans a lot of wealth. Call these values PSIRPEHTA (I’m working on a better acronym).
I basically buy that the values we get will be similar to just giving existing humans massive amounts of wealth, but I’m less sold that this will result in outcomes which are well described as “primarily selfish”.
I feel like your comment is equivocating between “the situation is similar to making existing humans massively wealth” and “of course this will result in primarily selfish usage similar to how the median person behaves with marginal money now”.
I basically buy that the values we get will be similar to just giving existing humans massive amounts of wealth, but I’m less sold that this will result in outcomes which are well described as “primarily selfish”.
Current humans definitely seem primarily selfish (although I think they also care about their family and friends too; I’m including that). Can you explain why you think giving humans a lot of wealth would turn them into something that isn’t primarily selfish? What’s the empirical evidence for that idea?
The behavior of billionares, which maybe indicates more like 10% of income spent on altruism.
ETA: This is still literally majority selfish, but it’s also plausible that 10% altruism is pretty great and looks pretty different than “current median person behavior with marginal money”.
(See my other comment about the percent of cosmic resources.)
The idea that billionaires have 90% selfish values seems consistent with a claim of having “primarily selfish” values in my opinion. Can you clarify what you’re objecting to here?
The literal words of “primarily selfish” don’t seem that bad, but I would maybe prefer majority selfish?
And your top level comment seems like it’s not talking about/emphasizing the main reason to like human control which is that maybe 10-20% of resources are spent well.
It just seemed odd to me to not mention that “primarily selfish” still involves a pretty big fraction of altruism.
I agree it’s important to talk about and analyze the (relatively small) component of human values that are altruistic. I mostly just think this component is already over-emphasized.
Here’s one guess at what I think you might be missing about my argument: 90% selfish values + 10% altruistic values isn’t the same thing as, e.g., 90% valueless stuff + 10% utopia. The 90% selfish component can have negative effects on welfare from a total utilitarian perspective, that aren’t necessarily outweighed by the 10%.
90% selfish values is the type of thing that produces massive factory farming infrastructure, with a small amount of GDP spent mitigating suffering in factory farms. Does the small amount of spending mitigating suffering outweigh the large amount of spending directly causing suffering? This isn’t clear to me.
(Alternatively, you could think that unaligned AIs will be 100% selfish, and this is clearly worse. But I’d want to understand how you could come to that conclusion, carefully. “Altruism” also encompasses a broad range of activities, and not all of it is utopian or idealistic from a total utilitarian perspective. For example, human spending on environmental conservation might be categorized as “altruism” in this framework, although personally I would say that form of spending is not very “moral” due to wild animal suffering.)
The 90% selfish component can have negative effects on welfare from a total utilitarian perspective, that aren’t necessarily outweighed by the 10%.
Yep, this can be true, but I’m skeptical this will matter much in practice.
I typically think things which aren’t directly optimizing for value or disvalue won’t have intended effects which are very important and that in the future unintended effects (externalities) won’t be that much of total value/disvalue.
When we see the selfish consumption of current very rich people, it doesn’t seem like the intentional effects are that morally good/bad relative to the best/worst uses of resources. (E.g. owning a large boat and having people think you’re high status aren’t that morally important relative to altruistic spending of similar amounts of money.) So for current very rich people the main issue would be that the economic process for producing the goods has bad externalities.
And, I expect that as technology advances, externalities reduce in moral importance relative to intended effects. Partially this is based on crazy transhumanist takes, but I feel like there is some broader perspective in which you’d expect this.
E.g. for factory farming, the ultimately cheapest way to make meat in the limit of technological maturity would very likely not involve any animal suffering.
Separately, I think externalities will probably look pretty similar for selfish resource usage for unaligned AIs and humans because most serious economic activities will be pretty similar.
Alternatively, you could think that unaligned AIs will be 100% selfish, and this is clearly worse.
I’d like to explicitly note that this I don’t think that this is true in expectation for a reasonable notion of “selfish”. Though I maybe think something which is sort of in this direction if we use a relatively narrow notion of altruism.
How are we defining selfish here? It seem like a pretty strong position to take on the topic of psychological egoism? Especially including family/friends in terms of selfish?
In your original post, you say:
All that extra wealth did not make us extreme moral saints; instead, we still mostly care about ourselves, our family, and our friends.
But I don’t know, it seems that as countries and individuals get wealthier, we seem to on the whole be getting better? Maybe factory farming acts against this, but the idea that factory farming is immoral and should be abolished exists and I think is only going to grow. I don’t think the humans are just slaves to our base wants/desires, and think that is a remarkably impoverished view of both individual human pyschology and social morality.
As such, I don’t really agree with much of this post. An AGI, when built, will be able to generate new ideas and hypotheses about the world, including moral ones. A strong-but-narrow AI could be worse (e.g. optimal-factory-farm-PT), but then the right response here isn’t really technical alignment, it’s AI governance and moral persuasion in general.
I basically buy that the values we get will be similar to just giving existing humans massive amounts of wealth, but I’m less sold that this will result in outcomes which are well described as “primarily selfish”.
I feel like your comment is equivocating between “the situation is similar to making existing humans massively wealth” and “of course this will result in primarily selfish usage similar to how the median person behaves with marginal money now”.
Current humans definitely seem primarily selfish (although I think they also care about their family and friends too; I’m including that). Can you explain why you think giving humans a lot of wealth would turn them into something that isn’t primarily selfish? What’s the empirical evidence for that idea?
The behavior of billionares, which maybe indicates more like 10% of income spent on altruism.
ETA: This is still literally majority selfish, but it’s also plausible that 10% altruism is pretty great and looks pretty different than “current median person behavior with marginal money”.
(See my other comment about the percent of cosmic resources.)
The idea that billionaires have 90% selfish values seems consistent with a claim of having “primarily selfish” values in my opinion. Can you clarify what you’re objecting to here?
The literal words of “primarily selfish” don’t seem that bad, but I would maybe prefer majority selfish?
And your top level comment seems like it’s not talking about/emphasizing the main reason to like human control which is that maybe 10-20% of resources are spent well.
It just seemed odd to me to not mention that “primarily selfish” still involves a pretty big fraction of altruism.
I agree it’s important to talk about and analyze the (relatively small) component of human values that are altruistic. I mostly just think this component is already over-emphasized.
Here’s one guess at what I think you might be missing about my argument: 90% selfish values + 10% altruistic values isn’t the same thing as, e.g., 90% valueless stuff + 10% utopia. The 90% selfish component can have negative effects on welfare from a total utilitarian perspective, that aren’t necessarily outweighed by the 10%.
90% selfish values is the type of thing that produces massive factory farming infrastructure, with a small amount of GDP spent mitigating suffering in factory farms. Does the small amount of spending mitigating suffering outweigh the large amount of spending directly causing suffering? This isn’t clear to me.
(Alternatively, you could think that unaligned AIs will be 100% selfish, and this is clearly worse. But I’d want to understand how you could come to that conclusion, carefully. “Altruism” also encompasses a broad range of activities, and not all of it is utopian or idealistic from a total utilitarian perspective. For example, human spending on environmental conservation might be categorized as “altruism” in this framework, although personally I would say that form of spending is not very “moral” due to wild animal suffering.)
Yep, this can be true, but I’m skeptical this will matter much in practice.
I typically think things which aren’t directly optimizing for value or disvalue won’t have intended effects which are very important and that in the future unintended effects (externalities) won’t be that much of total value/disvalue.
When we see the selfish consumption of current very rich people, it doesn’t seem like the intentional effects are that morally good/bad relative to the best/worst uses of resources. (E.g. owning a large boat and having people think you’re high status aren’t that morally important relative to altruistic spending of similar amounts of money.) So for current very rich people the main issue would be that the economic process for producing the goods has bad externalities.
And, I expect that as technology advances, externalities reduce in moral importance relative to intended effects. Partially this is based on crazy transhumanist takes, but I feel like there is some broader perspective in which you’d expect this.
E.g. for factory farming, the ultimately cheapest way to make meat in the limit of technological maturity would very likely not involve any animal suffering.
Separately, I think externalities will probably look pretty similar for selfish resource usage for unaligned AIs and humans because most serious economic activities will be pretty similar.
I’d like to explicitly note that this I don’t think that this is true in expectation for a reasonable notion of “selfish”. Though I maybe think something which is sort of in this direction if we use a relatively narrow notion of altruism.
How are we defining selfish here? It seem like a pretty strong position to take on the topic of psychological egoism? Especially including family/friends in terms of selfish?
In your original post, you say:
But I don’t know, it seems that as countries and individuals get wealthier, we seem to on the whole be getting better? Maybe factory farming acts against this, but the idea that factory farming is immoral and should be abolished exists and I think is only going to grow. I don’t think the humans are just slaves to our base wants/desires, and think that is a remarkably impoverished view of both individual human pyschology and social morality.
As such, I don’t really agree with much of this post. An AGI, when built, will be able to generate new ideas and hypotheses about the world, including moral ones. A strong-but-narrow AI could be worse (e.g. optimal-factory-farm-PT), but then the right response here isn’t really technical alignment, it’s AI governance and moral persuasion in general.