Well the closest analogue we have today is factory farmed animals. We use them in a way that causes tremendous suffering. We don’t really mean to cause the suffering, but it’s a by product of how we use them.
And another, perhaps even better, analogue is slavery. Maybe we’ll end up essentially enslaving digital minds because it’s useful to do so—if we were to give them too much freedom they wouldn’t as effectively do what we want them to do.
Creating digital minds just so that they can live good lives is a possibility, but I’d imagine if you would ask someone on the street if we should do this, they’d look at you like you were crazy.
Again, I’m not sure how things will pan out, and I would welcome strong arguments that suffering is unlikely, but it’s something that does worry me.
That’s true—but the difference is that both animals and slaves are sub-optimal; even our modern, highly domesticated food stock doesn’t thrive in dense factory farm conditions, nor willingly walks into the abattoir. And an ideal slave wouldn’t really be a slave, but a willing and dedicated automaton.
By contrast, we are discussing optimized machines—less optimized would mean less work being done, more resource use and less corporate profit. So we should expect more ideal digital servants (if we have them at all). A need to “enslave” them suggests that they are flawed in some way.
The dictates of evolution and nature need not apply here.
To be clear, I’m not entirely dismissing the possibility of tormented digital minds, just the notion that they are equally plausible.
You’re basically saying happier machines will be more productive and so we are likely to make them to be happy?
Firstly we don’t necessarily understand consciousness enough to know if we are making them happy, or even if they are conscious.
Also, I’m not so sure if happier means more productive. More computing power, better algorithms and more data will mean more productive. I’m open to hearing arguments why this would also mean the machine is more likely to be happy.
Maybe the causality goes the other way—more productive means more happy. If machines achieve their goals they get more satisfaction. Then maybe happiness just depends on how easy the goals we give it is. If we set AI on an intractable problem and it never fulfills it maybe it will suffer. But if AIs are constantly achieving things they will be happy.
I’m not saying you’re wrong just that it seems there’s a lot we still don’t know and the link between optimization and happiness isn’t straightforward to me.
I actually agree with a lot of this—we probably won’t intend to make them sentient at all, and it seems likely that we may do so accidentally, or that we may just not know if we have done so or not.
I’m mildly inclined to think that if ASI knows all, it can tell us when digital minds are or aren’t conscious. But it seems very plausible that we either don’t create full ASI, or that we do, but enter into a disempowerment scenario before we can rethink our choices about creating digital minds.
So yes, all that is reason to be concerned in my view. I just depart slightly from your second to last paragraph. To put a number on it, I think that this is at least half as likely as minds that are generally happy. Consciousness is a black box to me, but I think that we should as a default put more weight on a basic mechanistic theory: positive valence encourages us towards positive action, negative valence threatens us away from dis-action or apathy. The fact that we don’t observe any animals that seem dominated by one or the other seems to indicate that there is some sort of optimal equilibrium for goal fulfillment; that AI goals are different in kind from evolution’s reproductive fitness goals doesn’t seem like an obviously meaningful difference to me.
Part of your argument centers around “giving” them the wrong goals. But goals necessarily mean sub-goals—shouldn’t we expect the interior life of a digital mind to be in large part about it’s sub-goals, rather than just ultimate goals? And if it is something so intractable that it can’t even progress, wouldn’t it just stop outputting? Maybe there is suffering in that; but surely not unending suffering?
Well the closest analogue we have today is factory farmed animals. We use them in a way that causes tremendous suffering. We don’t really mean to cause the suffering, but it’s a by product of how we use them.
And another, perhaps even better, analogue is slavery. Maybe we’ll end up essentially enslaving digital minds because it’s useful to do so—if we were to give them too much freedom they wouldn’t as effectively do what we want them to do.
Creating digital minds just so that they can live good lives is a possibility, but I’d imagine if you would ask someone on the street if we should do this, they’d look at you like you were crazy.
Again, I’m not sure how things will pan out, and I would welcome strong arguments that suffering is unlikely, but it’s something that does worry me.
That’s true—but the difference is that both animals and slaves are sub-optimal; even our modern, highly domesticated food stock doesn’t thrive in dense factory farm conditions, nor willingly walks into the abattoir. And an ideal slave wouldn’t really be a slave, but a willing and dedicated automaton.
By contrast, we are discussing optimized machines—less optimized would mean less work being done, more resource use and less corporate profit. So we should expect more ideal digital servants (if we have them at all). A need to “enslave” them suggests that they are flawed in some way.
The dictates of evolution and nature need not apply here.
To be clear, I’m not entirely dismissing the possibility of tormented digital minds, just the notion that they are equally plausible.
You’re basically saying happier machines will be more productive and so we are likely to make them to be happy?
Firstly we don’t necessarily understand consciousness enough to know if we are making them happy, or even if they are conscious.
Also, I’m not so sure if happier means more productive. More computing power, better algorithms and more data will mean more productive. I’m open to hearing arguments why this would also mean the machine is more likely to be happy.
Maybe the causality goes the other way—more productive means more happy. If machines achieve their goals they get more satisfaction. Then maybe happiness just depends on how easy the goals we give it is. If we set AI on an intractable problem and it never fulfills it maybe it will suffer. But if AIs are constantly achieving things they will be happy.
I’m not saying you’re wrong just that it seems there’s a lot we still don’t know and the link between optimization and happiness isn’t straightforward to me.
I actually agree with a lot of this—we probably won’t intend to make them sentient at all, and it seems likely that we may do so accidentally, or that we may just not know if we have done so or not.
I’m mildly inclined to think that if ASI knows all, it can tell us when digital minds are or aren’t conscious. But it seems very plausible that we either don’t create full ASI, or that we do, but enter into a disempowerment scenario before we can rethink our choices about creating digital minds.
So yes, all that is reason to be concerned in my view. I just depart slightly from your second to last paragraph. To put a number on it, I think that this is at least half as likely as minds that are generally happy. Consciousness is a black box to me, but I think that we should as a default put more weight on a basic mechanistic theory: positive valence encourages us towards positive action, negative valence threatens us away from dis-action or apathy. The fact that we don’t observe any animals that seem dominated by one or the other seems to indicate that there is some sort of optimal equilibrium for goal fulfillment; that AI goals are different in kind from evolution’s reproductive fitness goals doesn’t seem like an obviously meaningful difference to me.
Part of your argument centers around “giving” them the wrong goals. But goals necessarily mean sub-goals—shouldn’t we expect the interior life of a digital mind to be in large part about it’s sub-goals, rather than just ultimate goals? And if it is something so intractable that it can’t even progress, wouldn’t it just stop outputting? Maybe there is suffering in that; but surely not unending suffering?