Hi Cate, thank you for your courage to express potentially controversial claims, and I upvoted (but not strongly) for this reason.
I am not a computer or AI scientist. But my guess is that you are probably right, if by “predictable” we mean “predictable to humans only”. For example, in a paper (not yet published) Peter Singer and I argue that self-driving cars should identify animals that might be on the way and dodge them. But we are aware that the costs of detection and computation will rise, and that the AI will have more constraints in its optimization problem. As a results the cars might be more expensive and they might be willing sacrifice some human welfare, such as by causing discomfort or scare to passengers while braking violently for a rat crossing.
But maybe this is not a reason to worry. If, like how most of the stakes/wellbeing lie in the future, most of the stakes and wellbeing lie with nonhuman animals, maybe that’s a bullet we need to bite. We (longtermists) probably wouldn’t say we worry that if an AI cares about the whole future it would be a lot less predictable with respect to the welfare of current people, we are likely to say this is how it should be.
Another reason to not over-worry is that human economics will probably constrain that from happening to a high extent. Using the self-driving car example again, if some companies’ cars care about animals, some don’t, the cars that don’t will, other things being equal, be cheaper and safer for humans. So unless we so miraculously convince all car producers to take care of animals, we probably won’t have the “problem” (which for me, that we won’t get “that problem” is the actual problem). The point probably goes beyond just economics, politics, culture, human psychology, possibly all have similar effects. My sense is that as far as humans are in control of the development of AI, AI is more likely to be too humancentric than not being humancentric enough.
Hi Cate, thank you for your courage to express potentially controversial claims, and I upvoted (but not strongly) for this reason.
I am not a computer or AI scientist. But my guess is that you are probably right, if by “predictable” we mean “predictable to humans only”. For example, in a paper (not yet published) Peter Singer and I argue that self-driving cars should identify animals that might be on the way and dodge them. But we are aware that the costs of detection and computation will rise, and that the AI will have more constraints in its optimization problem. As a results the cars might be more expensive and they might be willing sacrifice some human welfare, such as by causing discomfort or scare to passengers while braking violently for a rat crossing.
But maybe this is not a reason to worry. If, like how most of the stakes/wellbeing lie in the future, most of the stakes and wellbeing lie with nonhuman animals, maybe that’s a bullet we need to bite. We (longtermists) probably wouldn’t say we worry that if an AI cares about the whole future it would be a lot less predictable with respect to the welfare of current people, we are likely to say this is how it should be.
Another reason to not over-worry is that human economics will probably constrain that from happening to a high extent. Using the self-driving car example again, if some companies’ cars care about animals, some don’t, the cars that don’t will, other things being equal, be cheaper and safer for humans. So unless we so miraculously convince all car producers to take care of animals, we probably won’t have the “problem” (which for me, that we won’t get “that problem” is the actual problem). The point probably goes beyond just economics, politics, culture, human psychology, possibly all have similar effects. My sense is that as far as humans are in control of the development of AI, AI is more likely to be too humancentric than not being humancentric enough.