P1 - Currently, and historically, low power being (animals, children, old dying people) are treated very cruelly if treating them cruelly benefits the powerful even in minor ways. Weak benefits for the powerful empirically justify cruelty at scale. P2 - There is no good reason to be sure the powerful wont have even minor reasons to be cruel to the powerless (ex: suffering sub-routines, human CEV might include spreading earth like life widely or respect for tradition) P3 - Inequality between agents is likely to become much more extreme as AI develops P4 - The scale of potenital suffering will increase by many orders of magnitude
C1 - We are fucked?
Personal Note—There is also no reason to assume me or my loved ones will remain relatively powerful beings
Currently, and historically, low power being (animals, children, old dying people) are treated very cruelly if treating them cruelly benefits the powerful even in minor ways.
This is true, but far less true recently than in the past, and far less true in the near past than in the far past. That trajectory seems between somewhat promising and incredibly good—we don’t have certainty, but I think the best guess is that in fact, it’s true that the arc of history bends towards justice.
I find the following simple argument disturbing:
P1 - Currently, and historically, low power being (animals, children, old dying people) are treated very cruelly if treating them cruelly benefits the powerful even in minor ways. Weak benefits for the powerful empirically justify cruelty at scale.
P2 - There is no good reason to be sure the powerful wont have even minor reasons to be cruel to the powerless (ex: suffering sub-routines, human CEV might include spreading earth like life widely or respect for tradition)
P3 - Inequality between agents is likely to become much more extreme as AI develops
P4 - The scale of potenital suffering will increase by many orders of magnitude
C1 - We are fucked?
Personal Note—There is also no reason to assume me or my loved ones will remain relatively powerful beings
C2 - Im really fucked!
This is true, but far less true recently than in the past, and far less true in the near past than in the far past. That trajectory seems between somewhat promising and incredibly good—we don’t have certainty, but I think the best guess is that in fact, it’s true that the arc of history bends towards justice.