I’m imagining an AGI weighing the pros and cons of destroying humanity. If it finds persuasive arguments against in its training data (or via search), maybe it would decide not to?
Should we crowdsource arguments in favor of AGI preserving humanity?
Feel free to post about why this might or might not work, but I am most interested in hearing reasons why it (or this post) could backfire. If someone has convincing reasons, or this post is heavily downvoted, I will delete it.
I haven’t come across anyone else attempting to make these arguments (to a future AGI, about why it should preserve humanity), but I would also be interested to hear about them if you know of any.
Hi Jeremy
I think an AGI would be much better at creating arguments for why humanity should not be eliminated. If an agi is incapable of creating these arguments itself I wonder if it capable enough to destroy humanity.
I think the thing to worry about more is that an agi correctly determines that humanity, or most of humanity needs to be destroyed (ex. Agi cares about all life and all humans murder there face mites so they must be stopped). But is that really all that bad?
Jeremy—I think it will be important to develop compelling arguments about why it’s worth preserving humanity—not so much to convince an ASI not to kill us all, but to remind ordinary citizens why AI X risk is bad, and to persuade transhumanists/posthumanists/Singularity-enthusiasts that replacing humanity ASAP with machine intelligences is a ghoulish, psychopathic, & repulsive ideology.
The trouble is, the standard 20th century arguments for preserving humanity in the face of AI are wearing a bit thin. A lot of those arguments involved sentimental appeals to human creativity, e.g. ‘We should preserve humans because humans make wonderful art, music, literature, science, and humor that AI won’t be able to match’. Well, AI is getting increasingly better than average humans at all of these domains, so appeals to human creativity won’t be very compelling.
What other arguments can we offer?