Jeremy—I think it will be important to develop compelling arguments about why it’s worth preserving humanity—not so much to convince an ASI not to kill us all, but to remind ordinary citizens why AI X risk is bad, and to persuade transhumanists/posthumanists/Singularity-enthusiasts that replacing humanity ASAP with machine intelligences is a ghoulish, psychopathic, & repulsive ideology.
The trouble is, the standard 20th century arguments for preserving humanity in the face of AI are wearing a bit thin. A lot of those arguments involved sentimental appeals to human creativity, e.g. ‘We should preserve humans because humans make wonderful art, music, literature, science, and humor that AI won’t be able to match’. Well, AI is getting increasingly better than average humans at all of these domains, so appeals to human creativity won’t be very compelling.
Jeremy—I think it will be important to develop compelling arguments about why it’s worth preserving humanity—not so much to convince an ASI not to kill us all, but to remind ordinary citizens why AI X risk is bad, and to persuade transhumanists/posthumanists/Singularity-enthusiasts that replacing humanity ASAP with machine intelligences is a ghoulish, psychopathic, & repulsive ideology.
The trouble is, the standard 20th century arguments for preserving humanity in the face of AI are wearing a bit thin. A lot of those arguments involved sentimental appeals to human creativity, e.g. ‘We should preserve humans because humans make wonderful art, music, literature, science, and humor that AI won’t be able to match’. Well, AI is getting increasingly better than average humans at all of these domains, so appeals to human creativity won’t be very compelling.
What other arguments can we offer?