I’d love to see research into what I called “human safety problems” (or sometimes “human-AI safety”), fleshing out the idea more or giving some empirical evidence as to how much of a problem it really is. Here’s a short description of the idea from AI design as opportunity and obligation to address human safety problems:
Many AI safety problems are likely to have counterparts in humans. AI designers and safety researchers shouldn’t start by assuming that humans are safe (and then try to inductively prove that increasingly powerful AI systems are safe when developed/trained by and added to a team of humans) or try to solve AI safety problems without considering whether their designs or safety approaches exacerbate human safety problems relative to other designs / safety approaches. At the same time, the development of AI may be a huge opportunity to address human safety problems, for example by transferring power from probably unsafe humans to de novo AIs that are designed from the ground up to be safe, or by assisting humans’ built-in safety mechanisms (such as moral and philosophical reflection).
I’d love to see research into what I called “human safety problems” (or sometimes “human-AI safety”), fleshing out the idea more or giving some empirical evidence as to how much of a problem it really is. Here’s a short description of the idea from AI design as opportunity and obligation to address human safety problems:
I go into a bit more detail in Two Neglected Problems in Human-AI Safety.
Wei Dai—thanks for these helpful links! Will have a look. :)