I agree that more of both is needed. Both need to be instantiated in actual code, though. And both are useless if researchers don’t care implement them.
I admit I would benefit from some clarification on your point—are you arguing that the article assumes a bug-free AI won’t cause AI accidents? Is it the case that this arose from Amodei et al.’s definition?: “unintended and harmful behavior that may emerge from poor design of real-world AI systems”. Poor design of real world AI systems isn’t limited to being bug-free, but I can see why this might have caused confusion.
are you arguing that the article assumes a bug-free AI won’t cause AI accidents?
I’m not—I’m saying that when you phrase it as accidents then it creates flawed perceptions about the nature and scope of the problem. An accident sounds like a onetime event that a system causes in the course of its performance; AI risk is about systems whose performance itself is fundamentally destructive. Accidents are aberrations from normal system behavior; the core idea of AI risk is that any known specification of system behavior, when followed comprehensively by advanced AI, is not going to work.
Better decision theory, which is much of what MIRI does, and better guiding philosophy.
I agree that more of both is needed. Both need to be instantiated in actual code, though. And both are useless if researchers don’t care implement them.
I admit I would benefit from some clarification on your point—are you arguing that the article assumes a bug-free AI won’t cause AI accidents? Is it the case that this arose from Amodei et al.’s definition?: “unintended and harmful behavior that may emerge from poor design of real-world AI systems”. Poor design of real world AI systems isn’t limited to being bug-free, but I can see why this might have caused confusion.
I’m not—I’m saying that when you phrase it as accidents then it creates flawed perceptions about the nature and scope of the problem. An accident sounds like a onetime event that a system causes in the course of its performance; AI risk is about systems whose performance itself is fundamentally destructive. Accidents are aberrations from normal system behavior; the core idea of AI risk is that any known specification of system behavior, when followed comprehensively by advanced AI, is not going to work.