(1) That’s not quite how I’d characterize the current technical agenda. Rather, I’d say that in order to build an AI aligned with human interests, you need to do three things: (a) understand how to build an AI that’s aligned with anything (could you build an AI that reliably builds as much diamond as possible?), (b) understand how to build an AI that assists you in correcting things-you-perceive-as-flaws (this doesn’t come for free, but it’s pretty important, because humans are bad at getting software right on the first try), and (c) figure out how to build a machine that can safely learn human values & intentions from training data.
We’re currently splitting our time between all these problems. It’s not that we haven’t focused on the value learning problem yet, rather, it’s that the value learning problem is only a fraction of the whole problem. We’ll keep working on all the parts, and I’m not sure which parts will yield first. I can’t give you a timeline on how long various parts will take; scientific progress is very hard to predict.
(2) I wouldn’t currently say that “formal logic is the arena in which MIRI’s technical work takes place”—if anything, “math in general” is the arena, and that will probably remain the case until we have a much better understanding of the problems we’re trying to solve (and how to solve simplified versions of them), at which point computer programming will become much more essential. Again, it’s hard to say how long it will take to get there, because scientific progress is hard to predict.
Formal logic is one of many tools useful in mathematics (alongside probability theory, statistics, linear algebra, etc.) that shows up fairly frequently in our work, but I don’t think of our work as “focused on formal logic.” I don’t think we’ll “move away from formal logic” at a particular time; rather, we’ll just use whichever mathematical tools look useful for the problems at hand. That will change as the problems change :-)
(1) That’s not quite how I’d characterize the current technical agenda. Rather, I’d say that in order to build an AI aligned with human interests, you need to do three things: (a) understand how to build an AI that’s aligned with anything (could you build an AI that reliably builds as much diamond as possible?), (b) understand how to build an AI that assists you in correcting things-you-perceive-as-flaws (this doesn’t come for free, but it’s pretty important, because humans are bad at getting software right on the first try), and (c) figure out how to build a machine that can safely learn human values & intentions from training data.
We’re currently splitting our time between all these problems. It’s not that we haven’t focused on the value learning problem yet, rather, it’s that the value learning problem is only a fraction of the whole problem. We’ll keep working on all the parts, and I’m not sure which parts will yield first. I can’t give you a timeline on how long various parts will take; scientific progress is very hard to predict.
(2) I wouldn’t currently say that “formal logic is the arena in which MIRI’s technical work takes place”—if anything, “math in general” is the arena, and that will probably remain the case until we have a much better understanding of the problems we’re trying to solve (and how to solve simplified versions of them), at which point computer programming will become much more essential. Again, it’s hard to say how long it will take to get there, because scientific progress is hard to predict.
Formal logic is one of many tools useful in mathematics (alongside probability theory, statistics, linear algebra, etc.) that shows up fairly frequently in our work, but I don’t think of our work as “focused on formal logic.” I don’t think we’ll “move away from formal logic” at a particular time; rather, we’ll just use whichever mathematical tools look useful for the problems at hand. That will change as the problems change :-)
Thank you for the response; it was helpful :^)