In retrospect we know that the AI project couldn’t possibly have succeeded at that stage. The hardware was simply not powerful enough. It seems that at least about 100 Tops is required for human-like performance, and possibly as much as 10^17 ops is needed. The computers in the seventies had a computing power comparable to that of insects. They also achieved approximately insect-level intelligence.
I would have guessed this is just a funny quip, in the sense that (i) it sure sounds like it’s just a throw-away quip, no evidence is presented for those AI systems being competent at anything (he moves on to other topics in the next sentence), “approximately insect-level” seems appropriate as a generic and punchy stand in for “pretty dumb,” (ii) in the document he is basically just thinking about AI performance on complex tasks and trying to make the point that you shouldn’t be surprised by subhuman performance on those tasks, which doesn’t depend much on the literal comparison to insects, (iii) the actual algorithms described in the section (neural nets and genetic algorithms) wouldn’t plausibly achieve insect-level performance in the 70s since those algorithms in fact do require large training processes (and were in fact used in the 70s to train much tinier neural networks).
(Of course you could also just ask Nick.)
I also think it’s worth noting that the prediction in that section looks reasonably good in hindsight. It was written right at the beginning of resurgent interest in neural networks (right before Yann LeCun’s paper on MNIST with neural networks). The hypothesis “computers were too small in the past so that’s why they were lame” looks like it was a great call, and Nick’s tentative optimism about particular compute-heavy directions looks good. I think overall this is a significantly better take than mainstream opinions in AI. I don’t think this literally affects your point, but it is relevant if the implicit claim is “And people talking about insect comparisons were lead astray by these comparisons.”
I suspect you are more broadly underestimating the extent to which people used “insect-level intelligence” as a generic stand-in for “pretty dumb,” though I haven’t looked at the discussion in Mind Children and Moravec may be making a stronger claim. I’d be more inclined to tread carefully if some historical people tried to actually compare the behavior of their AI system to the behavior of an insect and found it comparable as in posts like this one (it’s not clear to me how such an evaluation would have suggested insect-level robotics in the 90s or even today, I think the best that can be said is that today it seems compatible with insect-level robotics in simulation today). I’ve seen Moravec use the phrase “insect-level intelligence” to refer to the particular behaviors of “following pheromone trails” or “flying towards lights,” so I might also read him as referring to those behaviors in particular. (It’s possible he is underestimating the total extent of insect intelligence, e.g. discounting the complex motor control performed by insects, though I haven’t seen him do that explicitly and it would be a bit off brand.)
ETA: While I don’t think 1990s robotics could plausibly be described as “insect-level,” I actually do think that the linked post on bee vision could plausibly have been written in the 90s and concluded that computer vision was bee-level, it’s just a very hard comparison to make and the performance of the bees in the formal task is fairly unimpressive.
I suspect you are more broadly underestimating the extent to which people used “insect-level intelligence” as a generic stand-in for “pretty dumb,” though I haven’t looked at the discussion in Mind Children and Moravec may be making a stronger claim.
I think that’s good push-back and a fair suggestion: I’m not sure how seriously the statement in Nick’s paper was meant to be taken. I hadn’t considered that it might be almost entirely a quip. (I may ask him about this.)
Moravec’s discussion in Mind Children is similarly brief: He presents a graph of the computing power of different animal’s brains and states that “lab computers are roughly equal in power to the nervous systems of insects.”He also characterizes current AI behaviors as “insectlike” and writes: “I believe that robots with human intelligence will be common within fifty years. By comparison, the best of today’s machines have minds more like those of insects than humans. Yet this performance itself represents a giant leap forward in just a few decades.” I don’t think he’s just being quippy, but there’s also no suggestion that he means anything very rigorous/specific by his suggestion.
Rodney Brooks, I think, did mean for his comparisons to insect intelligence to be taken very seriously. The idea of his “nouvelle AI program” was to create AI systems that match insect intelligence, then use that as a jumping-off point for trying to produce human-like intelligence. I think walking and obstacle navigation, with several legs, was used as the main dimension of comparison. The Brooks case is a little different, though, since (IIRC) he only claimed that his robots exhibited important aspects of insect intelligence or fell just short insect intelligence, rather than directly claiming that they actually matched insect intelligence. On the other hand, he apparently felt he had gotten close enough to transition to the stage of the project that was meant to go from insect-level stuff to human-level stuff.
A plausible reaction to these cases, then, might be:
OK, Rodney Brooks did make a similar comparison, and was a major figure at the time, but his stuff was pretty transparently flawed. Moravec’s and Bostrom’s comments were at best fairly off-hand, suggesting casual impressions more than they suggest outcomes of rigorous analysis. The more recent “insect-level intelligence” claim is pretty different, since it’s built on top of much more detailed analysis than anything Moravec/Bostrom did, and it’s less obviously flawed than Brooks’ analysis. The likelihood that it reflects an erroneous impression is, therefore, a lot lower. The previous cases shouldn’t actually do much to raise our suspicion levels.
I think there’s something to this reaction, particularly if there’s now more rigorous work being done to operationalize and test the “insect-level intelligence” claim. I hadn’t yet seen the recent post you linked to, which, at first glance, seems like a good and clear piece of work. The more rigorous work is done to flesh out the argument, the less I’m inclined to treat the Bostrom/Moravec/Brooks cases as part of an epistemically relevant reference class.
My impression a few years ago was that the claim wasn’t yet backed by any really clear/careful analysis. At least, the version that filtered down to me seemed to be substantially based on fuzzy analogies between RL agent behavior and insect behavior, without anyone yet knowing much about insect behavior. (Although maybe this was a misimpression.) So I probably do stand by the reference class being relevant back then.
Overall, to sum up, my position here is something like: “The Bostrom/Moravec/Brooks cases do suggest that it might be easy to see roughly insect-level intelligence, if that’s what you expect to see and you’re relying on fuzzy impressions, paying special attention to stuff AI systems can already do, or not really operationalizing your claims. This should make us more suspicious of modern claims that we’ve recently achieved ‘insect-level intelligence,’ unless they’re accompanied by transparent and pretty obviously robust reasoning. Insofar as this work is being done, though, the Bostrom/Moravec/Brooks cases become weaker grounds for suspicion.”
I do think my main impression of insect <-> simulated robot parity comes from very fuzzy evaluations of insect motor control vs simulated robot motor control (rather than from any careful analysis, of which I’m a bit more skeptical though I do think it’s a relevant indicator that we are at least trying to actually figure out the answer here in a way that wasn’t true historically). And I do have only a passing knowledge of insect behavior, from watching youtube videos and reading some book chapters about insect learning. So I don’t think it’s unfair to put it in the same reference class as Rodney Brooks’ evaluations to the extent that his was intended as a serious evaluation.
The Nick Bostrom quote (from here) is:
I would have guessed this is just a funny quip, in the sense that (i) it sure sounds like it’s just a throw-away quip, no evidence is presented for those AI systems being competent at anything (he moves on to other topics in the next sentence), “approximately insect-level” seems appropriate as a generic and punchy stand in for “pretty dumb,” (ii) in the document he is basically just thinking about AI performance on complex tasks and trying to make the point that you shouldn’t be surprised by subhuman performance on those tasks, which doesn’t depend much on the literal comparison to insects, (iii) the actual algorithms described in the section (neural nets and genetic algorithms) wouldn’t plausibly achieve insect-level performance in the 70s since those algorithms in fact do require large training processes (and were in fact used in the 70s to train much tinier neural networks).
(Of course you could also just ask Nick.)
I also think it’s worth noting that the prediction in that section looks reasonably good in hindsight. It was written right at the beginning of resurgent interest in neural networks (right before Yann LeCun’s paper on MNIST with neural networks). The hypothesis “computers were too small in the past so that’s why they were lame” looks like it was a great call, and Nick’s tentative optimism about particular compute-heavy directions looks good. I think overall this is a significantly better take than mainstream opinions in AI. I don’t think this literally affects your point, but it is relevant if the implicit claim is “And people talking about insect comparisons were lead astray by these comparisons.”
I suspect you are more broadly underestimating the extent to which people used “insect-level intelligence” as a generic stand-in for “pretty dumb,” though I haven’t looked at the discussion in Mind Children and Moravec may be making a stronger claim. I’d be more inclined to tread carefully if some historical people tried to actually compare the behavior of their AI system to the behavior of an insect and found it comparable as in posts like this one (it’s not clear to me how such an evaluation would have suggested insect-level robotics in the 90s or even today, I think the best that can be said is that today it seems compatible with insect-level robotics in simulation today). I’ve seen Moravec use the phrase “insect-level intelligence” to refer to the particular behaviors of “following pheromone trails” or “flying towards lights,” so I might also read him as referring to those behaviors in particular. (It’s possible he is underestimating the total extent of insect intelligence, e.g. discounting the complex motor control performed by insects, though I haven’t seen him do that explicitly and it would be a bit off brand.)
ETA: While I don’t think 1990s robotics could plausibly be described as “insect-level,” I actually do think that the linked post on bee vision could plausibly have been written in the 90s and concluded that computer vision was bee-level, it’s just a very hard comparison to make and the performance of the bees in the formal task is fairly unimpressive.
I think that’s good push-back and a fair suggestion: I’m not sure how seriously the statement in Nick’s paper was meant to be taken. I hadn’t considered that it might be almost entirely a quip. (I may ask him about this.)
Moravec’s discussion in Mind Children is similarly brief: He presents a graph of the computing power of different animal’s brains and states that “lab computers are roughly equal in power to the nervous systems of insects.”He also characterizes current AI behaviors as “insectlike” and writes: “I believe that robots with human intelligence will be common within fifty years. By comparison, the best of today’s machines have minds more like those of insects than humans. Yet this performance itself represents a giant leap forward in just a few decades.” I don’t think he’s just being quippy, but there’s also no suggestion that he means anything very rigorous/specific by his suggestion.
Rodney Brooks, I think, did mean for his comparisons to insect intelligence to be taken very seriously. The idea of his “nouvelle AI program” was to create AI systems that match insect intelligence, then use that as a jumping-off point for trying to produce human-like intelligence. I think walking and obstacle navigation, with several legs, was used as the main dimension of comparison. The Brooks case is a little different, though, since (IIRC) he only claimed that his robots exhibited important aspects of insect intelligence or fell just short insect intelligence, rather than directly claiming that they actually matched insect intelligence. On the other hand, he apparently felt he had gotten close enough to transition to the stage of the project that was meant to go from insect-level stuff to human-level stuff.
A plausible reaction to these cases, then, might be:
I think there’s something to this reaction, particularly if there’s now more rigorous work being done to operationalize and test the “insect-level intelligence” claim. I hadn’t yet seen the recent post you linked to, which, at first glance, seems like a good and clear piece of work. The more rigorous work is done to flesh out the argument, the less I’m inclined to treat the Bostrom/Moravec/Brooks cases as part of an epistemically relevant reference class.
My impression a few years ago was that the claim wasn’t yet backed by any really clear/careful analysis. At least, the version that filtered down to me seemed to be substantially based on fuzzy analogies between RL agent behavior and insect behavior, without anyone yet knowing much about insect behavior. (Although maybe this was a misimpression.) So I probably do stand by the reference class being relevant back then.
Overall, to sum up, my position here is something like: “The Bostrom/Moravec/Brooks cases do suggest that it might be easy to see roughly insect-level intelligence, if that’s what you expect to see and you’re relying on fuzzy impressions, paying special attention to stuff AI systems can already do, or not really operationalizing your claims. This should make us more suspicious of modern claims that we’ve recently achieved ‘insect-level intelligence,’ unless they’re accompanied by transparent and pretty obviously robust reasoning. Insofar as this work is being done, though, the Bostrom/Moravec/Brooks cases become weaker grounds for suspicion.”
I do think my main impression of insect <-> simulated robot parity comes from very fuzzy evaluations of insect motor control vs simulated robot motor control (rather than from any careful analysis, of which I’m a bit more skeptical though I do think it’s a relevant indicator that we are at least trying to actually figure out the answer here in a way that wasn’t true historically). And I do have only a passing knowledge of insect behavior, from watching youtube videos and reading some book chapters about insect learning. So I don’t think it’s unfair to put it in the same reference class as Rodney Brooks’ evaluations to the extent that his was intended as a serious evaluation.
Yeah, FWIW I haven’t found any recent claims about insect comparisons particularly rigorous.