At a glance, the only mentions of âlong-term memoryâ in If Anyone Builds It, Everyone Dies (IABIED) by Yudkowsky and Soares are in the context of a short science fiction story. Itâs just stipulated that a fictional AI system has it. Itâs only briefly mentioned, and there is no explanation of what long-term memory consists of, how it works, or how it was developed. Itâs similar to Dataâs âpositronic brainâ in Star Trek: The Next Generation or any number of hand-waved technological concepts in sci-fi. Do you think this is a good answer to my objection? If so, can you explain why?
Unless there are other passages from the book that Iâm missing (maybe because they use a different phrasing), the mentions of âlong-term memoryâ in the book donât seem to have the centrality to the authorsâ arguments or predictions that you implied. I donât see textual evidence that Yudkowsky and Soares âname continual learning as the major breakthrough requiredâ, unless you count those very brief mentions in the sci-fi story.
I think one of the main problems with AI 2027â˛s story is the impossibility of using current AI systems for AI research, as I discussed in the post. There is a chicken-and-egg problem. LLMs are currently useless at actually doing research (as opposed to just helping with research as a search engine, in the exactly same way Google helps with research). AI systems with extremely weak generalization, extremely poor data efficiency, and without continual learning (or online learning) cannot plausibly do research well. Somehow this challenge has to be overcome, and obviously an AI that canât do research canât do the research to give itself the capabilities required to do research. (That would be an AI pulling itself up by its bootstraps. Or, in the terminology of the philosopher Daniel Dennett, it would be a skyhook.) So, it comes down to human researches to solve this challenge.
From what Iâve seen, when AI 2027 talks about an AI discovery/âinnovation being made by human researchers, the authors just kind of give their subjective best guess of how long that discovery/âinnovation will take to be made. This is not a new or original objection, of course, but I share the same objection as others. Any of these discoveries/âinnovations could take a tenth as long or ten times as long as the authors guess. So, AI 2027 isnât a scientific model (which I donât think is what the authors explicitly claim, although thatâs the impression some people seem to have gotten). Rather, itâs an aggregation of a few peopleâs intuitions. It doesnât really serve as a persuasive piece of argumentation for most people for mainly that reason.
At a glance, the only mentions of âlong-term memoryâ in If Anyone Builds It, Everyone Dies (IABIED) by Yudkowsky and Soares are in the context of a short science fiction story. Itâs just stipulated that a fictional AI system has it. Itâs only briefly mentioned, and there is no explanation of what long-term memory consists of, how it works, or how it was developed. Itâs similar to Dataâs âpositronic brainâ in Star Trek: The Next Generation or any number of hand-waved technological concepts in sci-fi. Do you think this is a good answer to my objection? If so, can you explain why?
Unless there are other passages from the book that Iâm missing (maybe because they use a different phrasing), the mentions of âlong-term memoryâ in the book donât seem to have the centrality to the authorsâ arguments or predictions that you implied. I donât see textual evidence that Yudkowsky and Soares âname continual learning as the major breakthrough requiredâ, unless you count those very brief mentions in the sci-fi story.
I think one of the main problems with AI 2027â˛s story is the impossibility of using current AI systems for AI research, as I discussed in the post. There is a chicken-and-egg problem. LLMs are currently useless at actually doing research (as opposed to just helping with research as a search engine, in the exactly same way Google helps with research). AI systems with extremely weak generalization, extremely poor data efficiency, and without continual learning (or online learning) cannot plausibly do research well. Somehow this challenge has to be overcome, and obviously an AI that canât do research canât do the research to give itself the capabilities required to do research. (That would be an AI pulling itself up by its bootstraps. Or, in the terminology of the philosopher Daniel Dennett, it would be a skyhook.) So, it comes down to human researches to solve this challenge.
From what Iâve seen, when AI 2027 talks about an AI discovery/âinnovation being made by human researchers, the authors just kind of give their subjective best guess of how long that discovery/âinnovation will take to be made. This is not a new or original objection, of course, but I share the same objection as others. Any of these discoveries/âinnovations could take a tenth as long or ten times as long as the authors guess. So, AI 2027 isnât a scientific model (which I donât think is what the authors explicitly claim, although thatâs the impression some people seem to have gotten). Rather, itâs an aggregation of a few peopleâs intuitions. It doesnât really serve as a persuasive piece of argumentation for most people for mainly that reason.