Hi Arturo. Thank you for the thoughtful and detailed assessment of the AI risk literature. Here are a few other sources you might be interested in reading:
AI Timelines: Where the arguments and the “experts” stand summarizes key sources of evidence on AI timelines. Namely, it finds that AI researchers believe AGI will likely arrive within the next few decades, that the human brain uses more computational power than today’s largest AI models but that future models will soon surpass human levels of compute, and that economic history suggests transformative changes to growth regimes are absolutely possible.
Jacob Cannell provides more details on the amount of computational power used by various biological and artificial systems. “The Table” is quite jarring to me.
Economic Growth Under Transformative AI by Phil Trammell and Anton Korinek reviews the growth theory literature in economics, finding that mainstream theories of economic growth admit the possibility of a “singularity” driven by artificial intelligence.
Tom Davidson’s model uses growth theory to specifically model AI progress. He assumes that AI will be able to perform 100% of economically relevant tasks once it uses the same amount of computation as the human brain. The model shows that this would lead to “fast takeoff”: the world will look very normal, yet in a matter of only a few years could see >30% GDP growth and the advent of superintelligent AI systems.
Natural Selection Favors AIs over Humans makes an argument that doesn’t depend on how far we are away from AGI—it will apply whenever advanced AI comes around.
To respond to your specific argument that:
Chat GTP does not “understand” what it says, because it has been mainly trained with texts. The universe chat GPT inhabits is made of words, and for chat GTP words are not related to their external world referents (because chat GTP does not live in the physical world). Words, for chat GTP are linked and clustered to other words. For chat GTP Language is a self-referential reality, and it is an incredible predator in that ocean made of interlinked words, not in the bio-physical reality made of flows of energy, material cycles and gene pool competition.
Until AIs are immersed in the real world or in a virtual world designed for realism, and their goals are substantially based on that realist virtual world, AGI is not close (no matter how powerful is the core pseudo-neural technology), and existential AI-risk is too low to measure.
To make an affirmative case, there has been lots of work using ChatGPT to operate in the physical world. Google’s SayCan found that their PaLM (a language model trained just like GPT) was successfully able to operate a robot in a physical environment. The PiQA benchmark shows that language models perform worse than humans but far better than random chance in answering commonsense questions about the physical world.
Moreover, recent work has given language models additional sensory modalities so they might transcend the world of text. ChatGPT plugins allows a language model to interact with any digital software interface that can be accessed via the web or code. GPT-4 is trained on both images and text. GATO is a single network trained on text, images, robotic control, and game playing. Personally I believe that AI could pose a threat without physical embodiment, but the possibility of physical embodiment is far from distant and has seen important progress over the past several years.
Historically, people like Gary Marcus and Emily Bender have been making that argument for years, but their predictions have largely ended up incorrect. Bender and Koller’s famous paper argues that language models trained on text will never be able to understand the physical world. Their prove their argument with a prompt in Appendix A on which GPT-2 performs terribly, but if you plug their prompt or any similar styling into ChatGPT, you’ll find that it clearly perceives the physical world. Many have doubted the language model paradigm, and so far, their predictions don’t hold up well.
I think your perception is spot on. The labs that are advancing towards AGI the fastest also profess to care about safety and do research on safety. Within the alignment field, many people believe that many other people’s research agendas are useless. There are varying levels of consensus about different questions—many people are opposed to racing towards AGI, and research directions like interpretability and eliciting latent knowledge are rarely criticized—but in many cases, making progress on AI safety requires having inside view opinions about what’s important and useful.