Got a q on LW about what I mean by AI reform, so sharing my reply here too:
Yeah, realize there’s not as much description as there could be. Basically a new camp in the 3 sided AGI debate (safety, boosters, ethics). Here’s a relevant excerpt:
There are already people who believe that the decisions made about this technology should not be left to those currently making them. Contrary to those who dismiss AI as a “stochastic parrot,” they take AI’s potential seriously. And contrary to classic AI safety, they recognize that there will be no purely technical “solutions” to the problems AI presents. Instead, they see governance as the essential lever. In this book, I’ll call them AI reformers.
Deep learning is incredibly powerful, but it’s being pointed at the wrong things by the wrong people for the wrong reasons. Indeed, its potential actually means it’s even more important that we change how it’s developed. The Obsoleting Project occasionally produces genuine breakthroughs—in medicine, in science—but these are the scraps that fall from the table, not the meal itself. This isn’t an accident; it’s just what markets reward.
Many people on the right and left would like to just press stop on AI. They hate what it’s done to the internet, they hate having it foisted upon them, and they hate the people making it. I get it. Really. (I miss being able to use em-dashes without drawing suspicion, and, sometimes, a sentence structure really isn’t just this, it’s also that!) But this understandable revulsion can blind us to the staggering potential of a reformed AI, where the immense power of deep learning is applied not to produce slop and madness, but to enable scientific discovery and unambiguously better lives—more AlphaFold and less chatbot psychosis.
Crucially, reformers recognize that AI’s present harms and its potential catastrophes aren’t separate problems requiring separate approaches—they’re symptoms of the same underlying dynamics: competitive pressure, concentrated power, and a staggering lack of accountability. The chatbot that encourages a teenager’s suicide and the hypothetical superintelligence that slips human control both emerge from organizations racing to deploy systems they don’t really understand. Getting serious about one means getting serious about the other. Reformers recognize that, if it were aimed in different directions, the technology could be so much better than it is. They realize that the people advancing the Obsoleting Project—not the various people looking to change or stop that project—are the ones with the real power, at least for now. They believe that a better world is possible, and so is better AI. In short, AI reform works toward the version of AI that the Obsoleting Project sells, but doesn’t actually make.
Got a q on LW about what I mean by AI reform, so sharing my reply here too:
Yeah, realize there’s not as much description as there could be. Basically a new camp in the 3 sided AGI debate (safety, boosters, ethics). Here’s a relevant excerpt:
There are already people who believe that the decisions made about this technology should not be left to those currently making them. Contrary to those who dismiss AI as a “stochastic parrot,” they take AI’s potential seriously. And contrary to classic AI safety, they recognize that there will be no purely technical “solutions” to the problems AI presents. Instead, they see governance as the essential lever. In this book, I’ll call them AI reformers.
Deep learning is incredibly powerful, but it’s being pointed at the wrong things by the wrong people for the wrong reasons. Indeed, its potential actually means it’s even more important that we change how it’s developed. The Obsoleting Project occasionally produces genuine breakthroughs—in medicine, in science—but these are the scraps that fall from the table, not the meal itself. This isn’t an accident; it’s just what markets reward.
Many people on the right and left would like to just press stop on AI. They hate what it’s done to the internet, they hate having it foisted upon them, and they hate the people making it. I get it. Really. (I miss being able to use em-dashes without drawing suspicion, and, sometimes, a sentence structure really isn’t just this, it’s also that!) But this understandable revulsion can blind us to the staggering potential of a reformed AI, where the immense power of deep learning is applied not to produce slop and madness, but to enable scientific discovery and unambiguously better lives—more AlphaFold and less chatbot psychosis.
Crucially, reformers recognize that AI’s present harms and its potential catastrophes aren’t separate problems requiring separate approaches—they’re symptoms of the same underlying dynamics: competitive pressure, concentrated power, and a staggering lack of accountability. The chatbot that encourages a teenager’s suicide and the hypothetical superintelligence that slips human control both emerge from organizations racing to deploy systems they don’t really understand. Getting serious about one means getting serious about the other. Reformers recognize that, if it were aimed in different directions, the technology could be so much better than it is. They realize that the people advancing the Obsoleting Project—not the various people looking to change or stop that project—are the ones with the real power, at least for now. They believe that a better world is possible, and so is better AI. In short, AI reform works toward the version of AI that the Obsoleting Project sells, but doesn’t actually make.