Another nice story! I consider this to be more realistic the previous one about open-source LLMs. In fact I think this sort of ‘soft power takeover’ via persuasion is a lot more probable than most people seem to think. That said, I do think that hacking and R&D acceleration are also going to be important factors, and my main critique of this story is that it doesn’t discuss those elements and implies that they aren’t important.
In addition to building more data centers, MegaAI starts constructing highly automated factories, which will produce the components needed in the data centers. These factories are either completely or largely designed by the AI or its subsystems with minimal human involvement. While a select few humans are still essential to the construction process, they are limited in their knowledge about the development and what purpose it serves.
It would be good to insert some paragraphs, I think, about how FriendlyFace isn’t just a single model but rather is a series of models being continually improved in various ways perhaps, and how FriendlyFace itself is doing an increasing fraction of the work involved in said improvement. By the time there are new automated factories being built that humans don’t really understand, presumably basically all of the actual research is being done by FriendlyFace and presumably it’s far smarter than it was at the beginning of the story.
Another nice story! I consider this to be more realistic the previous one about open-source LLMs. In fact I think this sort of ‘soft power takeover’ via persuasion is a lot more probable than most people seem to think. That said, I do think that hacking and R&D acceleration are also going to be important factors, and my main critique of this story is that it doesn’t discuss those elements and implies that they aren’t important.
It would be good to insert some paragraphs, I think, about how FriendlyFace isn’t just a single model but rather is a series of models being continually improved in various ways perhaps, and how FriendlyFace itself is doing an increasing fraction of the work involved in said improvement. By the time there are new automated factories being built that humans don’t really understand, presumably basically all of the actual research is being done by FriendlyFace and presumably it’s far smarter than it was at the beginning of the story.
As replied on LW, we’ll discuss this. Thanks!