Certainly, working with human-level AIs gives you learning opportunities that are simply unavailable otherwise. It is likewise certainly true that there are things you can learn about how to fight a grizzly bear that you can learn only while fighting a grizzly bear. That does not mean that you should get into a fight with a grizzly bear when you are armed only with your bare hands. Sadly, humanity currently has no weapon other than it’s own bare hands (metaphorically speaking) in this particular fight with the grizzly bear. Acquiring an effective weapon will probably take a few more decades, and it would be foolish to engage in any combat with the bear before then.
The reason I believe it will take a few more decades to acquire the metaphorical effective weapon is that MIRI thinks it will take at least 3 more decades to figure out how to align an AI more capable than us. (They’ve been working on the problem for about 20 years now.)
In this metaphor, the grizzly bear is not really an AI, but rather the community of AI capability researchers, constantly sharing information, and the human combatant is all of humanity, which is being dragged into the fight by the capability researchers and also by those “alignment” researchers who are up for fighting with bare hands (although that of course is not how they would describe it).
Also, it is hard to know before launching an AI whether it will be human-level or ten times human-level. Were any of the designers of GPT-4 able, for example, to predict that GPT-4 would score in the 90th percentile on the Uniform Bar Exam? No they were not able to. They were as surprised as the rest of us.
Certainly, working with human-level AIs gives you learning opportunities that are simply unavailable otherwise. It is likewise certainly true that there are things you can learn about how to fight a grizzly bear that you can learn only while fighting a grizzly bear. That does not mean that you should get into a fight with a grizzly bear when you are armed only with your bare hands. Sadly, humanity currently has no weapon other than it’s own bare hands (metaphorically speaking) in this particular fight with the grizzly bear. Acquiring an effective weapon will probably take a few more decades, and it would be foolish to engage in any combat with the bear before then.
The reason I believe it will take a few more decades to acquire the metaphorical effective weapon is that MIRI thinks it will take at least 3 more decades to figure out how to align an AI more capable than us. (They’ve been working on the problem for about 20 years now.)
In this metaphor, the grizzly bear is not really an AI, but rather the community of AI capability researchers, constantly sharing information, and the human combatant is all of humanity, which is being dragged into the fight by the capability researchers and also by those “alignment” researchers who are up for fighting with bare hands (although that of course is not how they would describe it).
Also, it is hard to know before launching an AI whether it will be human-level or ten times human-level. Were any of the designers of GPT-4 able, for example, to predict that GPT-4 would score in the 90th percentile on the Uniform Bar Exam? No they were not able to. They were as surprised as the rest of us.