your brain isfully accessible to your enemy for your entire lifetime of plotting
This sounds like you are assuming that mechanistic interpretability has somehow been solved. We are nowhere near on track for that to happen in time!
Also, re “it will be borderline omnipotent”: this is not required for doom. ~Human level AI hackers copied a million times, and sped up a million times, could destroy civilisation.
It doesn’t seem to me that titotal is assuming MI is solved; having direct access to the brain doesn’t give you full insight into someone’s thoughts either, because neuroscience is basically a pile of unsolved problems with growing-but-still-very-incomplete-picture of low-level and high-level details. We don’t even have a consensus on how memory is physically implemented.
Nonetheless, if you had a bunch of invasive probes feeding you gigabytes/sec of live data from the brain of the genius general of the opposing army, it would be extremely likely to be useful information.
A really interesting thing is that, at the moment, this appears in practice to be a very-asymmetrical advantage. The high-level reasoning processes that GPT-4 implements don’t seem to be able to introspect about fine-grained details, like “how many tokens are in a given string”. The information is obviously and straightforwardly part of the model, but absent external help the model doesn’t seem to bridge the gap between low-level implementation details and high-level reasoning abilities—like us.
Ok, so the “brain” is fully accessible, but that is near useless with the level of interpretability we have. We know way more human neuroscience by comparison. It’s hard to grasp just how large these AI models are. They have of the order of a trillion dimensions. Try plotting that out in Wolfram Alpha or Matlab..
It should be scary in itself that we don’t even know what these models can do ahead of time. It is an active area of scientific investigation to discover their true capabilities, after the fact of their creation.
This sounds like you are assuming that mechanistic interpretability has somehow been solved. We are nowhere near on track for that to happen in time!
Also, re “it will be borderline omnipotent”: this is not required for doom. ~Human level AI hackers copied a million times, and sped up a million times, could destroy civilisation.
It doesn’t seem to me that titotal is assuming MI is solved; having direct access to the brain doesn’t give you full insight into someone’s thoughts either, because neuroscience is basically a pile of unsolved problems with growing-but-still-very-incomplete-picture of low-level and high-level details. We don’t even have a consensus on how memory is physically implemented.
Nonetheless, if you had a bunch of invasive probes feeding you gigabytes/sec of live data from the brain of the genius general of the opposing army, it would be extremely likely to be useful information.
A really interesting thing is that, at the moment, this appears in practice to be a very-asymmetrical advantage. The high-level reasoning processes that GPT-4 implements don’t seem to be able to introspect about fine-grained details, like “how many tokens are in a given string”. The information is obviously and straightforwardly part of the model, but absent external help the model doesn’t seem to bridge the gap between low-level implementation details and high-level reasoning abilities—like us.
Ok, so the “brain” is fully accessible, but that is near useless with the level of interpretability we have. We know way more human neuroscience by comparison. It’s hard to grasp just how large these AI models are. They have of the order of a trillion dimensions. Try plotting that out in Wolfram Alpha or Matlab..
It should be scary in itself that we don’t even know what these models can do ahead of time. It is an active area of scientific investigation to discover their true capabilities, after the fact of their creation.