Dylan, this seems to overlook my two central points: that no matter how smart an ASI is, or how much compute it can do, its ability to model human biochemistry and physiology is going to be limited by the biomedical data it can access, and by the slow speed of feedback about whether any given interventions (e.g. to promote longevity) are actually working, with minimal tradeoffs and side-effects. It can’t solve a problem that it can’t model accurately.
An ASI solving very hard empirical problems (e.g. ‘curing aging’) has to have the training data needed to solve the problem. At the moment, we probably have only 0.01% of the data that we’d need. And gathering that data would require informed consent from millions of people. And I don’t imagine you’d be happy to give an ASI full autonomous power to gather such biomedical data, at scale, however it wants… that leads straight to a body horror dystopia.
I do tend to favor longer AGI/TAI timelines than many for roughly these reasons. But I don’t think you are exactly right about the AI data access trend. For one, whether or not me or Americans at large are “happy to give an ASI full autonomous power to gather such biomedical data”, China will be.
I tentatively I expect capabilities with real-world economic importance to come to some extent in the US as well, even if the most radical and transformative stuff requires further integration into the physical world for modeling. And at that point there may simply be a iterative process of greater and greater integration, as public perception improves and dependence increases. The complication here is moral backlash of some sort, which I note you’ve written about before. I agree that this is plausible, I simply wouldn’t call it probable. Things look more bi-modal to me; most likely we get the outcome I’ve described above (mild harms could still be disregarded by China), or we get a longer slow down before curing aging.
Dylan, this seems to overlook my two central points: that no matter how smart an ASI is, or how much compute it can do, its ability to model human biochemistry and physiology is going to be limited by the biomedical data it can access, and by the slow speed of feedback about whether any given interventions (e.g. to promote longevity) are actually working, with minimal tradeoffs and side-effects. It can’t solve a problem that it can’t model accurately.
An ASI solving very hard empirical problems (e.g. ‘curing aging’) has to have the training data needed to solve the problem. At the moment, we probably have only 0.01% of the data that we’d need. And gathering that data would require informed consent from millions of people. And I don’t imagine you’d be happy to give an ASI full autonomous power to gather such biomedical data, at scale, however it wants… that leads straight to a body horror dystopia.
I do tend to favor longer AGI/TAI timelines than many for roughly these reasons. But I don’t think you are exactly right about the AI data access trend. For one, whether or not me or Americans at large are “happy to give an ASI full autonomous power to gather such biomedical data”, China will be.
I tentatively I expect capabilities with real-world economic importance to come to some extent in the US as well, even if the most radical and transformative stuff requires further integration into the physical world for modeling. And at that point there may simply be a iterative process of greater and greater integration, as public perception improves and dependence increases. The complication here is moral backlash of some sort, which I note you’ve written about before. I agree that this is plausible, I simply wouldn’t call it probable. Things look more bi-modal to me; most likely we get the outcome I’ve described above (mild harms could still be disregarded by China), or we get a longer slow down before curing aging.