Agree that looking at different dimensions is more fruitful.
I also agree that size isn’t important in itself, but it might correlate with understandability.
I may overall agree with AI code understandability being closer to code than the human brain. But I think you’re maybe a bit quick here: yes, we’ll have a known design and intentional objective on some level. But this level may be quite far removed from “live” cognition. E.g. we may know a lot about developmental psychology or the effects of genes and education, but not a lot about how to modify an adult human brain in order to make specific changes. The situation could be similar from an AI system’s perspective when trying to improve itself.
Copyability does seem like a key difference that’s unlikely to change as AI systems become more advanced. However, I’m not sure if it points to rapid takeoffs as opposed to orthogonal properties. (Though it does if we’re interested in how quickly the total capacity of all AI system grows, and assume hardware overhang plus sufficiently additive capabilities between systems.) To the extent it does, the mechanism seems to be relevantly different from recursive self-improvement—more like “sudden population explosion”.
Well, I guess copyability would help with recursive self-improvement as follows: it allows to run many experiments in parallel that can be used to test the effects of marginal changes.
Agree that looking at different dimensions is more fruitful.
I also agree that size isn’t important in itself, but it might correlate with understandability.
I may overall agree with AI code understandability being closer to code than the human brain. But I think you’re maybe a bit quick here: yes, we’ll have a known design and intentional objective on some level. But this level may be quite far removed from “live” cognition. E.g. we may know a lot about developmental psychology or the effects of genes and education, but not a lot about how to modify an adult human brain in order to make specific changes. The situation could be similar from an AI system’s perspective when trying to improve itself.
Copyability does seem like a key difference that’s unlikely to change as AI systems become more advanced. However, I’m not sure if it points to rapid takeoffs as opposed to orthogonal properties. (Though it does if we’re interested in how quickly the total capacity of all AI system grows, and assume hardware overhang plus sufficiently additive capabilities between systems.) To the extent it does, the mechanism seems to be relevantly different from recursive self-improvement—more like “sudden population explosion”.
Well, I guess copyability would help with recursive self-improvement as follows: it allows to run many experiments in parallel that can be used to test the effects of marginal changes.