This is a very interesting paper and while it covers a lot of ground that I have described in the introduction, the actual cubic growth model used has a number of limitations, perhaps the most significant of which is the assumption that it considers the causal effect of an intervention to diminish over time and converge towards some inevitable state: more precisely it assumes as , where is some desirable future state and A and B are some distinct interventions at present.
Please correct me if I am wrong about this.
However, the introduction considers not just interventions fading out in terms of their ability to influence future events but often the sheer unpredictability of them. In fact, much like I did, the idea from chaos theory is cited:
.… we know
on theoretical grounds that complex systems can be extremely sensitive to initial
conditions, such that very small changes produce very large differences in later con-
ditions (Lorenz, 1963; Schuster and Just, 2006). If human societies exhibit this sort
of “chaotic” behavior with respect to features that determine the long-term effects
of our actions (to put it very roughly), then attempts to predictably influence the
far future may be insuperably stymied by our inability to measure the present state
of the world with arbitrary precision.
But the model does not consider any of these cases.
In any case, by the author’s own analysis ( which is based on a large number of assumptions), there are several scenarios where the outcome is not favorable to the longtermist.
Again, interesting work, but this modeling framework is not very persuasive to begin with (regardless of which way the final results point to).
Great points again!
I have only cursorily examined the links you’ve shared (bookmarked them for later) but I hope the central thrust of what I am saying does not depend too strongly on being closely familiar with the contents of those.
A few clarifications are in order. I am really not sure about AGI timelines and that’s why I am reluctant to attach any probability to it. For instance, the only reason I believe that there is less than 50% chance that we will have AGI in the next 50 years is because we have not seen it yet and IMO it seems rather unlikely to me that the current directions will lead us there. But that is a very weak justification. What I do know is that there has to be some radical qualitative change for artificial agents to go from excelling in narrow tasks to developing general intelligence.
That said, it may seem like nit-picking but I do want to draw the distinction between “not significant progress” and “no progress at all” towards AGI. Not only am I stating the former, I have no doubt that we have made incredible progress with algorithms in general. I am less convinced about how much those algorithms help us get closer towards an AGI. (In hindsight, it may turn out that our current deep learning approaches such as GANs contain path-breaking proto-AGI ideas /principles, but I am unable to see it that way).
If we consider a scale of 0-100 where 100 represents AGI attainment and 0 is some starting point in the 1950s, I have no clear idea whether the progress we’ve made thus far is close to 5 or 0.5 or even 0.05. I have no strong arguments to justify one or the other because I am way too uncertain about how far the final stage is.
There can also be no question with respect to the other categories of progress that you have highlighted such as compute power and infrastructure and large datasets -indeed I see these as central to the remarkable performance we have come to witness with deep learning models.
The perspective I have is that while acknowledging plenty of progress in understanding several processes in the brain such as signal propagation, mapping of specific sensory stimuli to neuronal activity, theories of how brain wiring at birth may have encoded several learning algorithms, they constitute piece-meal knowledge and they still seem quite a few strides removed the bigger question—how do we attain high level cognition, develop abstract thinking, be able to reason and solve complex mathematical problems ?
I agree that we don’t necessarily have to reproduce the exact wiring or the functional relation in order to create a general intelligence (which is why I mentioned the equivalence classes).
Finite number of genes implies finite steps/information/computation (and that is not disputable of course) but the number of potential wiring options in the brain and functional forms between input and output is exponentially large. (It is in principle, infinite, if we want to reproduce the exact function, but we both agree that that may not be necessary). Pure exploratory search may not be feasible and one may make the case that with appropriate priors and assuming some modular structure of the brain, the search space will reduce considerably, but still how much of a quantitative grip do we have on this? And how much rests on speculation?