I think this type of belief comes from a flattening understanding of the difficulty of doing things: it’s assumed that because doing well on a math olympiad is hard, and that curing death is hard, that if you can make an AI do the former it will soon be able to do the latter. But in fact, curing death is so much more difficult to do than a math olympiad that it breaks the scale.
You can also see this in the casual conflation of things like “cure cancer” and “cure death”. The latter is many, many, many orders of magnitude more difficult than the former: claiming that the latter would occur at the same time as the former is an incredibly extraordinary claim, and it requires commensurate evidence to back it up.
The chief argument in favour of this is “recursive self improvement”, but intelligence is not a magic spell that you can just dial up to infinity. There are limits in the forms of empirical knowledge, real-world resources and computational complexity. Certainly current day AI trends seems to be limited by scaling laws that would be impractical a pretty fucking long way from god-like intelligence.
I fully agree with this post.
I think this type of belief comes from a flattening understanding of the difficulty of doing things: it’s assumed that because doing well on a math olympiad is hard, and that curing death is hard, that if you can make an AI do the former it will soon be able to do the latter. But in fact, curing death is so much more difficult to do than a math olympiad that it breaks the scale.
You can also see this in the casual conflation of things like “cure cancer” and “cure death”. The latter is many, many, many orders of magnitude more difficult than the former: claiming that the latter would occur at the same time as the former is an incredibly extraordinary claim, and it requires commensurate evidence to back it up.
The chief argument in favour of this is “recursive self improvement”, but intelligence is not a magic spell that you can just dial up to infinity. There are limits in the forms of empirical knowledge, real-world resources and computational complexity. Certainly current day AI trends seems to be limited by scaling laws that would be impractical a pretty fucking long way from god-like intelligence.