Hey Steve, thanks for those thoughts! I think I’m not more qualified than the wikipedia community to argue for or against Moore’s law, that’s why I just quoted them. So can’t give more thoughts on that unfortunately.
But even if Moore’s law would continue forever, I think that the data argument would kick in. If we have infinite compute but limited information to learn from, that’s still a limited model. Applying infinite compute to the MNIST dataset will give you a model that won’t be much better than the latest Kaggle competitor on that dataset.
So then we end up again at the more hand-wavy arguments for limits to the growth of knowledge and predictability of our world in general. Would be curious where I’m losing you there.
Hey Steve, thanks for those thoughts! I think I’m not more qualified than the wikipedia community to argue for or against Moore’s law, that’s why I just quoted them. So can’t give more thoughts on that unfortunately.
But even if Moore’s law would continue forever, I think that the data argument would kick in. If we have infinite compute but limited information to learn from, that’s still a limited model. Applying infinite compute to the MNIST dataset will give you a model that won’t be much better than the latest Kaggle competitor on that dataset.
So then we end up again at the more hand-wavy arguments for limits to the growth of knowledge and predictability of our world in general. Would be curious where I’m losing you there.