In some sense I agree with gwern that the reason ML hasn’t generated a lot of value is because people haven’t put in the work (both coding and otherwise) needed to roll it out to different domains, but (I think unlike gwern) the main inference I make from that that it wouldn’t have been hugely profitable to put in the work to create ML-based applications (or else more people would have been diverted from other coding tasks to the task of rolling out ML applications).
I mostly agree with that with the further caveat that I tend to think the low value reflects not that ML is useless but the inertia of a local optima where the gains from automation are low because so little else is automated and vice-versa (“automation as colonization wave”). This is part of why, I think, we see the broader macroeconomic trends like big tech productivity pulling away: many organizations are just too incompetent to meaningful restructure themselves or their activities to take full advantage. Software is surprisingly hard from a social and organizational point of view, and ML more so. A recent example is coronavirus/remote-work: it turns out that remote is in fact totally doable for all sorts of things people swore it couldn’t work for—at least when you have a deadly global pandemic solving the coordination problem...
As for my specific tweet, I wasn’t talking about making $$$ but just doing cool projects and research. People should be a little more imaginative about applications. Lots of people angst about how they can possibly compete with OA or GB or DM, but the reality is, as crowded as specific research topics like ‘yet another efficient Transformer variant’ may be, as soon as you add on a single qualifier like, ‘DRL for dairy herd management’ or ‘for anime’, you suddenly have the entire field to yourself. There’s a big lag between what you see on Arxiv and what’s out in the field. Even DL from 5 years ago, like CNNs, can be used for all sorts of things which they are not at present. (Making money or capturing value is, of course, an entirely different question; as fun as This Anime Does Not Exist may be, there’s not really any good way to extract money. So it’s a good thing we don’t do it for the money.)
Ah yeah, that makes sense—I agree that a lot of the reason for low commercialization is local optima, and also agree that there are lots of cool/fun applications that are left undone right now.
In some sense I agree with gwern that the reason ML hasn’t generated a lot of value is because people haven’t put in the work (both coding and otherwise) needed to roll it out to different domains, but (I think unlike gwern) the main inference I make from that that it wouldn’t have been hugely profitable to put in the work to create ML-based applications (or else more people would have been diverted from other coding tasks to the task of rolling out ML applications).
I mostly agree with that with the further caveat that I tend to think the low value reflects not that ML is useless but the inertia of a local optima where the gains from automation are low because so little else is automated and vice-versa (“automation as colonization wave”). This is part of why, I think, we see the broader macroeconomic trends like big tech productivity pulling away: many organizations are just too incompetent to meaningful restructure themselves or their activities to take full advantage. Software is surprisingly hard from a social and organizational point of view, and ML more so. A recent example is coronavirus/remote-work: it turns out that remote is in fact totally doable for all sorts of things people swore it couldn’t work for—at least when you have a deadly global pandemic solving the coordination problem...
As for my specific tweet, I wasn’t talking about making $$$ but just doing cool projects and research. People should be a little more imaginative about applications. Lots of people angst about how they can possibly compete with OA or GB or DM, but the reality is, as crowded as specific research topics like ‘yet another efficient Transformer variant’ may be, as soon as you add on a single qualifier like, ‘DRL for dairy herd management’ or ‘for anime’, you suddenly have the entire field to yourself. There’s a big lag between what you see on Arxiv and what’s out in the field. Even DL from 5 years ago, like CNNs, can be used for all sorts of things which they are not at present. (Making money or capturing value is, of course, an entirely different question; as fun as This Anime Does Not Exist may be, there’s not really any good way to extract money. So it’s a good thing we don’t do it for the money.)
Ah yeah, that makes sense—I agree that a lot of the reason for low commercialization is local optima, and also agree that there are lots of cool/fun applications that are left undone right now.