Fantastic post overall, but I have a nitpick about what I think may be an important issue.
Additionally, automated coding becomes pretty good, i.e. it reliably more than doubles the output of an unassisted coder. This has the effect that much more code is generated by ML models and that more people who don’t have a coding background are able to generate decent code. An effect of this is that the overall amount of code explodes and the quality degrades because it just becomes impossible for humans to properly check and verify the code.
I strongly doubt this part, and my expectation is the opposite—human-written code is pretty mediocre, and most production systems rely on millions of lines of code, much of which is unnecessary. The reason code isn’t better is because programmer time is more expensive than the short-term benefits of improving the code. Automated tools are likely to reduce this gap, and write far tighter and simpler code. In addition, maintainability becomes far easier when automated rewrites are possible. And checking code generated by narrow AI isn’t actually as hard as checking human code for correctness when there are NAI systems which are far better than humans at writing well documented unit tests. Similarly, it can excel at generating computationally verifiably correct code—something humans can do, slowly and painfully.
Fantastic post overall, but I have a nitpick about what I think may be an important issue.
I strongly doubt this part, and my expectation is the opposite—human-written code is pretty mediocre, and most production systems rely on millions of lines of code, much of which is unnecessary. The reason code isn’t better is because programmer time is more expensive than the short-term benefits of improving the code. Automated tools are likely to reduce this gap, and write far tighter and simpler code. In addition, maintainability becomes far easier when automated rewrites are possible. And checking code generated by narrow AI isn’t actually as hard as checking human code for correctness when there are NAI systems which are far better than humans at writing well documented unit tests. Similarly, it can excel at generating computationally verifiably correct code—something humans can do, slowly and painfully.
Interesting perspective. Hadn’t thought about it this way but seems like a plausible scenario to me.