ML optimizing ML inputs: reduced data centre energy cost, reduced cost of acquiring training data, supposedly improved semiconductor designs.
ML aiding ML researchers. e.g. >3% of new Google code is now auto-suggested without amendment.
ML replacing parts of ML research. Nothing too splashy but steady progress: automatic data cleaning and feature engineering, autodiff (and symbolic differentiation!), meta-learning network components (activation functions, optimizers, …), neural architecture search.
Classic direct recursion. Self-play (AlphaGo) is the most striking example but it doesn’t generalise, so far. Purported examples with unclear practical significance: Algorithm Distillation and models finetuned on their ownoutput.[1]
The proliferation of crappy bootleg LLaMA finetunes using GPT as training data (and collapsing when out of distribution) makes me a bit cooler about these results in hindsight.
Break self-improvement into four:
ML optimizing ML inputs: reduced data centre energy cost, reduced cost of acquiring training data, supposedly improved semiconductor designs.
ML aiding ML researchers. e.g. >3% of new Google code is now auto-suggested without amendment.
ML replacing parts of ML research. Nothing too splashy but steady progress: automatic data cleaning and feature engineering, autodiff (and symbolic differentiation!), meta-learning network components (activation functions, optimizers, …), neural architecture search.
Classic direct recursion. Self-play (AlphaGo) is the most striking example but it doesn’t generalise, so far. Purported examples with unclear practical significance: Algorithm Distillation and models finetuned on their own output.[1]
See also this list
Treachery:
https://arxiv.org/abs/2102.07716
https://lukemuehlhauser.com/treacherous-turns-in-the-wild/
The proliferation of crappy bootleg LLaMA finetunes using GPT as training data (and collapsing when out of distribution) makes me a bit cooler about these results in hindsight.
Thanks, really helpful!
Buckman’s examples are not central to what you want but worth reading: https://jacobbuckman.com/2022-09-07-recursively-self-improving-ai-is-already-here/