Thanks for sharing, Matrice! Very relevant, although I do not think anything there undermines the points made by Anson. Toby concludes by saying “And of course it is also important to know how much any of this generalises to other suites of tasks”. I expect the half-life to be shorter for broader tasks which track economic value more closely.
Toby Ord has shown a simple mathematical model can model the performance of AI agents on longer-duration tasks, such that such that each additional nine of reliability requires two more years.
Thanks for sharing, Matrice! Very relevant, although I do not think anything there undermines the points made by Anson. Toby concludes by saying “And of course it is also important to know how much any of this generalises to other suites of tasks”. I expect the half-life to be shorter for broader tasks which track economic value more closely.