At least re the Epoch ECI vs METR’s time horizons, I’m pretty certain they’re expressing the same thing, but in different units, and linear ELO/ECI progress in a Bradley-Terry model is exactly the same as exponential improvements.
I haven’t read much of the rest of the article, so I can’t decide whether it’s right, but I do think that there is likely a mistake in differentiating the progress rates based on ECI vs Time Horizons.
At least re the Epoch ECI vs METR’s time horizons, I’m pretty certain they’re expressing the same thing, but in different units, and linear ELO/ECI progress in a Bradley-Terry model is exactly the same as exponential improvements.
I haven’t read much of the rest of the article, so I can’t decide whether it’s right, but I do think that there is likely a mistake in differentiating the progress rates based on ECI vs Time Horizons.