There are severe problems with the METR time horizons graph, so severe the graph should just be thrown out. You can read about those problems in detail here and here. For instance:
“most of the longer tasks” don’t even have baselines; their time-horizons are just being guesstimated.
Data that is guessed rather than measured isn’t really data. When you look at those dots on the graph and the dotted trend line, you naturally assume that the dots and the line are based on some kind of actual data that someone measured. You don’t tend to think that someone just guessed the “data” and put it on a graph.
But as bad as this is, it might not even the worst problem with the METR graph. There are multiple other problems that might be worse.
There are severe problems with the METR time horizons graph, so severe the graph should just be thrown out. You can read about those problems in detail here and here. For instance:
Data that is guessed rather than measured isn’t really data. When you look at those dots on the graph and the dotted trend line, you naturally assume that the dots and the line are based on some kind of actual data that someone measured. You don’t tend to think that someone just guessed the “data” and put it on a graph.
But as bad as this is, it might not even the worst problem with the METR graph. There are multiple other problems that might be worse.