Benchmarks will become useless due to eval awareness¹
I think they will become over saturated and disconnected from real world usage such that they will be pretty useless, but I don’t think eval awareness is what will make them lose their value
I think they will become over saturated and disconnected from real world usage such that they will be pretty useless, but I don’t think eval awareness is what will make them lose their value