Benchmarks will become useless due to eval awareness¹
Some yes, some no. There are a great many benchmarks measuring a great many things. More than eval awareness, I’d be worried they’ll become useless due to reward hacking.
Some yes, some no. There are a great many benchmarks measuring a great many things. More than eval awareness, I’d be worried they’ll become useless due to reward hacking.