Section 4 is completely over my head I have to confess.
Edit: But the abstract gives me what I wanted to know :) : “To quantify the capabilities of AI systems in terms of human capabilities, we propose a new metric: 50%-task-completion time horizon. This is the time humans typically take to complete tasks that AI models can complete with 50% success rate”
Section 4 is completely over my head I have to confess.
Edit: But the abstract gives me what I wanted to know :) : “To quantify the capabilities of AI systems in terms of human capabilities, we propose a new metric: 50%-task-completion time horizon. This is the time humans typically take to complete tasks that AI models can complete with 50% success rate”